{"id":26304,"date":"2023-03-05T12:50:16","date_gmt":"2023-03-05T12:50:16","guid":{"rendered":"http:\/\/localhost\/helpold\/?p=26304"},"modified":"2025-01-02T14:12:41","modified_gmt":"2025-01-02T10:42:41","slug":"rclone-migration","status":"publish","type":"post","link":"https:\/\/www.arvancloud.ir\/help\/en\/rclone-migration\/","title":{"rendered":"Migrating to ArvanCloud Object Storage Using Rclone"},"content":{"rendered":"<p><a href=\"https:\/\/rclone.org\/\" rel=\"nofollow noopener\" target=\"_blank\">Rclone<\/a> is an effective alternative to rsync. Designed to communicate with cloud storages that are compatible with S3, this tool helps you migrate and transfer data from one bucket to another, even if the buckets are hosted on different service providers.<br \/>\nUsing the open-source Rclone tool, you will be able to back up data from one server or a cloud storage and store it on another server, object storage or cloud. This software offers several options, including bucket management, mounting virtual drives via a bucket, managing files, and locating duplicate files in addition to backups.<br \/>\nBy adding ArvanCloud Object Storage to the list of Rclone providers, using it for ArvanCloud users will be extremely easy and optimal.<br \/>\nThis guide will introduce and examine the method of transferring data using Rclone.<\/p>\n<h2>Installing Rclone<\/h2>\n<p>Depending on your operating system, you will need to perform several steps.<\/p>\n<h3>Installing Rclone on Windows<\/h3>\n<p>To install this software on a Windows operating system, you simply must download it from the <a href=\"https:\/\/rclone.org\/downloads\/\" rel=\"nofollow noopener\" target=\"_blank\">Rclone website<\/a>. After that, you should execute the downloaded file via the appropriate path and through CLI.<\/p>\n<h3>Installing Rclone on Linux<\/h3>\n<p>You will need to follow the steps below to install Rclone on Unix-based operating systems:<\/p>\n<ul>\n<li>Connect to your server or system as the root user.<\/li>\n<li>Update APT:\n<pre dir=\"ltr\"><code>apt update &amp;&amp; apt upgrade -y<\/code><\/pre>\n<\/li>\n<li>By running the following commands respectively, download and install Rclone:\n<pre class=\"wysiwyg-text-align-left\" dir=\"ltr\"><code>git clone https:\/\/github.com\/rclone\/rclone.git\r\ncd rclone\r\ngo build<\/code><\/pre>\n<p class=\"wysiwyg-text-align-left\" dir=\"ltr\">Before running these commands, you must install git and go1.17 on your operating system. Do this by running the following commands:<\/p>\n<pre class=\"wysiwyg-text-align-left\" dir=\"ltr\"><code>apt-get install git\r\ntar -C \/usr\/local -xzf go1.17.5.linux-amd64.tar.gz\r\nwget https:\/\/dl.google.com\/go\/go1.17.7.linux-amd64.tar.gz\r\nsudo tar -xvf go1.17.7.linux-amd64.tar.gz\r\nsudo mv go \/usr\/local\r\nexport GOROOT=\/usr\/local\/go\r\nexport GOPATH=$HOME\/Projects\/Proj1\r\nexport PATH=$GOPATH\/bin:$GOROOT\/bin:$PATH<\/code><\/pre>\n<\/li>\n<li>Once Rclone is installed, type the following command to verify that the version and the installation are correct:\n<pre class=\"wysiwyg-text-align-left\" dir=\"ltr\"><code>.\/rclone version<\/code><\/pre>\n<\/li>\n<\/ul>\n<p>To make it easier to run the .\/rclone commands, you can define an alias to rclone in ~\/.bashrc.<\/p>\n<h2>Rclone Configuration and Settings<\/h2>\n<p>Use the following command to start configuring Rclone:<\/p>\n<pre dir=\"ltr\"><code>\u00a0rclone config<\/code><\/pre>\n<p>You will see the following output in case you have not already set up Rclone on your system:<\/p>\n<p><img fetchpriority=\"high\" decoding=\"async\" class=\"size-full wp-image-25565 aligncenter\" src=\"https:\/\/www.arvancloud.ir\/help\/wp-content\/uploads\/2023\/02\/config.png\" alt=\"\" width=\"787\" height=\"158\" \/><\/p>\n<p>When you have already configured Rclone, a different output will be displayed, but the new configuration will not affect the next steps.<\/p>\n<p>Type n to create a new remote control or configuration. Then you will be prompted to enter the name of your choice. In this case, we have chosen ArvanCloud.<\/p>\n<p>The following output will be shown:<\/p>\n<p><img decoding=\"async\" class=\"size-full wp-image-25568 aligncenter\" src=\"https:\/\/www.arvancloud.ir\/help\/wp-content\/uploads\/2023\/02\/providers.png\" alt=\"\" width=\"781\" height=\"739\" \/><\/p>\n<p>Type S3 and press Enter. Here&#8217;s what the output looks like:<\/p>\n<p><img decoding=\"async\" class=\"size-full wp-image-25570 aligncenter\" src=\"https:\/\/www.arvancloud.ir\/help\/wp-content\/uploads\/2023\/02\/arvan.png\" alt=\"\" width=\"528\" height=\"773\" \/><br \/>\nType ArvanCloud and press Enter to confirm Arvan Cloud as the S3 provider. The output below will be displayed:<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-25572 aligncenter\" src=\"https:\/\/www.arvancloud.ir\/help\/wp-content\/uploads\/2023\/02\/false.png\" alt=\"\" width=\"765\" height=\"166\" \/><br \/>\nType False and press Enter as to enter the credentials in the next step. You will be shown an output similar to the image below at this point:<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-25574 aligncenter\" src=\"https:\/\/www.arvancloud.ir\/help\/wp-content\/uploads\/2023\/02\/acces.png\" alt=\"\" width=\"471\" height=\"94\" \/><br \/>\nCopy your access key from the panel and enter it.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-25576 aligncenter\" src=\"https:\/\/www.arvancloud.ir\/help\/wp-content\/uploads\/2023\/02\/secret.png\" alt=\"\" width=\"465\" height=\"88\" \/><br \/>\nCopy your secret key from the panel and enter it.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-25578 aligncenter\" src=\"https:\/\/www.arvancloud.ir\/help\/wp-content\/uploads\/2023\/02\/endpoint.png\" alt=\"\" width=\"513\" height=\"155\" \/><br \/>\nType 1 for Asiatech and 2 for Shahriar, and then press Enter.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-25580 aligncenter\" src=\"https:\/\/www.arvancloud.ir\/help\/wp-content\/uploads\/2023\/02\/location.png\" alt=\"\" width=\"452\" height=\"163\" \/><br \/>\nEnter the endpoint value from Tabriz (2) and Tehran (1). The output will be like the one in the image:<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-25582 aligncenter\" src=\"https:\/\/www.arvancloud.ir\/help\/wp-content\/uploads\/2023\/02\/acl.png\" alt=\"\" width=\"776\" height=\"463\" \/><br \/>\nChoose the ACL you wish to use and type it in. For example, we will choose private here. Then enter n to see a summary of the configuration:<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-25584 aligncenter\" src=\"https:\/\/www.arvancloud.ir\/help\/wp-content\/uploads\/2023\/02\/no.png\" alt=\"\" width=\"364\" height=\"321\" \/><br \/>\nType y to confirm the configuration. This will give you the following output:<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-25586 aligncenter\" src=\"https:\/\/www.arvancloud.ir\/help\/wp-content\/uploads\/2023\/02\/final.png\" alt=\"\" width=\"297\" height=\"187\" \/><br \/>\nIn the end, type q to exit the configuration.<\/p>\n<p><strong>Note<\/strong>: When you want to transfer data from another provider&#8217;s bucket to the provider you have just configured, simply repeat the steps to set up a new remote. You can easily select the desired provider in steps 2 and 3.<\/p>\n<h2>Transferring Data<\/h2>\n<p>There are two commands to transfer data between two buckets:<\/p>\n<ol>\n<li>Use the copy command to copy the data from the source to the destination:\n<pre dir=\"ltr\"><code>rclone copy --progress &lt;SOURCE_BACKEND&gt;:&lt;SOURCE_PATH&gt; &lt;DEST_BACKEND&gt;:&lt;DEST_PATH&gt;<\/code><\/pre>\n<p>For example, the following command will copy the data from the my-first-bucket bucket in the ArvanCloud remote, which we set earlier, to the my-second-bucket bucket in the same remote. Using the &#8211;progress flag, we can track the progress of this process.<\/p>\n<pre dir=\"ltr\"><code>rclone copy --progress ArvanCloud:my-first-bucket ArvanCloud:my-second-bucket<\/code><\/pre>\n<\/li>\n<li>The sync command will copy the data from the source to the destination, except that it will delete the objects that are in the destination bucket but not in the source bucket:\n<pre dir=\"ltr\"><code>rclone sync --progress &lt;SOURCE_BACKEND&gt;:&lt;SOURCE_PATH&gt; &lt;DEST_BACKEND&gt;:&lt;DEST_PATH&gt;<\/code><\/pre>\n<p dir=\"ltr\">The following command, for example, will copy the data from the my-first-bucket bucket in the ArvanCloud remote we set previously to the my-third-bucket in the AmazonAws remote. Any data that is in my-third-bucket and not in my-first-bucket will be removed.<\/p>\n<\/li>\n<\/ol>\n<p>You can find more information and an overview of the available commands in the <a href=\"https:\/\/rclone.org\/s3\/\" rel=\"nofollow noopener\" target=\"_blank\">official Rclone documentation<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Rclone is an effective alternative to rsync. Designed to communicate with cloud storages that are compatible with S3, this tool helps you migrate and transfer data from one bucket to another, even if the buckets are hosted on different service providers. Using the open-source Rclone tool, you will be able to back up data from [&hellip;]<\/p>\n","protected":false},"author":79,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"r1_post_views_count":"773","feedback_yes":"0","feedback_no":"1","ads_id_sidebar":"","ads_id_content":"","footnotes":""},"categories":[247,253],"tags":[],"class_list":["post-26304","post","type-post","status-publish","format-standard","hentry","category-cloud-storage","category-backup-migration"],"lang":"en","translations":{"en":26304},"pll_sync_post":[],"_links":{"self":[{"href":"https:\/\/www.arvancloud.ir\/help\/wp-json\/wp\/v2\/posts\/26304","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.arvancloud.ir\/help\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.arvancloud.ir\/help\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.arvancloud.ir\/help\/wp-json\/wp\/v2\/users\/79"}],"replies":[{"embeddable":true,"href":"https:\/\/www.arvancloud.ir\/help\/wp-json\/wp\/v2\/comments?post=26304"}],"version-history":[{"count":4,"href":"https:\/\/www.arvancloud.ir\/help\/wp-json\/wp\/v2\/posts\/26304\/revisions"}],"predecessor-version":[{"id":28286,"href":"https:\/\/www.arvancloud.ir\/help\/wp-json\/wp\/v2\/posts\/26304\/revisions\/28286"}],"wp:attachment":[{"href":"https:\/\/www.arvancloud.ir\/help\/wp-json\/wp\/v2\/media?parent=26304"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.arvancloud.ir\/help\/wp-json\/wp\/v2\/categories?post=26304"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.arvancloud.ir\/help\/wp-json\/wp\/v2\/tags?post=26304"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}