Categories
AWS S3

Backup solution with AWS S3 Sync

Recently I had the opportunity to upgrade a windows computer. This computer was running Windows 8.1 and was not performing well. CPU and memory settings were sitting between 60-90%. I also noticed that using Google Chrome made things worse. After my initial analysis, I decided to upgrade to Windows 10 and start from scratch. This computer has 8GB of memory and a 1TB hard drive disk.

Before I started the upgrade process, I wanted to keep a backup of 1500 pictures and videos. In this post, I want to share how I was able to use AWS S3 Sync command to backup these files. Ready? Let’s get started.

First, make sure you have the AWS CLI installed. Follow this article for more details. Let’s create a new IAM user with S3 permissions. Go to the aws management console, search for IAM and create a new user. Enter a user name and make sure you check Programmatic access. We will use the access key id and secret access key to setup our CLI locally. Now click on Next: Permissions.

Select Attach existing policies directly and type s3 in the filter policies. Select AmazonS3FullAccess.

Click on Next: Tags and add any tags if needed. Click on Next: Review and finally create user. Make sure you download the csv file that contains the access key id and secret access key. Another option is to copy these values from the confirmation page.

With the access key id and secret access key, we can setup the CLI locally. Open up a command prompt or terminal and type “aws –version”.

If you see a similar output to the above image, your CLI is setup correctly. In the same terminal window, type aws configure and enter access key id, secret access key, default region name, and output format. For this article, I’m using us-east-1 for my default region and json for my ouput.

With the CLI setup correctly, we can create a new bucket. Run this command in your terminal, “aws s3api create-bucket –bucket [yourbucketname] –region us-east-1”. Let’s verify our S3 bucket was created successfully with this command, “aws s3 ls”.

With our s3 bucket in place, we can run the sync command to copy our local images and videos to this new bucket. Run this command “aws s3 sync . s3://agileraymond-s3”. The sync command will copy any files in my current directory to the s3 bucket named agileraymond-s3. Since I needed to copy 1500 images and videos, I ran multiple sync commands in parallel using this format.

aws s3 sync . s3://agileraymond-s3/Pictures

aws s3 sync . s3://agileraymond-s3/Videos

aws s3 sync . s3://agileraymond-s3/Downloads

Don’t forget to be in the right directory for this to work. After a couple of hours, I was able to verify all files made it to my bucket. I continued with the Windows 10 upgrade and after the initial setup, I was able to re-install the aws cli and ran the aws sync with a different format. This time I needed to copy files from my s3 bucket to my local pc. I used “aws sync s3://agileraymond-s3/Pictures ./” to copy Pictures to my local Pictures folder.

In summary, I was able to backup 1500 images and videos with the aws sync command. It was a very simple process in my opinion. If you want to explore different options with the sync command, take a look at the documentation page.