Amer's Corner on the Interwebs

Blog

😎 Super-Efficient Data Organization and Backup

Date: 20 Sep 2021


I like having my data very organized. It gives me a sense of order. I have tried different approaches. I would like to share my current approach of how I organize my stuff and how I back it up.

Organization

Currently, I have around 885 GB of files in my laptop. Backing up such an amount of data would take a lot of time and not all of it is unique/important. So, one of my previous approaches was to categorize the data into Core and Non-core. Core would contain all the important files, which I did not wanna lose. Non-core would contain all the other files like movies and series. I would only have to backup the Core files. This approach worked well.

Later I realized that this method was a little binary. There was a middle ground necessary: a category of files which are not unique, but, if deleted, would be difficult to collect again. Like my collection of documentaries.

So I started following a new method. I divided files into three categories:

The idea is that 1 should the most important, unique data that will be backed up regularly.

1 is the smallest folder. 2 is bigger. And 3 is the biggest.

I try to clean up my common folders like Desktop, Downloads, Pictures, etc, often.

I also dump all the data from my phone into my laptop regularly and try to keep it sorted.

Backup

I use a 1 TB hard disk for backup. Before the categorization, it used to take so much time, as all the data was backed up. Now, I only need to backup folder 1.

I heard about the software rsync. Tried using it. But, couldn't completely figure it out. Then, luckily, I found Grsync, a graphical program for rsync. This program increased my efficiency so much! It feels magical!

If I clean up the files in my laptop and I want the deleted files to also be deleted from my backup disk, I just add --delete in the Additional options.

Once in a while, I use the 'Always checksum' option, which takes longer to process, but makes sure that the backup is not corrupt, by comparing the hashes.

Lastly, one of the most amazing thing about rsync is that if you backup a huge file, change a small part of the original file and then backup again, it will not copy the whole file. Instead, it identifies the portion that was changed and copies only that part. It's such an intelligent design 🤩!

Here's the algorithms rsync uses: https://en.wikipedia.org/wiki/Rsync#Algorithm