rsync is the perfect utility for keeping two local or remote directories synchronized with each other. It works efficiently by only transferring the differences between the files in two directories. In this guide, we’ll show you the various commands you can use with rsync to transfer, backup, and synchronize your files.
rsync has become a native utility found on Linux systems. Every distribution supports it, and it’s even been ported to Windows, MacOS, and BSD. The usefulness of rsync cannot be overstated.
Although keeping two directories in sync with each other seems like a straightforward task, the amount of options you can pass to rsync is staggering. It’s touted as the ideal backup and file transfer tool by many, and for good reason. With a little knowledge of how it works, you’ll soon be using it for all of your file transfer tasks.
The most basic rsync command that you’ll want to familiarize yourself with uses the following syntax:
With this command, rsync will synchronize the contents of directory1 with directory2. It will do exactly the same thing as the cp (copy) command, except it checks files for differences and only transfers the changes, making the total size of the transfer smaller.
Let’s talk about the options passed to rsync. We included -av in our command because they’re two of the most common options to have in an rsync command. So common, in fact, it’s likely you’ll be using them almost every time you run rsync. Here’s what they do:
-a = Archive mode. This does a few things, basically wrapping up the most common options into a single flag. It makes the transfer recursive and tells rsync to transfer symbolic links, device files, and special files. It also preserves the permissions, owner, group, and modified time of each file. In other words, the files are kept identical (right down to the permission settings and mtime) from the source directory to the destination directory.-v = Increase verbosity. Without this option, rsync won’t produce any output unless it encounters some kind of error. When you have this option enabled, rsync will display which file it’s currently transferring and gives you the opportunity to see a list of everything that’s been transferred at the end.The other thing to note about our command syntax above is the trailing slash on directory names. Without the trailing slashes, rsync would create directory1 inside of directory2. This may be fine in some situations, but our goal here is to keep the file structure of the two directories exactly the same.
Let’s take a look at some other rsync options you’re likely to need.
--delete = Delete extraneous files from the destination directory. With this option, rsync will synchronize two directories and if the destination directory contains a file that isn’t present in the source directory, it will delete it.Notice that some_file.txt was transferred from the source directory to the destination like in our previous example, but not before some_other_file.txt was deleted from the destination directory, since it isn’t part of the source directory’s contents.
-n = Dry run. Running an rsync command with the --delete option can be a little risky. You may end up deleting something that you didn’t realize was only present in the destination directory. The -n option allows you to execute your rsync command without actually making any changes. This gives you a chance to examine the output and make sure you’re not about to make any undesirable changes.I usually backup between two Ubuntu machines, but I now want to use rsync to backup my Ubuntu server with a disk residing on a Mac. But I do not get it to work properly. The first time all is well, but apparently there is a problem with special characters between the mac and ubuntu machine, since every time I run the rsync operation after that. Rsync -rtb –backup-dir=”backup $ (date +%Y-%m-%d)” –delete /Users/tnelson/Desktop /Volumes/DocsBackup In the above command, the -b option has been added to -r and -t, creating the -rtb option. We’ve also used the –backup-dir command to specify the name of the backup folder to use. Rsync is a command-line tool built-in to Mac OS X that allows you to synchronize files between two folders on a machine or between two machines on a network. Rsync is an incredibly useful tool that has been used by Unix administrators for many years. Aug 14, 2017 Automate rsync backup. Now the best benefit and powerful part is to automate backup. You can achieve this feature by creating cron job on Linux OS. Run ‘crontab -e’ to create cron jobs. It will open an empty file where you can configure the command that you want to run at a desired time. We look at the best backup software here, including Acronis, ChronoSync, Carbon Copy Cloner, Carbonite and SuperDuper. As with Time Machine you can use the backup drive - or clone - to recover your.
This time, adding -n to our rsync command allows us to see that our some_file.txt file is going to be transferred and the some_other_file.txtIskysoft imedia converter deluxe for mac catalina bay. file will be deleted. rsync reminds us that this is just a dry run in the final line of the output. Apart from the “DRY RUN” text, this is the exact output rsync would give us if we took away the -n flag.
If you have files or directories that you want rsync to skip over, there are a couple of options for telling rsync what to exclude.
--exclude = This option allows you to specify what to exclude from within the rsync command.--exclude-from = With this option, you can tell rsync to refer to a file that contains a list of all the files to exclude.To exclude a specific file with --exclude:
If you need to exclude more than one file, just use additional --exclude flags:
Note that you can also specify patterns by using wildcards:
The other option is to create a plaintext file where you list each file you wish to exclude on a separate line. Refer to this file with the --exclude-from option:
rsync detects changes between files by inspecting the modified time and size of a file. That’s how it’s able to quickly parse tons of files and know if there are changes that need to be transferred. Although this should suffice for nearly every possible scenario, you can also make rsync perform a checksum on each file to ensure that your files are 1:1 exact copies.
I personally find this option useful when I want to check for data degradation on one of my backups. If silent data corruption occurs to a file, checksuming your files with rsync should detect the problem.
-c = Transfer files based on checksum differences, not size and modified time.Requiring rsync to checksum a bunch of files is both CPU intensive and time consuming, so don’t expect a fast rsync if you’re using this on a fair amount of files.
If you’re using the -c option, please be sure to pair it with -n (dry run) as well. If rsync detects any differences, this will allow you an opportunity to check the condition of both files to see why the checksums don’t match. If one is corrupted, you’ll know which version to keep better than rsync will.
rsync can use SSH to synchronize a local directory with a remote directory (or vice versa). The command syntax is the same as we’ve seen above, except that we need to include another option.
-e ssh = Use SSH as remote shell.This command will synchronize directory1 on our local system with directory2 on a remote system, using SSH to authenticate and subsequently the SSH tunnel to transfer all of the data. Note that we’re still using the -n (dry run) flag in this example. It’s always best to see what changes we’re making first.
Side note: If you’re making a script that uses rsync over SSH, it’s a good idea to configure RSA keys for logging into SSH without a password. Otherwise, your script will stop to prompt you for a password each time.
There are a couple of options that are useful specifically when working with remote file transfers.
-z = Use compression. With this option, rsync will compress data before sending it over the network. It knows not to compress files that already use their own compression, like JPEG files. This option will put some additional stress on your CPU, but could save you a lot of bandwidth and time if you’re doing a big transfer.-P = This flag combines two common options, --partial and --progress. The former allows a transfer to be interrupted and resumed where it left off, and the latter will show you the current progress of whichever file is being transferred.That’s about it for everyday rsync options. The vast majority of your rsync needs should be satisfied by basing your commands on the examples we’ve shown you. Still, we’ve barely scraped the surface of rsync. Check out the man page for rsync if you’re interested in further customization.