Tuesday 28/10/2008
The rsync(1) command can be used perform bulk file transfer as well as incremental updates on previously synchronized filesystems. The rsync command allows the user to tunnel via ssh(1) or rsh(1) with ssh being the default. Directories can be recursively copied using the -a option and the user and hostname specified using typicall ssh syntax (user@host:pathname)
The pathname given to exclude-from= lists one pathname per line of the paths not to include in the local synchronization. This allows local directories to be kept (as otherwise the --delete would remove them if they're not in the source location). The --exclude= option is the same, except only a single pathname to exclude can be specified.
The rync(1) options for source and destination follow the same syntax as for ssh, i.e., user@host:pathname. The --dry-run (or -n) options can be specified to show what rsync(1) would do without actually doing it.
bash $ rsync -auv --exclude-from=/tmp/rsync.exclude --delete srchost:/tmp/foo /tmp/foo/rsync
The format of the exlude list file is one entry per line, which are a patterns to exclude. Each entry can use wildcards (*). The following example excludes the directories BAR and FOO and their contents from the prior rsync(1) command:
bash $ /tmp/rsync.exclude rsync/BAR/** rsync/FOO/**
The rsync can be executed using ssh(1) and a private/public key pair. To do this requries the generation of ssh public/private key pair without passphrase and further these need to be installed into the appriate users (both remote and local). In the following scenario, a user called rsyncdood on srchost will rsync data to rsyncdood on remotehost. The following will steps will enable passphrase-less rsync operation between the two hosts
'''
{{.images/stock/warn.png}} Assumption: A user ''rsyncdood'' exists on both hosts involed in the rsync
==== 2.1 Generate the passphrase-less key pairs ====
The Openssh package provides [[man?ssh-keygen|ssh-keygen(1)]] (see [[ssh]] for more details) which will generate a private key pair. The following command will geneate a 2048 bit RSA key, placing the key pairs into the location ''/home/ryncdood/.ssh'', with the private key being called ''rsync-key'' and public key being called ''rsync-key.pub'':
'''
bash $ ssh-keygen -t rsa -b 2048 -f /home/rsyncdood/.ssh/rsync-key
Generating public/private rsa key pair.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/rsyncdood/.ssh/rsync-key.
Your public key has been saved in /home/rsyncdood/.ssh/rsync-key.
The key fingerprint is:
73:a0:1c:2b:d7:0e:ea:37:74:6f:78:3f:22:80:e3:7f foo@bobcat
The key's randomart image is:
+--[ RSA 2048]----+
| .o.o + o.. |
+-----------------+
The ssh key pair generated in #2.1 needs to be installed to the relevant users on the relevant hosts. Continuing the example, the keys are installed to user rsyncdood on srchost and remotehost, into their .ssh directories. It is important that the permissions are correct on the .ssh directory (0700) and on the private key files (0600):
rsyncood@srchost bash ~ [1:0]$ ls -laF ~/.ssh total 24 drwx------ 2 rsyncdood rsyncdood 4096 2009-05-15 11:44 ./ drwxr-xr-x 16 rsyncdood rsyncdood 4096 2009-05-15 12:36 ../ -rw------- 1 rsyncdood rsyncdood 790 2009-05-15 11:44 authorized_keys -rw-r--r-- 1 rsyncdood rsyncdood 2670 2009-05-15 11:26 known_hosts -rw------- 1 rsyncdood rsyncdood 1675 2009-05-15 11:38 rsync-key -rw-r--r-- 1 rsyncdood rsyncdood 394 2009-05-15 11:38 rsync-key.pub rsyncood@remotehost bash ~ [1:0]$ ls -laF ~/.ssh total 24 drwx------ 2 rsyncdood rsyncdood 4096 2009-05-15 11:44 ./ drwxr-xr-x 16 rsyncdood rsyncdood 4096 2009-05-15 12:36 ../ -rw------- 1 rsyncdood rsyncdood 790 2009-05-15 11:44 authorized_keys -rw-r--r-- 1 rsyncdood rsyncdood 2670 2009-05-15 11:26 known_hosts -rw------- 1 rsyncdood rsyncdood 1675 2009-05-15 11:38 rsync-key -rw-r--r-- 1 rsyncdood rsyncdood 394 2009-05-15 11:38 rsync-key.pub
After the keys and permissions are installed, the public keys needs to be added to the authorized_keys file, so as to permit the use of the key during ssh operations. This can be done simply by cat'ing the public key on the remotehost as follows:
rsyncdood@remotehost $ cat ~/.ssh/rsync-key >> ~/.ssh/authorized_keys
Ensure that the sshd configuration (/etc/ssh/sshd_config) property PermitRootLogin is set to at least forced-commands-only (or to 'yes' for wider access). Restart the sshd on relevant hosts if any changes are made.
Executing rsync over ssh requires use of the -e (execute) option to rsync, in addition to any rsync options like compression, verbose, add/update and so on. In the following example, the rsync command executes over a directory called /home/something on srchost, sending the contents to remotehost:/export/public via ryncdood's ssh keys. The sync is for update (u) in verbose mode (v):
bash $ rsync -avz -e "ssh -i /home/rsyncdood/.ssh/rsync-key" --delete /home/something rsyncdood@remotehost:/export/public
Stuart Moorfoot 28 Oct 2008 foo@bund.com.au