Setting up cron to do a remote backup with rsync


I use rsync with cron to backup ISGSP.net and these technical notes each night to a backup server (Solaris\sparc 8 with a tape drive).
By default, rsync uses the ssh protocol to do a remote copy (rsync is a secure replacement for rcp), therefore we need to make sure ssh and keychain are avalible on this host.  keychain will give cron passwordless login to the remote host.

Make sure rsync is installed.
    # whereis rsync

If the only return is /usr/ports/net/rsync then you need to add rsync with ports or packages.
If your return includes /usr/local/bin/rsync  then rsync is installed.  Rsync is not installed by default.

We need to write a shell script for cron.  Everyone has there own idea's on how this should be done.  Here is the way I like.

Creating a directory call "bin" in your home directory is a great way/place to store your scripts/executables in a central location.  Why bin?  bin is short for binary.  Executables are usually binaries (but also scripts).

Create a directory in youe $HOME to store your own executables.
    $ mkdir bin

Move to that directory.
    $ cd bin

Create a new script.
For an explaination of the first line read the ssh and keychain page
For an explaination of the second line see below "Notes on rsync"
    $ vi myscript
            source /usr/home/$USER/.keychain/[hostname]-sh > /dev/null
            /usr/local/bin/rsync -aq /path/to/backup remotehost:/remote/path

Make the script executable (you can't run the script if it is not executable)
    $ chmod u+x myscript

Test the script.  $HOME/bin is not in your path.  You can add it to your path or type the path to it.  Your current working directory (which is now $HOME/bin) is repersented by a "."  so the path to a file in your current working directory would be "./" (does that look like a period followed by a forward slash?)
    $ ./myscript

Add the script to your crontab.  Make sure your $EDITOR is set.
Cron and $EDITOR are explained below in "Notes on cron".
    $crontab -e
        45 3 * * *  /home/$USER/bin/myscript



Notes on cron

The cron deamon is a periodic scheduler.
Cron reads your cron table (everyone can have thier own) and runs the commands at the time you specify.
You interact with your cron table using the command crontab.
crontab uses the EDITOR variable to create a new, or edit an exsiting cron table.

To set the EDITOR variable to your favorate editor add the following line to your ~/.profile or equvalent(depends on your shell)
    export EDITOR=/pathto/editor
I use bash v2 as my shell and my favorate editor is vi so I will add the following to my ~/.bash_profile
    export EDITOR=vi

To list all exsiting entrys in your cron table, type:
    $ crontab -l

To remove all exsiting entrys in your cron table, type:
    $ crontab -r

To create a new, or edit an exsiting cron table, type:
    $ crontab -e
This will open your cron table in your $EDITOR

Fields in your cron table.
There are six basic fields for each entry in the cron table.  The first five are when you want it to run and the sixth is what you want to run.  The fields are blank space delimited.

The first field is for minutes.  The values can be from 0 to 59
The second field is for hours.  The values can be from 0 to 23
The third field is for day of the month.  The values can be from 1 to 31
The forth field is for months.  The values can be from 1 to 12.
The fifth field is for day of the week.  The values can be from 0 to 6.  (0=Sunday on many versions but not always)

If you want multple values in one field, seperate with a comma.  Example:  0,6,12,18
If you want a range of values in one field, indicate with a hyphen.  Example: 0-5
If you want to use all values for one field, indcate with an asterisk.  Example: *

The last field is the path to the command or executable script you want to run.  It is advised that you point to a script you have created to run.  This way you can test the script before you enter it in the cron table.

Example.  To run a command every 15 minutes, Monday to Friday, from 9am to 5pm, all year long

0,15,30,45   9-17   *  *  1-5   /pathto/myexecutable/script



Notes on rsync

rsync is used to synchronize files in different locations.  This could be two directories on the same server or two directories on different servers.  It only sends the differences between the files which speeds up the transfer considerably, well after the initial transfer.

The line in the script above explained
            /usr/local/bin/rsync -aq /path/to/backup remotehost:/remote/path

/usr/local/bin/rsync
Call your commands by using the complete path.

-aq
Options passed to the command.  q=quiet, suppresses messages from remote host.  a=archive, recursive, preserving almost everything (permisions UID etc...).

/path/to/backup
[source]This is the path to the directory you wish to make a backup of.

remotehost:/remote/path
[destination] remotehost could be the IP address or hostname. ":"seperates hostname from the path to store the files on the remote host.

My real script's second line is:
/usr/local/bin/rsync -aq /home/isgsp/html 192.168.10.48:.

This will copy recursively everything in  /home/isgsp/html to $HOME/html on the remote host 192.168.10.48

There is a lot more that rsync can do, it rocks.  Read the man page or do a search on google.


Index