/ Scheduling duplicacy backups

 

Personal photos

© 2006-2021 Vlad Didenko.

ALL RIGHTS RESERVED.

The content rights are not available for sale or licensing. Any use of the content except by the website owners is prohibited, unless specified per-page otherwise.

This website is for personal photos and posts, for the benefit of family and friends. As such it has a minimal needed set of features. No backward compatibility with old browsers is considered.

Most photos are processed to match a personal perception at the time of capture, and some photos are processed creatively. No guarantees presented about photos resemblibling a reality in any sense. No guarantees presented that a technology information posted on the website is usefull or harmless for your environment.

this post:
Creative Commons License
Using systemd to schedule duplicacy backupsVlad DidenkoCreative Commons Attribution-ShareAlike 4.0 International License

I really like the combination of Backblaze B2 and Duplicacy for an off-site backup. It is easy to setup and Duplicacy provides a very reasonable Quick Start guide as well as specific back-end details.

Once you have properly configured the backup source directories (see the documentation above), you can run backups manually from the backups' source directories:

duplicacy backup -stats

Now you should schedule backup jobs to run from all backup source directories at your preferred schedule. While using a classical crontab is still an option, here is a way to do it using systemd with more control over the jobs and extra benefits like proper logging. Here I show how to schedule the jobs as an unprivileged user, without using system-wide permissions.

The setup shows how to minimise future work when adding or removing extra directories to the backup schedule. While I prefix this setup as dpcy, nothing stops you from using a different prefix - just keep it consistent and without other collisions.

First you need a service template file. All backup runs will be derived from this template. Put it at ~/.config/systemd/user/dpcy@.service:

[Unit]
Description=Backup %I with duplicacy
Documentation=https://github.com/gilbertchen/duplicacy/wiki/Quick-Start

[Service]
Restart=no
WorkingDirectory=%I
ExecStart=/home/vlad/bin/duplicacy backup -stats
StandardOutput=journal
StandardError=journal
SyslogIdentifier=duplicacy@%i
PIDFile=%t/duplicacy@%i.pid

As the duplicacy executable path is different on your system you need to adjust the value of ExecStart in this sevice template.

Next, create the actual scheduling files, timers in systemd terms. In my setup all backup jobs will run at the same timeframe on a Sunday, with a 15 min variance.

For example, imagine that you have the following directories which you want to back up with duplicacy:

/home/archive
/home/family
/home/music
/home/photos

Get the systemd-escape values of the paths, to avoid surprises:

$ systemd-escape /home/{archive,family,music,photo}
-home-archive -home-family -home-music -home-photo

Then create a same file with varying names, pointing to your directories. Here is the first copy of the file, ~/.config/systemd/user/dpcy@-home-archive.timer:

[Unit]
Description=Backup the configured location with duplicacy

[Install]
WantedBy=timers.target

[Timer]
RandomizedDelaySec=900
OnCalendar=Sun, 00:42

Naturally, the [Timer] parameters may be adjusted per the documentation of systemd.timer and systemd.time.

When the timer fires, systemd will expect to be able to run a unit named in a same way as this timer file, except ending with the .service postfix, so dpcy@-home-archive.service. While looking for such service it will use the dpcy@.service template which we created above. The remplate will use the parameter -home-archive as the escaped value for %i. The %I macro in the template file will be substituted with the unescaped value of /home/archive.

And here are the other timer files created, checked, and systemsd is told to enable all timers:

$ cd .config/systemd/user/

$ ln dpcy@-home-archive.timer dpcy@-home-family.timer

$ ln dpcy@-home-archive.timer dpcy@-home-music.timer

$ ln dpcy@-home-archive.timer dpcy@-home-photo.timer

$ stat -c "%i %n" dpcy@*
43913832 dpcy@-home-archive.timer
43913832 dpcy@-home-family.timer
43913832 dpcy@-home-music.timer
43913832 dpcy@-home-photo.timer
43913874 dpcy@.service

$ systemctl --user enable dpcy@*.timer
Created symlink /home/vlad/.config/systemd/user/timers.target.wants/dpcy@-home-archive.timer → /home/vlad/.config/systemd/user/dpcy@-home-archive.timer.
Created symlink /home/vlad/.config/systemd/user/timers.target.wants/dpcy@-home-family.timer → /home/vlad/.config/systemd/user/dpcy@-home-family.timer.
Created symlink /home/vlad/.config/systemd/user/timers.target.wants/dpcy@-home-music.timer → /home/vlad/.config/systemd/user/dpcy@-home-music.timer.
Created symlink /home/vlad/.config/systemd/user/timers.target.wants/dpcy@-home-photo.timer → /home/vlad/.config/systemd/user/dpcy@-home-photo.timer.

To check the last runs’s status, next planned runs and recent log entries you can use the systemctl command:

$ systemctl --with-dependencies --before --user status dpcy@*.timer

Once the jobs have run you should be able to check their log output with joutnalctl, similar to:

$ journalctl --catalog --quiet --user-unit=dpcy@*.service --since="$(date -I --date="last Saturday")"

If at some point of time you want to schedule another duplicacy backup, say of /home/business, then just:

$ systemd-escape /home/business
-home-business

$ cd .config/systemd/user/

$ ln dpcy@-home-archive.timer dpcy@-home-business.timer

$ systemctl --user enable dpcy@-home-business.timer
Created symlink /home/vlad/.config/systemd/user/timers.target.wants/dpcy@-home-business.timer → /home/vlad/.config/systemd/user/dpcy@-home-business.timer.

If you happen to edit a template or a timer, or delete one altogether, do not forget to run the reload:

systemctl --user daemon-reload

Finally, do not forget to regularly test and prune your backups. The last point is not subtle - If you happen to change many files often it is very feasible to store more than you comfortably want to pay for in the cloud.

2021-02-14

 ∼   ⦾   ∽