[idea] allow commands to run before and after snapshot creation

[zilexa]

To prevent data corruption, no data should be written to a subvolume when a snapshot is being created. For example, data could be written to a database. A (docker) container could be writing data to a mapped volume etc.

A user can easily prevent this by using btrbk in a script that will simply stop all actions/apps/containers before running btrbk and then start them again.

However, this way, apps/containers could be unavailable/offline for a while, since the entire backup process could take long, sending snapshots to various local or remote locations, while actually creating snapshots only takes a second.

My suggestion would be to allow btrbk to run a command before actual snapshot happens. And run another command when the snapshot creation process has finished. This could even be limited to a specific subvolume in the .conf file.

For example, docker-compose users should run docker-compose stop and docker-compose start before and after snapshot creation.

Would be nice if such an option to "command to run before snapshot" and "command to run after snapshot" could be added to the .conf file.

[richardm1]

Have a look at ExecStartPre and ExecStartPost and ExecStartCondition here. One might add these to the /usr/lib/systemd/system/btrbk.service unitfile.

I've yet to try this but I might soon if I can't get a handle on btrfs quotas/qgroups causing my system to freeze for 5-10 seconds after every btrbk run. I'll use them to disable and re-enable quotas before and after snapshots. Fingers crossed it'd work.

[zilexa]

Btrbk service file? As far as I know btrbk runs when you or your script executes "btrbk". Why would you install it as a service running in the background?

[richardm1]

Btrbk service file? As far as I know btrbk runs when you or your script executes "btrbk". Why would you install it as a service running in the background?

To take hourly/daily snapshots. Systemd is my spirit guide. 😄

[Unit]
Description=btrbk backup
Documentation=man:btrbk(1)

[Service]
Type=oneshot
ExecStart=/usr/bin/btrbk run

My timer. This is modified to use monotonic timing as I don't have any need for snaps exactly on the hour.

Description=btrbk hourly snaps

[Timer]
OnBootSec=10min
OnUnitActiveSec=1h

[Install]
WantedBy=multi-user.target

[zilexa]

Ah I understand. But no, I already run a command before and after I run btrbk. Your suggestion doesn't change that, you just do it via systemd service, I just use cron and a bash script.

See my first post again. I am asking to run a command before snapshotting starts and run a command after it finishes. That's not the same as running a command before btrbk starts and again when it finishes.

Snapshotting is only 1 action taken by btrbk and takes less than a second, while scanning for subvolumes, snapshots, deleting them, sending them to backup locations (especially that) can take much more time.

My point is: a snapshot of an active system writing data to its source subvolume is not a good idea. That's why I suggest the ability to run a command before doing that and one after, and to have the least amount of "downtime", only do it before/after snapshotting, instead of the way you and I do it now :)

[Sepero]

Interesting feature request. As a work around, you could structure your cron script like this:

1. pre-btrbk.sh
2. btrbk snapshot
3. post-btrbk.sh
4. btrbk resume

[zilexa]

Didn't know about "resume", thanks! I will try that. Still hope this will be implemented in the future.

[digint]

See my first post again. I am asking to run a command before snapshotting starts and run a command after it finishes. That's not the same as running a command before btrbk starts and again when it finishes.

The reason why this does not exist (yet) is that adding pre/post hooks is not that trivial, there are many places you might want a hook (snapshot, send/receive, archive, ...), it is not always clear what arguments to pass for the hooks, and you might want to control to run the hooks on the client / remote / both, ...

To keep things simple (for now), the idea is to write scripts using btrbk list <subcommand> <filtre> --format=raw, and run them aound btrbk <mycmd> in cron.

[zilexa]

Thanks for the feedback @digint, for now I will try the suggestion from @Sepero as its very simple, I think I can modify the btrbk-mail script even though I'm no expert in bash scripts, to run those commands instead of just btrbk run.

[zilexa]

I have now applied the suggestion from @Sepero in the btrbk-mail.sh script. I can confirm it works as expected! https://github.com/zilexa/Homeserver/blob/master/docker/HOST/btrbk/btrbk-mail.sh

What I did:

1. renamed the brtbk_command to btrbk_command1 with value snapshot and created btrbk_command2 with value resume.
2. Duplicated the whole section of ebegin "Running btrbk backups", now there is one for snapshots that contains the stop and start commands to stop and start containers and the old one for backups afterwards.
3. Note I use a global env variable that I set on my host, ${LOG_USER}, this way I can run a script with sudo/root, but run a specific command in that script with the logged in, non-root user. Since docker is installed normally and should not be executed as root.

Any improvements/suggestions to my changes to the script are welcome ofcourse.