search

digint / btrbk

Tool for creating snapshots and remote backups of btrfs subvolumes
https://digint.ch/btrbk/
GNU General Public License v3.0
1.72k stars 124 forks source link

Add pre/post snapshot command #58

Open Nicop06 opened 8 years ago

Nicop06 commented 8 years ago

It would be great to have a way to execute a command before / after creating or deleting a snapshot. This command should have as argument the name of the snapshot(s) that have been deleted.

I use btrbk to backup my SSD drive to my HDD. However, I would like to be able to boot my HDD with the last backup snapshot in case my SSD fails. To do so, I can create a writable snapshot of the HDD and replace the /etc/fstab so that it contains the HDD drive using a shell script. This script then only needs to be called by btrbk for the root subvolume with the path of the backup as an argument.

I can develop the functionality and submit a pull request if you think it would be useful to others. I am sure there are many more use cases where this can be used. I can also submit the script once it's well tested.

digint commented 8 years ago

This has already been discussed in #41:

"The reason why this is not implemented is security-related: If btrbk runs user-defined commands, it makes it possible for an attacker to run ANY commands as root by altering the config file. I don't like this."

You can e.g. use btrbk list latest --format=raw /path/to/backup/location as base for a script (there is also an example in the FAQ for this)

Nicop06 commented 8 years ago

My bad, should have read the manual / old issues. Thank you for the tips, that should work perfectly for my script. Also, many programs running as root already allow user defined commands such as syslog-ng. A configuration file used by a root process should always have at most 755 to prevent such backdoor.

But maybe running btrbk as root can also be a problem in some cases, especially for remote backup as I usually disable ssh root access. I think it could be a good to make btrbk work as a regular user. As a reference, how to delete a snapshot as a regular user: http://unix.stackexchange.com/questions/88932/why-cant-a-regular-user-delete-a-btrfs-subvolume.

By the way, thank you for this great tool, this is part of the reason why I migrated all my computers to BTRFS.

digint commented 8 years ago

A configuration file used by a root process should always have at most 755 to prevent such backdoor.

You are completely right here. Actually it's already very wise to have the config file UID=root MODE=755, as even snapshots can impose a security risk (e.g. by snapshotting/receiving some private subvolume to a less restricted place).

But the problem with btrbk hooks goes some further, as these will then also be called via ssh on remote hosts, and without very proper care this can easily break more things. Adapting "ssh_filter_btrbk.sh" to support free-form commands would also not be straightforward.

All this just give me a bad feeling, so I decided not to provide command hooks in btrbk, but rather provide script-friendly output.

I think it could be a good to make btrbk work as a regular user

Yes, but this is not possible right now:

By the way, thank you for this great tool, this is part of the reason why I migrated all my computers to BTRFS.

Nice to hear! Good for you, I had to write my own backup tool before migrating all my hosts :)

Nicop06 commented 8 years ago

Nice to hear! Good for you, I had to write my own backup tool before migrating all my hosts :)

I had a custom shell script for ZFS snapshot rotation before as I didn't like the other tools, but I was surprised to find a tool that almost matched all my needs.

btrfs subvolume delete (BTRFS_IOC_SNAP_DESTROY) needs root, unless user_subvol_rm_allowed mount option is set (I never tested this, and setting this mount option is probably a bad idea)

I tried the user_subvol_rm_allowed and it seems to work, and should be safe enough as you can only delete snapshot where you have write+exec rights (much like rm for directory). For reference, the associated kernel code: http://lxr.free-electrons.com/source/fs/btrfs/ioctl.c#L2373

btrfs subvolume list (BTRFS_IOC_TREE_SEARCH) needs root. btrbk needs this to fetch all subvolume information

I do not understand why only root can do that. I will ask BTRFS developers as it might not be necessary. If I am not wrong, you can list snapshots on ZFS with only read access, and they even have right management to allow creation / deletion / snapshot of sub-volumes. This might not be needed in BTRFS, but allowing snapshot listing shouldn't require extensive privileges.

Micha-Btz commented 8 years ago

@Nicop06

I do not understand why only root can do that. I will ask BTRFS developers as it might not be necessary. If I am not wrong, you can list snapshots on ZFS with only read access, and they even have right management to allow creation / deletion / snapshot of sub-volumes. This might not be needed in BTRFS, but allowing snapshot listing shouldn't require extensive privileges.

did you have news on this? i would like to run the backup script as user

penguin-wrangler commented 1 year ago

I haven't seen my particular use-case for pre/post snapshot hooks mentioned, so I will ask:

I've been using my own home-rolled brtfs snapshot-based backup solution, but am exploring btrbk because it seems so much more thorough. One thing I do in my script is to freeze/thaw VM filesystems with virsh before/after the snapshot of the volume containing those VM's disk images, so that the VM snapshots are application-consistent.

None of the comments I can find mention a solution for this. Doing our own pre/post command before and after the full 'btrbk run' is too long of a time span. I require that the VM filesystems freeze/thaw be of very minimal duration, i.e. just as long as it takes to create the snapshot and no more.

Are you still dead-set against providing a hook mechanism? I could of course add it myself, but would hate to have to maintain an out-of-tree patch.

digint commented 1 year ago

Doing our own pre/post command before and after the full 'btrbk run' is too long of a time span.

What you can do to minimize the duration:

  1. freeze VM
  2. run btrbk snapshot -p (consider using filters to only snapshot your VMs)
  3. thaw VM
  4. run btrbk resume

See the actions matrix in btrbk(1), COMMANDS for more details.

Are you still dead-set against providing a hook mechanism? I could of course add it myself, but would hate to have to maintain an out-of-tree patch.

No, actually I'm missing such a feature myself for some use cases :-/

As described above, it's not that simple to implement, and I'll have to put some more thoughts on how exactly such hooks would look like.

At the moment my priority is to finish the "action-cp" stuff (drop-in replacement for "btrbk archive"). ... and I'm quite busy with other stuff as well, so don't expect this too soon.

penguin-wrangler commented 1 year ago

What you can do to minimize the duration:

  1. freeze VM
  2. run btrbk snapshot -p (consider using filters to only snapshot your VMs)
  3. thaw VM
  4. run btrbk resume

See the actions matrix in btrbk(1), COMMANDS for more details.

Thanks Axel! That ability to break down and run only specified segments of the full backup process is just what I had missed and needed to accomplish my goal. Much appreciated.

I especially love that I was able to pretty-much use btrbk as a drop-in replacement for my cobbled-together solution and yet still take advantage of existing snapshots and continue incremental backups from there without having to do very large full backups again to get started. Sweet!