grke / burp

burp - backup and restore program
http://burp.grke.net
Other
485 stars 77 forks source link

Strip paths during backup #247

Open Boldie opened 9 years ago

Boldie commented 9 years ago

I miss a method to strip paths during backup process. I am using snap-shots to make my backups in a consistent state. Unfortunately this will introduce at least a path element. I know I can strip it while doing the restore but this is quite inconvenient, because I habe to strip some paths and others not, so I habe to perform multiple restores. My favorit way would be to specify the include path in a manner like rsync does: /snapshotA/./home/ should backup the path /snapshotA/home which should be backed up as /home only.

grke commented 9 years ago

Hello, That is OK, except that I think the magic '/./' syntax is awkward and would probably be quite hard to implement in burp without creating lots of edge case bugs. Probably better/safer would be to make it possible to use the 'strip' option from the restore so that you can set it during the backup.

Boldie commented 9 years ago

You are true, but In my case it is interesting to make the definition how much to strip dependent on the include path. My use-case looks like the following: I habe a /home in a lvm and a / in another lvm. Now I mount a snapshot of the /home to /backHome and of the / to a backRoot and back them up now but I also habe a /boot which I back up without snapshot.

grke commented 9 years ago

OK, maybe something like this:

include = /backHome include = /backRoot include = /boot strip_prefix = /backHome strip_prefix = /backRoot

Boldie commented 9 years ago

That sounds good.

dbenoy commented 9 years ago

Perhaps, alternatively, burp could have an option to do automatic snapshotting, such as detecting if the filesystem is LVM or btrfs and creating the appropriate temporary mount, and backing up out of that instead, as needed, whenever it's asked to backup anything?

tokudan commented 9 years ago

Windows with VSS has a sane API to take snapshots and access them. Linux has nothing similar, due to the high amount of filesystems. My systems (standard debian jessie) supports 36 different filesystems after loading all filesystem modules: # grep -v nodev /proc/filesystems | wc -l 36 If you're using LVM to snapshot them, you need to write a specific subroutine for every filesystem how to mount that filesystem on the snapshot device (does it need fsck? does it need specifc mount options, as it's a clone? does it need specific mount options, as it's read-only?). If you're restricting the snapshot support to btrfs, then you've got a different problem. Which of the subvolumes actually needs to be cloned? btrfs is not a simple filesystem as ext2/3/4/vfat... you practically have an unlimited number of subvolumes that can be hidden within other subvolumes. And just figuring out which subvolume is mounted on a specific path can be a lot of work. And then, what do you do when you encounter btrfs on an LVM device? Flip a coin to decide which type of snapshot you use? :)

So automatic snapshotting on Linux is pretty much impossible to solve without the admin of the system taking care of the logic.

Stripping paths during a backup is a lot easier and you can use backup_script_pre and backup_script_post to take care of the snapshots. Just mount them on the same directory everytime. Maybe have a look at mount --bind/--rbind if you have to mount them at different paths for some other reason.

dbenoy commented 9 years ago

It doesn't seem like an insurmountable challenge at all, even by that description, especially if some restraint is shown and there isn't an attempt to support every possible filesystem in existence.

As for this solution being much easier, perhaps I should open a separate ticket, then, since the two aren't mutually exclusive.

dbenoy commented 9 years ago

311 opened.