search

kimono-koans / httm

Interactive, file-level Time Machine-like tool for ZFS/btrfs/nilfs2 (and even Time Machine and Restic backups!)
https://crates.io/crates/httm
Mozilla Public License 2.0
1.33k stars 28 forks source link

Proposal: A btrfs port of httm #33

Closed kimono-koans closed 2 years ago

kimono-koans commented 2 years ago

Reposted from r/btrfs:

...

When last I asked, I, perhaps too hastily, decided a port wasn't for me, because httm couldn't be made to easily work for all auto-snapshot programs (because the snapshot subvolumes/directories were never in the same place).

Now, I think searching for the most common subvolume mount points (.snapshot for Snapper, etc.) and/or letting the user set another directory via an environment variable (e.g., BTRFS_RELATIVE_SNAPSHOT_DIR) would be enough to cover most use btrfs use cases.

I'd like to ask:

  1. Is this a sane approach?
  2. Should (when do) I ever need to care about "@" subvolumes? That is -- if my PWD is /var/log, and I have a snapshot subvolume for /var/log, the location of .snapshots will generally always be at /var/log/.snapshots/?
  3. What are the most common snapshot-ing schemes used by btrfs users besides snapper (.snapshots) and timeshift (timeshift-btrfs/snapshots)?
  4. Is there anyone here, who keeps lots of snapshots (100s-1000s), who would be willing to test the software on their system?
  5. Could some snapper/timeshift users post their detailed mount and btrfs-subvolume(8) information so that I'm sure what I'm doing comports with how btrfs is used in practice? Perhaps to a Github issue I open? I'd particularly be interested in anyone who backs up (send/receive) their root volumes locally to another volume/pool.
  6. When you wish to view your snapshots read-only, are subvolumes automounted as needed or must you use some other process to mount?
KhalilSantana commented 2 years ago

I'd like to start this by saying I don't fully understand what your tool is doing, but it seems to be a sort of timeline version control for getting old versions of a file back. With that in mind, I'll try to answer your questions:

Is this a sane approach?

I wouldn't do it that way, let users configure where your tool looks for snapshots of any given subvolume, or scan the filesystem for other subovlumes containing that subvolume's UUID in the parent subvolume field.

Example: if you want to look for snapshots of subvolume @home (UUID=c4f3...) then run the equivalent of btrfs subv show and check if it has the parent UUID set to "c4fe"

Should (when do) I ever need to care about "@" subvolumes? That is -- if my PWD is /var/log, and I have a snapshot subvolume for /var/log, the location of .snapshots will generally always be at /var/log/.snapshots/?

The @ naming scheme is just that, a naming scheme born out of Ubuntu and subsequently hardcoded into some software (Timeshift), but it bears no significance beyond that. I call it the "flat layout with ubuntu-style naming" for that reason.

No you shouldn't assume that /var/log's snapshots will be in /var/log/.snapshots.

What are the most common snapshot-ing schemes used by btrfs users besides snapper (.snapshots) and timeshift (timeshift-btrfs/snapshots)?

Those are about it, I've personally never seen many other layouts except for hand-made ones.

Is there anyone here, who keeps lots of snapshots (100s-1000s), who would be willing to test the software on their system?

Sure, once you have a beta release or something just ping me and I'll give it a test (I use Timeshift). Also ask on #btrfs on libera.chat (also available via Matrix), I'm sure you'd be able to find some other people to give it a go.

Could some snapper/timeshift users post their detailed mount and btrfs-subvolume(8) information so that I'm sure what I'm doing comports with how btrfs is used in practice? Perhaps to a Github issue I open? I'd particularly be interested in anyone who backs up (send/receive) their root volumes locally to another volume/pool.

I've written at length about Timeshift, and my personal layout, here, it will also answer many questions you might have: https://github.com/archlinux/archinstall/issues/781

The TL:DR for timeshift is toplevel/@ mounted as / and toplevel/@home mounted as /home, the rest timeshift figures out by itself.

When you wish to view your snapshots read-only, are subvolumes automounted as needed or must you use some other process to mount?

Read-only vs Read-write is not relevant here. If a subvolume is mounted, then all child subvolumes will appear as normal folders inside it. That is, if toplevel is mounted at /mnt/, then you can see both the /mnt/@ and /mnt/@home subvolumes, etc.

Hopefully that was somewhat helpful, if not, just hop in on the IRC/Matrix channel, or ask here.

kimono-koans commented 2 years ago

Appreciate you taking a look!

Snapper should just work as it is very similar to the ZFS convention.

Most interested in testing Timeshift as I wasn't able to get it working on my btrfs-test zvol. Any help you/others could provide would be much appreciated. 

# if you don't already have cargo/rust
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh 
cargo install --git https://github.com/kimono-koans/httm/tree/btrfs_port

Thanks!

kimono-koans commented 2 years ago

Closing this issue for btrfs-snapper with the following commit: https://github.com/kimono-koans/httm/pull/34

Right now, it's WONTFIX for Timeshift (btrfs-timeshift). Why? Because Timeshift is hard to test, and requires a different parsing for mount (or some other means). If some one wants to make it work and maintain it, it is definitely possible, but for me, because I'm a ZFS guy and it's so hard to test Timeshift, it needs needs to be clearly partitioned away from the working ZFS code.

Thanks for your help!