bit-team / backintime

Back In Time - An easy-to-use backup tool for GNU/Linux using rsync in the back
https://backintime.readthedocs.io
GNU General Public License v2.0
2.05k stars 203 forks source link

Add describing comments to default exclude list entries (was: Add in-filesystem snapshots to default exclusion list) #1583

Open noyannus opened 10 months ago

noyannus commented 10 months ago

Many users run a distro that comes with a default in-filesystem snapshot, possibly with default status "activated".

Path, or even existence, of the created snapshots may not be obvious to users (dotfiles and -folders, btrfs subvolume, ...), and will clutter BiT backups, if not excluded.

Please include the most common paths to such backups in the default exclusion list.

emtiu commented 10 months ago

Are you sure that btrfs subvolumes are picked up by rsync and land in Back In Time backups? Have you tested this, and could you provide some examples of paths that should be excluded?

noyannus commented 10 months ago

Are you sure that btrfs subvolumes are picked up by rsync and land in Back In Time backups?

Unfortunately not. I have hopped distro a while ago from one creating subvolumes with TimeShift to one that places snapshots under / (Snapper).

I'd recommend for the SUSE distro family, and likely for everyone using Snapper to exclude */.snapshots/*. Snapper defaults to place snapshots in root of the snapshotted subvolume, e.g. /.snapshots /home/.snapshots. It can use subvolumes mounted on them, but they would look the same to BiT.

buhtz commented 10 months ago

What is Snapper? Can you give us a link please?

I would say that */.snapshots/* is to broad (sensitive) to make it a default exclude.

noyannus commented 10 months ago

Snapper is a btrfs snapshotting tool, somewhat like Timeshift.

It is not widely known; probably because it originates from a German company whose distros are underrepresented in the anglophone Linux world.

A good start info is the Arch Wiki. With a GUI, Snapper is really neat.

A search for 'timeshift vs snapper' brings up useful comparisons (or so my notes from some years ago say...)

aryoda commented 10 months ago

Snapper source code: https://github.com/openSUSE/snapper

Ubuntu provides the packages snapper and snapper-gui so Debian may have these packages too...

aryoda commented 10 months ago

I think the path .snapshot must be excluded and is by default located under the root folder.

But since snapper can store snapshots in "subvolumes" (I have no idea if this term means the same for all file systems) it may be more complicated.

BiT is not meant to make backups of a system (OS) root folder but I guess the /.snapshot folder could also appear in a mounted pure data partition.

Do we know examples (apps) where excluding /.snapshot would lead to a problem because it contains real data and should be included in BiT backups?

noyannus commented 10 months ago

Snapshots of /home partitions are not unheard of.

A quick search for application "~/.snapshots/" finds no page with both. I think it is reasonably safe to exclude ~/.snapshots/, but should be accompanied with a brief warning in the readme.

buhtz commented 10 months ago

Ubuntu provides the packages snapper and snapper-gui so Debian may have these packages too...

Of course it is in Debian. Ubuntu/Canonical would never work them self. They only take the work of others and put their own label on it. 😆 😈

I see no problem with excluding ~/.snapshots/ and /.snapshots/ by default. It is different from the initial proposal */.snapshots/*.

noyannus commented 10 months ago

I see no problem with excluding ~/.snapshots/ and /.snapshots/ by default. It is different from the initial proposal /.snapshots/.

Yeah, I had not thougth through the ramifications. Feel free to close the issue.

emtiu commented 10 months ago

I think the default exclude list for all BiT users doesn't need to be changed.

btrfs is not very common (yet), and tools like Snapper even less so. If a user has both Snapper and Back In Time running, they probably understand the implications, and can adjust the exclude list for themselves.

But my opinion is not very strong, and a change to the default exclude list is not "expensive" (for us as maintainers) :)

noyannus commented 10 months ago

btrfs has matured and gained stability in recent years. It's hard to find reliable data for and popularity. One data point, this poll in the Endeavour forum from 1 1/2 years ago, shows it nearly as liked as ext4 (43% vs 49%; and ext4 means something!). Outside Endeavour, with distros for an on average less technically inclined user base, it may be somewhat less, though.

Snapper is more obscure, but seems to be on the rise as a Timeshift alternative with overlapping but in parts different functionality.

emtiu commented 10 months ago

Thanks, that's interesting to know :)

noyannus commented 10 months ago

Your're welcome. :-)

aryoda commented 10 months ago

I think our growing exclusion list demands for providing comments to the users (and us) in the GUI to explain why a pattern is in the exclusion list (eg. via tool tips or a second "Notes" column). Otherwise it is difficult to understand the impact of (not) excluding some folders.

image

Is providing comments for exclusion list entries worth a new feature request issue?

aryoda commented 10 months ago

I see no problem with excluding ~/.snapshots/ and /.snapshots/ by default.

Can we agree on adding this to our default exclusion list? Any objections?

Edit: This could be done together with adding mozilla and discord files to the exclusion list (https://github.com/bit-team/backintime/issues/1555#issuecomment-1788228360)

buhtz commented 10 months ago

OK, for me.

noyannus commented 10 months ago

I think our growing exclusion list demands for providing comments to the users (and us) in the GUI to explain why a pattern is in the exclusion list (eg. via tool tips or a second "Notes" column). Otherwise it is difficult to understand the impact of (not) excluding some folders.

Just an idea: The exclusion patterns list has enough room for a second column with a very brief explanation, e.g.


/tmp/*         Temporary files, deleted at every boot
/mysecrets/    Must never leave this machine

BiT could then provide a default text in a user editable text field (table cell), so users can add their own notes to the defaults and annotate their own exclusions. With this exclusion table made horizontally scrollable, the notes could be arbitrarily long.

Germar commented 10 months ago

I think the default exclude list for all BiT users doesn't need to be changed.

I'd second this. The default exclude is just for inexperienced users running backups of their home folder. Everyone else need to research on his/her own what need to exclude. Especially when it comes to backups of FS_ROOT.

Adding a comment line for each in-/exclude is a good idea. And if you already touch that part maybe add a checkbox to en-/disable each line.