Documentation about snapshot retention

[l0f4r0]

Hi,

It would be nice to have some documentation explaining how snapshot retention works with timeshift (how hourly/daily/weekly... snaps interfere with each other in the planning).

For example we could imagine something similar (it doesn't seem to work the same though) to this borgbackup doc: https://borgbackup.readthedocs.io/en/stable/usage/prune.html

NB: is it correct that it works that following way?

• hourly: keep last x snaps
• daily: keep last x end-of-day snaps
• weekly: keep last x end-of-week snaps (but today not included... Why?)
• monthly: keep last x end-of-month snaps (but today not included... Why?)
• boot: I think it should be keep last x boot snaps but I don't understand since I have snaps considered as Boot while I didn't boot my machine...

NB: levels interlap with each other (1 snap can be associated with multiple levels at the same time)

Thanks in advance :)

[DavidInRacine]

Timeshift documentation is sparse, with the result being trial and error. Worse is the potential for lost data.

Quoting from TeeJee's Timeshift user's guide (bold typeface added) we often forget that Timeshift was not written to be a backup tool. It is an OS-rollback tool. We are trying to shoehorn backup functionality, sometimes with unwanted results:

User Data is Excluded by Default

Timeshift is designed to protect system files and settings. It is NOT a backup tool and is not meant to protect user data. Entire contents of users' home directories are excluded by default. This has two advantages:

• You don't need to worry about your documents getting overwritten when you restore a previous snapshot to recover the system.

• Your music and video collection in your home directory will not waste space on the backup device.

You can selectively include items for backup from the Settings window. Selecting the option "Include hidden items" from the Users tab will backup and restore the .hidden files and directories in your home folder. These folders contain user-specific config files and can be included in snapshots if required.

Note: It is not recommended to include user data in backups as it will be overwritten when you restore the snapshot.

(https://github.com/teejee2008/timeshift)

[l0f4r0]

In my opinion, Timeshift is a system (by default) backup tool. It's not a pure snapshot tool in that it copies data the first time before creating hard links the next ones (so it's not just about pointers/metadata). One can even put the copy on an external partition if need be. I think Timeshift speaks about snapshots just to insist on the rollback aspect and the ease of operations (like "restore points" for Windows).

Anyway, snapshot or backup, user data or system data, I think it's important to explain somewhere how retention works so the user can define his strategy knowingly :)

[DavidInRacine]

Agreed with your last sentence.

Following TeeJee's guidance I excluded my Home folder from Timeshift. I instead use DejaDup (aka Backups) to maintain weekly snapshots of my Home folder, so TWO "backup" utilities need to be working in tandem. DejaDup is a mostly functional app with a bare bones UI. Sometimes it fails to restore my Home folders due to permissions complaints. ADVICE: make sure to set all subfolder permissions correctly using chmod throughout the entire directory structure before making your snapshot. Unfortunately, newly installed or reinstalled software can change subfolder permissions layers down without your knowledge, so then DejaDup flags the parent folder archive as un-restorable.

By contrast, trying to include my Home folder in Timeshift resulted in multiple failed restored volumes for me across multiple distros. The restored volumes never successfully booted and emergency mode could not fix the problems. It was a total mess involving much wasted time (aka learning experience). I lost my Home folders on multiple occasions since I began transitioning from Mac OS X in November, 2022. Fortunately my Home folders did not contain anything important – just my time, which apparently has little value to linux OS. [snark]

The linux backup experience is fraught. It is a massive step backward from the simplicity of Apple's Time Machine and Shirt Pocket's SuperDuper! Were I not extremely determined to leave Mac OS (for a long list of reasons) I would have given up on linux because of its backup woes.

Some voices in the linux community seem to be in denial, or they tell us to use rsync via the command line... which itself is fraught. Linux is extremely finicky about UUIDs being exactly correct everywhere and at all times, whereas Mac OS X seems much more forgiving to the end user.

Just my 2C. YMMV.

[geekpete]

User data backup is a workable use case if the restore phase is only done as cherry picking specific data from a particular point in time snapshot via the Browse Files functionality, similar to Mac OS X Time Machine allows. It might not be by design, but it's still a super useful ability to allow users to keep point in time copies of their user data and have access to it in the future if it changes or is deleted, as long as the risks are clearly called out. It might be considered a more advanced usage vs a use case for a typical user though. It's something that is useful and could be documented as such.

[gauvinalexandre]

Hi! Relating to @l0f4r0's question, I need to know if I set up Timeshift to do daily snapshots and keep only, let's say, only 14 snapshots, Timeshift will delete the snapshot I create manullay throughout the time. In other words, I want the manually created snapshots to always be saved, and tohers to clean up after 14. Is this possible? Is it how Timeshift works? Thanks for your answers!

[l0f4r0]

I need to know if I set up Timeshift to do daily snapshots and keep only, let's say, only 14 snapshots, Timeshift will delete the snapshot I create manullay throughout the time. In other words, I want the manually created snapshots to always be saved, and tohers to clean up after 14. Is this possible? Is it how Timeshift works?

I don't know how retention works for manual snaps (maybe they are not affected by automatic deletion because they are out of the M/W/D/H/B retention strategy?). I think you'll have to find out by experimenting because I see no mention about it... Anyway, it accentuates the need about a better documentation!

[gauvinalexandre]

Thanks @l0f4r0 for your answer! To be clear, here's an example of my desired usage. Eventually I'd like to have a linear history of Timeshift snapshots looking like this, with a retention of 6 weeks:

1. [manual] Fresh system installation
2. [manual] First system update
3. [manual] All software installed
4. [auto] Weeky snapshot 1
5. [auto] Weeky snapshot 2
6. [auto] Weeky snapshot 3
7. [auto] Weeky snapshot 4
8. [manual] New system update
9. [auto] Weeky snapshot 5
10. [auto] Weeky snapshot 6
11. [auto] Weeky snapshot 7
12. [auto] Weeky snapshot 8
13. [manual] Special event

As shown, timely automatic snapshots can be deleted after the expire (weekly snaps 1 and 2), but manual snaps must always remain unless manually deleted. If this is not the actual mechanic of Timeshift, should I submit a feature request? Anybody else thinks like me that this is a good idea (or not)?

[l0f4r0]

Undestood @gauvinalexandre, it's clear.

But before submitting a feature request, I think you should do some testing because maybe it already works as you expect. I suggest you do your tests apart from your real usecase / production and you take hourly snaps to be quicker than weekly. Tell us your conclusions but I hope manual snaps are not affected by the M/W/D/H/B retention strategy...