linuxmint / timeshift

System restore tool for Linux. Creates filesystem snapshots using rsync+hardlinks, or BTRFS snapshots. Supports scheduled snapshots, multiple backup levels, and exclude filters. Snapshots can be restored while system is running or from Live CD/USB.
2.5k stars 91 forks source link

Is saving backups to a mounted network share (NFS/SMB/etc) still something that can't be implemented? #140

Open dougthor42 opened 1 year ago

dougthor42 commented 1 year ago

Is your feature request related to a problem? Please describe. The 3-2-1 rule of backups: 3 copies of your data on 2 separate media with 1 offsite. Depending on your definition of "media", a drive that exists within the same physical host is considered the same "media".

More and more people have NAS these days - Synology, QNAP (bleh), or BYO with something like Unraid. A backup solution should be able to use such hardware as a backup target location.

In my mind, I extend the 3-2-1 rule and think that data should look like:

  1. Active data: this is what you currently work on
  2. RAID: disk fault tolerance. Not a backup!
  3. Backup 1: local or near-local, fast recovery. This is used for recovering from misconfiguration and bad software deploys. Example: BTRFS snapshots, ESXi snapshots, etc.
  4. Backup 2: on-site but separate physical hardware. Eg: a NAS in a separate MDF / IDF / room / nearby building. This is used for recovering from hardware failure or localized events (eg: small fire, water damage, etc.)
  5. Backup 3: off-site. Eg: AWS/GCP/Azure, building in a different city/state/country. This is used for recovery from catastropic events (hurricane, earthquake, etc) and potentially archival storage if wanted.
  6. Backup 4: offline: Eg: Tape, USB drive, cold HDD, etc. Used for archival and massive cryptoware recovery. Though backup 2 and 3 should also be resilient to cryptoware :grin:

Perhaps the goal timeshift is to only address Backup 1 (local). If that's the case, that's fine. But if the goal is also address Backup 2 (separate physical hardware), then network targets are a must.

Describe the solution you'd like It would be great if we could save the snapshots to a network share, such as /mnt/my_nas/backups/. This network share might be NFS, SMB, or other. To first order I'd expect that NFS is (a) easiest and (b) most common.

Describe alternatives you've considered In short: not using timeshift :disappointed:.

I've thought about using dd (and cron and N supporting shell scripts):

Additional context There are many feature requests for such a thing in the original repository. Some of them were dismissed as "wontfix" due to a core design decision that prevents saving backups to a network path from working correctly.

There are also related issues in this repo:

token47 commented 1 year ago

I just gave a shot on NBD and it worked very well. Before giving more details, let me state the obvious:

I don't care too much about those things because I'm comfortable with fixing things manually, or removing interrupted snapshots to start again, or even restoring by re-installing a new system from scratch and just using the backup to copy some files over (specially my home, which I do include in the backup although this tool was supposed to be system only).

On the remote server, also running linux (in my case debian, it's an open media vault installation), you create a giant file and format it as ext4.

Then you configure NBD server to export that file. You will need to play with /etc/nbd-server/config and maybe other files. Too many variables to write a tutorial here. On the client, you mount that filesystem over the network using nbd-client. Same, read the manuals.

That /dev/nbd0 device will appear to the system as a normal disk, it will even be listed as "disk" type in the lsblk tool and therefore the timeshift tool will see it and allow you to select it. Just click on that, start your first backup, it just works.

You will need to automate connecting/umounting/disconnecting before/after running the backups. I use ARGO for gnome to easily connect in one click, but that's outside the scope here too.

Hope this helps.

token47 commented 1 year ago

Ok, spoke too soon. While the disk do appear in the list, can be selected and the backup do occur normaly, after it finishes the snapshot list appears empty. If you ask for another backup, it will be full again. If you take a look into the remote disk, everything is there the same as in a local disk (subdirectory timeshift with snapshots).

Does anyone have any idea why? This one was so close...

nathanielatom commented 1 year ago

@token47 did you end up having any luck using a network share?

Came from this stack exchange question https://askubuntu.com/questions/1234965/timeshift-backup-to-samba-nas-drive

token47 commented 1 year ago

@token47 did you end up having any luck using a network share?

Came from this stack exchange question https://askubuntu.com/questions/1234965/timeshift-backup-to-samba-nas-drive

Unfortunately no.

Reading the source code, it uses lsblk to filter by "disk" on the TYPE column. I read somewhere that someone changed the source to allow more types, but I did not want to do that (forking and keeping a software like this is not something that I want to do). Then I was trying to find a way of using a network disk that would present itself as "disk". Creating a loopback file in the network drive and mounting that as an EXT4 image locally is a good solution (kind of like what MacOS does with time machine) but unfortunately it shows as "loop" type and is excluded by timeshift when presenting the disks to use as target. Then I discovered that Network Block Device (nbd) will do something similar (with the advantage of mounting directly from the network, without the need to first mount SMB and then mount the loopback), and it's type appear as "disk". Bingo, timeshift presented the disk and successfully made a backup over it. If I go to the disk, everything is there, the same as an external HD. The problem comes after that, there is some logic in the code that is preventing timeshift from seeing the backup it just made and the list of backups stays empty, even though it just made it. So I gave up, too much hassle for something that should be simple. I hope the developers will just allow and implement support for this. I know it makes it more difficult to restore, but I just want the data copied, I can restore manually if necessary.

nathanielatom commented 1 year ago

Thanks for the info. Would an alternative route (hopefully simple) just be to allow the user to specify an arbitrary path instead of just /timeshift so you could specify the location to be a folder in the SMB mountpoint?

token47 commented 1 year ago

Thanks for the info. Would an alternative route (hopefully simple) just be to allow the user to specify an arbitrary path instead of just /timeshift so you could specify the location to be a folder in the SMB mountpoint?

Not necessarily. SMB mountpoints are not supported for a good reason, it's not a filesystem with the same characteristics that timeshift expects (where rsync can clone stuff using hard links). So, directly using smb would not be an option. Even NFS would be problematic, I imagine.

What can be done and would work well, if the software would let it, would be to create an ext4 or btrfs formatted image inside an SMB share/mountpoint and that image file to be mounted as ext4 using loopback mount. This is just perfect in the point of view of having all the features expected for the filesystem and would definitely work for doing backups. The tool would need to have support for this (mouting a file instead of a device), but I would be ok with pre-mounting it myself and just passing a custom path that timeshift can use. This can probably already be done by changing the UUID inside the config file, but it would break every time timeshift tried to write the config file again and I didn't want to keep updating that.

Now, I think the whole controversy about not supporting network backups is not about how to technically support a remote disk but instead how useful is it since you are not going to be able to restore from it as you would easily do from a local harddrive. In my specific case, I don't care about restores. I just want to have a snapshotted frequent backup of all my files somewhere, if I have any problem with my laptop I will just re-install it fresh and copy selected files from the network backup back later. Timeshift is just a convenience (compared to configuring rsnapshot script by hand).

token47 commented 1 year ago

Hummm, today I realized that my timeshift has been broken for some time now. And it seems I was hitting the issue https://github.com/linuxmint/timeshift/issues/152 -- it seems to have stopped working when rsync was updated in my ubuntu.

The problem I was having on my tests with NBD were exactly the same, so it may have been just a huge coincidence that I had this problem just when I was testing and it seemed that NBD was not working correctly. I'll test again soon.

mgumbley-resilient commented 11 months ago

I wonder if iSCSI might be used for this - to mount a block device over the network, format it as ext4, and use it to back up with timeshift? Haven't tried it.