search

helloSystem / Utilities

Utilities written in PyQt5, meant for use with helloSystem
BSD 2-Clause "Simplified" License
28 stars 29 forks source link

Backup utility #155

Open probonopd opened 3 years ago

probonopd commented 3 years ago

image

We need a Backup utility that can make backups to local media, the local area network, and to Cloud providers. Probably the initial focus should be on Cloud Backup.

Cloud Backup

Requirements

Possible underlying tools

Possible cloud service providers

grahamperrin commented 3 years ago

Under a broader heading e.g. Backup options …

From https://github.com/helloSystem/hello/issues/32#issue-730983594:

…

ZFS features I want to make use of

  • …
  • …
  • Replication (network backups): I want to backup (image) the whole system to a network device, which I can restore from ("bare metal") like with a Time Machine network backup. Doing subsequent backups should copy only what has changed ("delta backups"). PC-BSD had a graphical installer that could restore systems from network servers (e.g., FreeNAS) using ZFS replication over SSH. I would want to re-create something like that.

Ideally I want to configure as little as possible as I am not a system administrator but a mere end user (zero configuration). …

sysutils/life-preserver

Port deleted around a year ago – abandoned upstream (https://github.com/trueos/lpreserver archived) – but let's toy with the possibility of forking and resurrecting this, with the aim of:

The GUI was relatively easy to use in TrueOS Desktop. From TrueOS 12: a review – Developers Log (2017-04-18):

image

From what I recall of earlier use of the OS, it was relatively easy to configure the security that was appropriate/required for network backups. https://www.phoronix.com/forums/forum/software/bsd-mac-os-x-hurd-others/896297-trying-out-the-freebsd-powered-trueos-with-its-custom-qt-desktop?p=896691#post896691 recalls:

… top notch. It graphical program that can send ZFS snapshots to a FreeNAS server, and also restore your system from boot CD from said ZFS snapshots. That is a really impressive program and they get a WELL DONE TRUEOS! for that.

☑

Some time after switching to FreeBSD-CURRENT I did install the port and whilst I always intended to begin using the software, I never did because I couldn't put my mind to the security aspect, which was relatively difficult (for me) without the TrueOS Desktop context.

A handful of other links, for posterity:

grahamperrin commented 3 years ago

Backup options … myriad, but I never read of anything, for FreeBSD, that came close to Apple's Time Machine experience.

Life Preserver was closest, to the best of my knowledge, but (without TrueOS Desktop) there was the starting hurdle.

Speeding through FreshPorts today, this caught my eye:

sysutils/py-zfs-autobackup – ZFS autobackup

Periodically backup ZFS filesystems to other locations

https://github.com/psy0rz/zfs_autobackup#readme

… and so on.

If Life Preserver can't be suitably resurrected then ZFS autobackup, or something like it, might form the basis of an alternative.

probonopd commented 3 years ago

If I understand it correctly, Life Perserver "only" automates the automatic scheduled taking and removal of local ZFS snapshots but does not help you to get those backed up e.g., to the Cloud. Or does it?

grahamperrin commented 3 years ago

Features of Life Preserver included automated send/receive with e.g. FreeNAS. In the screenshot above there's the cloud icon with an up-arrow; the Replication button.

Defocusing from TrueNAS

https://www.google.com/search?q=ZFS+send%2Freceive+cloud&tbs=li%3A1#unfucked&tbs=qdr:y ▶ rsync.net: ZFS Replication to the cloud is finally here—and it’s fast | Ars Technica (2015-12-17)

Constraining the search results to the past year ▶ a variety of discussions, I don't keep abreast of ZFS-friendly cloud offerings but for one of the first search results that I randomly picked – recent Cloud backup solution that is ZFS-friendly : zfs on Reddit – rsync.net was the (still) the first answer to a question.

probonopd commented 3 years ago

https://www.rsync.net/pricing.html: Interesting, but pricing starts with a 400 GB Minimum Order at 10 USD per month, which is prohibitive if you want to store e.g., only ~/.config

probonopd commented 3 years ago

Backup your ZFS snapshots to S3: https://github.com/presslabs/z3

grahamperrin commented 3 years ago

Yeah I'm not recommending any particular product or service (we have backup discussions galore elsewhere on the Internet), just focusing on ZFS and the only snapshot-oriented app that was memorable to me.

I imagine that we'll also find web app interfaces to ZFS snapshots, send and receive.

grahamperrin commented 3 years ago

Food for thought, via https://old.reddit.com/r/zfs/comments/kwo8t1/recursive_mount_of_snapshots/gj94q6e/:

https://github.com/michabbs/rclone-sync-zfs

probonopd commented 3 years ago

Backup strategy idea: Restic for fast backups & restores https://people.freebsd.org/~dch/posts/2021-03-05-restic-backups.html

probonopd commented 3 years ago

Idea: Instead of pruning old backups periodically, why not prune the oldest ones at the time directly after new ones have been made, and show the process as "complete" to the user only once the pruning is complete?

probonopd commented 1 year ago

Started a Backup.app that (for now) just saves the zroot/var/home dataset in a backup.zfs file.

Could/should be expanded to

Anyone wants to help me make it?

The beginnings are at https://github.com/helloSystem/Utilities/tree/master/Under%20Construction/Backup.app