ocf / projects

Overview of technical OCF projects
3 stars 0 forks source link

Change backups from Box to b2 #65

Open ethanhs opened 3 years ago

ethanhs commented 3 years ago

My plan is: 1) make /opt/backups/test/restic 2) backup mysql and git to that folder via restic (<10GB aka b2 free tier) 3) test backing up via syncing with b2 via rclone 4) Empty the bucket 5) test backing up via syncing that folder with b2 via the b2 tool 6) get BoD approval for b2 costs 7) move mysql/git to back up to /opt/backups/live/restic 8) set up whichever of b2/rclone we decide on to sync /opt/backups/live/restic to b2 9) move things one at a time from rsnapshot to restic 10) ??? 11) Profit

If a new staffer wants to work together on setting up the restic backup configuration I'm happy to do so.

kpengboy commented 3 years ago

If we're going to revamp the backups system, let's also shard backups per user? That way we could purge just one user's backups if they request it.

We probably should also consider further optimizations, if possible, such as:

ethanhs commented 3 years ago

Hmm I do like the idea of sharding per user, but its not clear to me how we would accomplish that. Having one repo per user seems very annoying to manage, so perhaps we can have restic run as root for each user backing up to the same repo.

Only backing up user dirs when the user has actually used them

If we do shard per-user we could skip the directory if it is empty, sure.

Not creating new backup sets for users who were sorried before their most recent backup set was created.

Restic is incremental, so it wouldn't add anything to the repo if nothing changed.