search

project-rainstorm / rainstorm

The easiest way to own your cloud. Your data. Your cloud. Rainstorm.
Other
10 stars 5 forks source link

[Backups] Given a rclone config -> do scheduled backups #107

Open nullcount opened 3 years ago

nullcount commented 3 years ago

Overview

Use restic to create a local backups repository with deduplication built right in. Configure rclone to use Tardigrade access token as a backend. Connect restic with rclone to do backups to storj.

Issue Scope

  1. Aquire a functional rclone config
  2. Handle the initial setup of a backups repo (one-time operation) NOTE: DO NOT include this in setup.sh since backups will be enabled for users after they have been using the software for a while.
  3. Perform routine backups (using cron or some worker)

Tardigrade

We're using the rainstorm project in tardigrade for EVERY user's backups. Within this project, we have a single bucket called "Backups". 

Backups
 L User 1
 L User 2
 L User 3
...

A unique access token will be generated for each user with read/write permissions enabled ONLY on that user's folder within the Backups bucket.

Rclone

rclone config will kick off an interactive menu which asks you for some information about your backend. We will use tardigrade. The result of this command is simply a new file at ~/.config/rclone/rclone.conf. In the future, this file will be generated by the auth server with a unique access token that can be revoked server side.

Restic

Restic must initialize a repository, whereby backups are encrypted using a password. (Explore whether it makes more sense to ask for user input to get their plaintext password OR whether we should just use the password HASH to automatically interface with restic)

After the initial repo is created, the first backup should be started. This will take a LONG time, as restic is essentially creating a git-like repo with all the files included in the backup.

Subsequent backups will be faster.

Useful Links: Rclone Tardigrade Restic Tardigrade

After this issue, check out #108 #109

asharron commented 3 years ago

Still working on this. Should have a MR up within the next few days.