hypercore-protocol / hyperdrive-daemon

Hyperdrive, batteries included.
MIT License
156 stars 24 forks source link

Add a `backup` command #75

Open da2x opened 4 years ago

da2x commented 4 years ago

Folks are gona need tools to back up their drives and especially their keys.

backup [opts] [backup-path.tar.gz]

Backup should contain all keys and data; plus a manifest that can be ingested by hyperdrive import --from-backup=.

andrewosh commented 4 years ago

This one's pretty crucial to have in the near-term, in my opinion, so thanks for opening this. I think it'd be best to start with the simple version of this, that backs up:

  1. The complete set of writable hypercores in ~/.hyperdrive/storage/cores
  2. The complete db at ~/.hyperdrive/storage/db

Think it's also important to differentiate between the cases where:

  1. You only want to back up the master key + names, so that you'll be able to restore writability.
  2. You want to back up the master key, names, and also all hypercore data (a full backup). This will likely just be a tarball containing the complete contents of ~/.hyperdrive/storage`

Corestore derives hypercore keys from a single master key and a name file, which is stored inside the hypercore's directory. This means that the backup manifest just needs to be some listing of the form (potentially a JSON file), alongside the database, all bundled together into a tarball:

(master-key)
(core-key1) -> (name1)
(core-key2) -> (name2)
...

Since this command can be implemented through FS operations on the storage directory directly, it should be straightforward to prototype without adding any new API methods.

I'd also suggest the the import command be restore, so that we don't have any naming collisions with the existing import/export commands.