search

cockroachdb / pebble

RocksDB/LevelDB inspired key-value database in Go
BSD 3-Clause "New" or "Revised" License
4.86k stars 449 forks source link

docs: document options for implementing backups #2993

Open cristaloleg opened 1 year ago

cristaloleg commented 1 year ago

Hi, for a long time, README has stated that backups are not supported compared to RocksDB.

How can backups be implemented on the user's side?

The most straightforward solution is to make a snapshot, iterate over the keys and save them into a separate file (whatever serialization format). To restore: open the file from the previous step, iterate over entries and do db.Put for every key.

Is there a better approach? More correct? Ready to use?

Thanks in advance.

Jira issue: PEBBLE-72

jbowens commented 1 year ago

Some alternatives you might want to consider:

  1. Do exactly as you said, but using Pebble's sstable format as the serialization format. See sstable.Writer. Then, when restoring, Ingest the sstables to link them into the LSM without needing to rewrite the keys. This is how CockroachDB's backup and restore works.
  2. Use DB.Checkpoint with the WithFlushedWAL option to create a checkpoint in another directory. The checkpoint directory is your backup and may be copied elsewhere and then removed from the local filesystem. To restore, copy the directory and all its contents into place and Open the database.
cristaloleg commented 1 year ago

Thank you for the quick answer!

I like the 1st approach 'cause it's proven b CockroachDB.

Feel free to close the issue if it's no-op and Pebble team isn't interested in documenting this somehow.