search

Level / awesome

An open list of awesome Level modules and resources.
MIT License
289 stars 22 forks source link

Advice on taking backup? #20

Open trygve-lie opened 10 years ago

trygve-lie commented 10 years ago

How do you guys and girls take backup of your level db data?

Do you (A) just copy out the files level create on disc or do you (b) implement a export feature which dumps all data in the database to file for later import?

If A: How does this work with having the db open by the running node program? Do you close down, backup and then open the database again or do you just copy out files wile running?

If B: Do you export all data every time or do you do diffs? If the database is large, how much would a full export slow down the other operations?

Or do you simply do more advanced stuff where you write to multiple databases on different machines (in different data centers) and look at that as backup?

Maybe we are missing some backup modules for level. At least there should be some advice on how to backup / avoid data loss in the documentation I think.

dominictarr commented 10 years ago

I havn't really done this my self because the stuff that I happen to have done could all be rebuilt from scratch, but I think copying all the files would work, or you could use a data replication module.

dominictarr commented 10 years ago

there could be some edge cases around copying the files if a compaction is currently underway... but I think this wouldn't be a problem if you snapshot the files (fs.readdir) and then copy those. especially if you copy the manifest file, the sst files, and then the log files.

ralphtheninja commented 9 years ago

@trygve-lie You can use level-hyper and do liveBackup(). Currently this is poorly documented, but I'm working on fixing that. Check level-test for an example of what this might look like. So the idea with liveBackup() is to be able to create backups without needing to close the underlying db.

ralphtheninja commented 7 years ago

Maybe we are missing some backup modules for level. At least there should be some advice on how to backup / avoid data loss in the documentation I think.

Amen.