search

sripathikrishnan / redis-rdb-tools

Parse Redis dump.rdb files, Analyze Memory, and Export Data to JSON
https://rdbtools.com
MIT License
5.09k stars 744 forks source link

Provide a way to restore a subset of keys into a running redis server from a RDB File #132

Open sripathikrishnan opened 6 years ago

sripathikrishnan commented 6 years ago

Very often, due to application bugs, teams lose or corrupt a subset of data within redis. For example, someone may have accidentally deleted some keys or would have run the wrong command.

There are a few ways to recover data in such cases - a) replaying AOF file after removing the offending command, or b) Restore an older RDB. Replaying the AOF is not always possible, because many cloud providers do not expose the AOF file, and not everyone has AOF enabled. Restoring an older RDB requires developers to spin up a new redis instance and then selectively copy data from the new instance to the existing instance with corrupted data.

This issue proposes another alternative - generating a stream of RESTORE commands. Developers can apply any of the existing filters to select a subset of keys from the RDB file, and then rdbtools will generate appropriate RESTORE commands. We should avoid connecting to redis servers directly. Instead, we should generate a stream of commands, and let the user pipe it to redis server using redis-cli.

Proposed syntax - rdb --c restore /var/redis/6379/dump.rdb

To restore specific key pattern to a redis server running on localhost rdb --c restore --key "users:.*" dump.rdb | redis-cli --pipe

By default, we will fail if the key already exists. We can pass the --force flag to replace existing keys. This will pass the REPLACE flag when invoking the restore command.

sripathikrishnan commented 6 years ago

@oranagra FYI - I'm working on this feature request, should have a pull request in a couple of days

oranagra commented 6 years ago

great.. i was waiting for a chance we'll add that feature to rdbtools. p.s. you may want to take a look at these: https://github.com/happybits/redisimp https://github.com/antirez/redis/pull/5045

pfcarrier commented 5 years ago

Thanks for such a great tool. Just sharing that if one is in needs of an immediate way to restore a subset of keys from an rdb file it can be achieved with this approach 

rdb --command protocol --key "happybits:*" some_dump.rdb | redis-cli --pipe