search

jeremyfa / node-redis-dump

Dump redis database into redis commands or json with command line or node.js
114 stars 34 forks source link

Out of memory #23

Open Freakachoo opened 5 years ago

Freakachoo commented 5 years ago

On a medium-big DBs it gives an error:

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
jeremyfa commented 5 years ago

Hi, this is due to how the dump is done: using KEYS *, which redis dump rely on, is an expensive and memory inefficient solution, which works very well on small databases, but is not designed to handle the bigger ones.

I would suggest to look at other options to export/backup your redis data (replication, incremental export etc...)

Freakachoo commented 5 years ago

Thank you for suggestions. There are also other dump solutions. I just thought, that maybe would be possible to fix this tool by using SCAN command.

jeremyfa commented 5 years ago

Yes, this can probably be improved using smarter commands, but I don't plan to do it myself as I don't need it personally. I may however review and accept any Pull Request to change that, given that it doesn't create regressions in existing features