search

delano / redis-dump

Redis to JSON and nack again
https://rubygems.org/gems/redis-dump
MIT License
818 stars 123 forks source link

stack level too deep (SystemStackError) #3

Closed newzealandpaul closed 12 years ago

newzealandpaul commented 12 years ago

When dumping a large redis database (1gb+ in size), redis-dump raises an exception after dumping about 262mb of data.

This is the last few lines of output (running with -D):

... 1320554699.0341: dump[-2199019514904509058, {"db"=>0, "key"=>"customer+43573945", "ttl"=>-1, "type"=>"set", "value"=>["1"], "size"=>1}] 1320554699.0341: customer+43573945 (set): 1B /Users/paul/.rvm/gems/ruby-1.9.2-p180/bin/redis-dump:19: stack level too deep (SystemStackError)

delano commented 12 years ago

Thanks for the report. I don't have any time to look into it right now but I'd happily pull in a patch.

delano commented 12 years ago

I wasn't able to reproduce the issue. However, I just pushed a new release (0.3) which includes some memory optimizations, speed improvements, and related changes. There's a new chunk_size option which specifies how many records to process at one time (default is 10,000).