nyagato-00 / predictor

Fast and efficient recommendations and predictions using Redis
MIT License
502 stars 41 forks source link

Redis::CommandError: BUSY Redis is busy running a script. You can only call SCRIPT KILL or SHUTDOWN NOSAVE. #41

Closed archseer closed 8 years ago

archseer commented 8 years ago

Hi, we're using this gem in combination with hiredis v0.6.0. We've imported about 100k items previously using add_to_matrix (no bang), and now we're trying to process the entire dataset. With the lua processor, the code runs, then after a minute or so crashes:

[7] pry(main)> recommender.process!
Redis::CommandError: BUSY Redis is busy running a script. You can only call SCRIPT KILL or SHUTDOWN NOSAVE.

Now interestingly, I tried chunking the processing, and it still seems to happen:

[7] pry(main)> recommender.all_items.each_slice(100) {|r| recommender.process_items!(*r); puts "chunk!" }
chunk!
chunk!
Redis::CommandError: BUSY Redis is busy running a script. You can only call SCRIPT KILL or SHUTDOWN NOSAVE.

Possibly related to some sort of a timeout in the redis client?

jhdavids8 commented 8 years ago

@chanks any ideas here?

chanks commented 8 years ago

A timeout sounds like a likely culprit. It looks like the default Redis client's timeout is five seconds and can be overridden with an option, e.g. Redis.new timeout: 30.

Does that fix your issue? If so, we'll probably want to add it to the docs about the lua processing strategy.

archseer commented 8 years ago

Yeah, that seems to have fixed it! Due to the blocking nature, we're probably going to go with union anyhow. I'll close the issue now.

chanks commented 8 years ago

:+1: