search

agibralter / ts-resque-delta

ThinkingSphinx + Resque delta indexing.
MIT License
69 stars 36 forks source link

Issue with Redis 3 #19

Closed budu closed 11 years ago

budu commented 12 years ago

Since upgrading to Redis 3, all ts_delta jobs fails with the following error:

Tried to use a connection from a child process without reconnecting. You need to reconnect to Redis after forking.
/home/budu/.rvm/gems/ruby-1.9.3-p125@cine-fete/gems/redis-3.0.1/lib/redis/client.rb:271:in `ensure_connected'
/home/budu/.rvm/gems/ruby-1.9.3-p125@cine-fete/gems/redis-3.0.1/lib/redis/client.rb:167:in `block in process'
/home/budu/.rvm/gems/ruby-1.9.3-p125@cine-fete/gems/redis-3.0.1/lib/redis/client.rb:242:in `logging'
/home/budu/.rvm/gems/ruby-1.9.3-p125@cine-fete/gems/redis-3.0.1/lib/redis/client.rb:166:in `process'
/home/budu/.rvm/gems/ruby-1.9.3-p125@cine-fete/gems/redis-3.0.1/lib/redis/client.rb:78:in `call'
/home/budu/.rvm/gems/ruby-1.9.3-p125@cine-fete/gems/redis-3.0.1/lib/redis.rb:644:in `block in setnx'
/home/budu/.rvm/gems/ruby-1.9.3-p125@cine-fete/gems/redis-3.0.1/lib/redis.rb:36:in `block in synchronize'
/home/budu/.rvm/rubies/ruby-1.9.3-p125/lib/ruby/1.9.1/monitor.rb:211:in `mon_synchronize'
/home/budu/.rvm/gems/ruby-1.9.3-p125@cine-fete/gems/redis-3.0.1/lib/redis.rb:36:in `synchronize'
/home/budu/.rvm/gems/ruby-1.9.3-p125@cine-fete/gems/redis-3.0.1/lib/redis.rb:643:in `setnx'
/home/budu/.rvm/gems/ruby-1.9.3-p125@cine-fete/gems/redis-namespace-1.2.0/lib/redis/namespace.rb:243:in `method_missing'
/home/budu/.rvm/gems/ruby-1.9.3-p125@cine-fete/gems/resque-lock-timeout-0.3.3/lib/resque/plugins/lock_timeout.rb:133:in `acquire_lock_algorithm!'
/home/budu/.rvm/gems/ruby-1.9.3-p125@cine-fete/gems/resque-lock-timeout-0.3.3/lib/resque/plugins/lock_timeout.rb:115:in `acquire_lock!'
/home/budu/.rvm/gems/ruby-1.9.3-p125@cine-fete/gems/resque-lock-timeout-0.3.3/lib/resque/plugins/lock_timeout.rb:171:in `around_perform_lock'
/home/budu/.rvm/gems/ruby-1.9.3-p125@cine-fete/gems/resque-1.15.0/lib/resque/job.rb:135:in `block (2 levels) in perform'
/home/budu/.rvm/gems/ruby-1.9.3-p125@cine-fete/gems/resque-1.15.0/lib/resque/job.rb:147:in `call'
/home/budu/.rvm/gems/ruby-1.9.3-p125@cine-fete/gems/resque-1.15.0/lib/resque/job.rb:147:in `perform'
/home/budu/.rvm/gems/ruby-1.9.3-p125@cine-fete/gems/resque-1.15.0/lib/resque/worker.rb:163:in `perform'
/home/budu/.rvm/gems/ruby-1.9.3-p125@cine-fete/gems/resque-1.15.0/lib/resque/worker.rb:130:in `block in work'
/home/budu/.rvm/gems/ruby-1.9.3-p125@cine-fete/gems/resque-1.15.0/lib/resque/worker.rb:116:in `loop'
/home/budu/.rvm/gems/ruby-1.9.3-p125@cine-fete/gems/resque-1.15.0/lib/resque/worker.rb:116:in `work'
/home/budu/.rvm/gems/ruby-1.9.3-p125@cine-fete/gems/resque-1.15.0/lib/resque/tasks.rb:27:in `block (2 levels) in <top (required)>'
agibralter commented 12 years ago

Ah, unfortunately I'm not too sure about how this works with Redis 3. Right now I'm using Resque 1.20.0 which I believe is not compatible (gem-wise) with Redis 3. Are you using a different version of Resque?

@ryansch may have some ideas...

budu commented 12 years ago

The version of Resque is even older (1.15), I'll probably just downgrade for now, I don't even understand why bundler did that!

ryansch commented 12 years ago

@budu Did upgrading to a more recent redis fix your issue?

rsiegel commented 11 years ago

I'm using Resque 1.22, redid 3.01 and had no problems. After adding rescue-status, all ok. Then I added resque-ui and get the above error. If I run the rake task (queue job) without the rescue-ui gem in gem file, works fine. With the rescue-ui gem, bad-news. Does anyone have an idea why? I'd like to keep using the ui.......

ryansch commented 11 years ago

@rsiegel I'm not familiar with that gem and it's not on rubygems.

rsiegel commented 11 years ago

We found the problem, and it is also mentioned in other places. The essence is that when Resque spawns/forks a process, the connection that existed is still open. However it's not reliable to use one connection for different processes so in the latest versions of Resque there is a test to check if this is the case. That's the error message. The solution is to refresh the connection (which creates a new one) after spawning/forking. This can be done with an after fork callback.

ryansch commented 11 years ago

I'm familiar with that problem. I'm glad you got it figured out!

Closing since we haven't heard from @budu in 5 months.