Redis::CommandError - CROSSSLOT Keys in request don't hash to the same slot

glensc commented 4 years ago

[2020-09-10 11:30:58] INFO  WEBrick 1.4.2
[2020-09-10 11:30:58] INFO  ruby 2.5.8 (2020-03-31) [x86_64-linux-musl]
[2020-09-10 11:30:58] INFO  WEBrick::HTTPServer#start: pid=1 port=9292
2020-09-10 11:30:59 - Redis::CommandError - CROSSSLOT Keys in request don't hash to the same slot:
        /usr/local/bundle/gems/redis-4.0.3/lib/redis/client.rb:124:in `call'
        /usr/local/bundle/gems/redis-4.0.3/lib/redis/cluster.rb:215:in `public_send'
        /usr/local/bundle/gems/redis-4.0.3/lib/redis/cluster.rb:215:in `try_send'
        /usr/local/bundle/gems/redis-4.0.3/lib/redis/cluster.rb:151:in `send_command'
        /usr/local/bundle/gems/redis-4.0.3/lib/redis/cluster.rb:72:in `call'
        /usr/local/bundle/gems/redis-4.0.3/lib/redis.rb:918:in `block in mget'
        /usr/local/bundle/gems/redis-4.0.3/lib/redis.rb:50:in `block in synchronize'
        /usr/local/lib/ruby/2.5.0/monitor.rb:235:in `mon_synchronize'
        /usr/local/bundle/gems/redis-4.0.3/lib/redis.rb:50:in `synchronize'
        /usr/local/bundle/gems/redis-4.0.3/lib/redis.rb:917:in `mget'
        /usr/local/bundle/gems/redis-4.0.3/lib/redis.rb:933:in `mapped_mget'
        /usr/local/bundle/gems/redis-namespace-1.6.0/lib/redis/namespace.rb:442:in `call_with_namespace'
        /usr/local/bundle/gems/redis-namespace-1.6.0/lib/redis/namespace.rb:328:in `method_missing'
        /usr/local/bundle/gems/resque-1.27.4/lib/resque/data_store.rb:243:in `workers_map'
        /usr/local/bundle/gems/resque-1.27.4/lib/resque/worker.rb:84:in `working'
        /usr/local/bundle/gems/resque-1.27.4/lib/resque.rb:469:in `working'
        /usr/local/bundle/gems/resque-1.27.4/lib/resque/server/views/working.erb:31:in `block in singleton class'
        /usr/local/bundle/gems/resque-1.27.4/lib/resque/server/views/working.erb:-6:in `instance_eval'
        /usr/local/bundle/gems/resque-1.27.4/lib/resque/server/views/working.erb:-6:in `singleton class'
        /usr/local/bundle/gems/resque-1.27.4/lib/resque/server/views/working.erb:-9:in `__tilt_47047493122780'
        /usr/local/bundle/gems/tilt-2.0.9/lib/tilt/template.rb:170:in `call'
        /usr/local/bundle/gems/tilt-2.0.9/lib/tilt/template.rb:170:in `evaluate'
        /usr/local/bundle/gems/tilt-2.0.9/lib/tilt/template.rb:109:in `render'
        /usr/local/bundle/gems/sinatra-2.0.4/lib/sinatra/base.rb:839:in `render'
        /usr/local/bundle/gems/sinatra-2.0.4/lib/sinatra/base.rb:682:in `erb'
        /usr/local/bundle/gems/resque-1.27.4/lib/resque/server.rb:128:in `partial'
        /usr/local/bundle/gems/resque-1.27.4/lib/resque/server/views/overview.erb:3:in `block in singleton class'
        /usr/local/bundle/gems/resque-1.27.4/lib/resque/server/views/overview.erb:-6:in `instance_eval'
        /usr/local/bundle/gems/resque-1.27.4/lib/resque/server/views/overview.erb:-6:in `singleton class'
        /usr/local/bundle/gems/resque-1.27.4/lib/resque/server/views/overview.erb:-9:in `__tilt_47047493122780'
        /usr/local/bundle/gems/tilt-2.0.9/lib/tilt/template.rb:170:in `call'
        /usr/local/bundle/gems/tilt-2.0.9/lib/tilt/template.rb:170:in `evaluate'
        /usr/local/bundle/gems/tilt-2.0.9/lib/tilt/template.rb:109:in `render'
        /usr/local/bundle/gems/sinatra-2.0.4/lib/sinatra/base.rb:839:in `render'
        /usr/local/bundle/gems/sinatra-2.0.4/lib/sinatra/base.rb:682:in `erb'
        /usr/local/bundle/gems/resque-1.27.4/lib/resque/server.rb:147:in `show'
        /usr/local/bundle/gems/resque-1.27.4/lib/resque/server.rb:176:in `block (2 levels) in <class:Server>'
        /usr/local/bundle/gems/sinatra-2.0.4/lib/sinatra/base.rb:1635:in `call'
        /usr/local/bundle/gems/sinatra-2.0.4/lib/sinatra/base.rb:1635:in `block in compile!'
        /usr/local/bundle/gems/sinatra-2.0.4/lib/sinatra/base.rb:992:in `block (3 levels) in route!'
        /usr/local/bundle/gems/sinatra-2.0.4/lib/sinatra/base.rb:1011:in `route_eval'
        /usr/local/bundle/gems/sinatra-2.0.4/lib/sinatra/base.rb:992:in `block (2 levels) in route!'
        /usr/local/bundle/gems/sinatra-2.0.4/lib/sinatra/base.rb:1040:in `block in process_route'
        /usr/local/bundle/gems/sinatra-2.0.4/lib/sinatra/base.rb:1038:in `catch'
        /usr/local/bundle/gems/sinatra-2.0.4/lib/sinatra/base.rb:1038:in `process_route'
        /usr/local/bundle/gems/sinatra-2.0.4/lib/sinatra/base.rb:990:in `block in route!'
        /usr/local/bundle/gems/sinatra-2.0.4/lib/sinatra/base.rb:989:in `each'
        /usr/local/bundle/gems/sinatra-2.0.4/lib/sinatra/base.rb:989:in `route!'
        /usr/local/bundle/gems/sinatra-2.0.4/lib/sinatra/base.rb:1097:in `block in dispatch!'
        /usr/local/bundle/gems/sinatra-2.0.4/lib/sinatra/base.rb:1076:in `block in invoke'
        /usr/local/bundle/gems/sinatra-2.0.4/lib/sinatra/base.rb:1076:in `catch'
        /usr/local/bundle/gems/sinatra-2.0.4/lib/sinatra/base.rb:1076:in `invoke'
        /usr/local/bundle/gems/sinatra-2.0.4/lib/sinatra/base.rb:1094:in `dispatch!'
        /usr/local/bundle/gems/sinatra-2.0.4/lib/sinatra/base.rb:924:in `block in call!'
        /usr/local/bundle/gems/sinatra-2.0.4/lib/sinatra/base.rb:1076:in `block in invoke'
        /usr/local/bundle/gems/sinatra-2.0.4/lib/sinatra/base.rb:1076:in `catch'
        /usr/local/bundle/gems/sinatra-2.0.4/lib/sinatra/base.rb:1076:in `invoke'
        /usr/local/bundle/gems/sinatra-2.0.4/lib/sinatra/base.rb:924:in `call!'
        /usr/local/bundle/gems/sinatra-2.0.4/lib/sinatra/base.rb:913:in `call'
        /usr/local/bundle/gems/rack-protection-2.0.4/lib/rack/protection/xss_header.rb:18:in `call'
        /usr/local/bundle/gems/rack-protection-2.0.4/lib/rack/protection/path_traversal.rb:16:in `call'
        /usr/local/bundle/gems/rack-protection-2.0.4/lib/rack/protection/json_csrf.rb:26:in `call'
        /usr/local/bundle/gems/rack-protection-2.0.4/lib/rack/protection/base.rb:50:in `call'
        /usr/local/bundle/gems/rack-protection-2.0.4/lib/rack/protection/base.rb:50:in `call'
        /usr/local/bundle/gems/rack-protection-2.0.4/lib/rack/protection/frame_options.rb:31:in `call'
        /usr/local/bundle/gems/rack-2.0.8/lib/rack/null_logger.rb:9:in `call'
        /usr/local/bundle/gems/rack-2.0.8/lib/rack/head.rb:12:in `call'
        /usr/local/bundle/gems/sinatra-2.0.4/lib/sinatra/show_exceptions.rb:22:in `call'
        /usr/local/bundle/gems/sinatra-2.0.4/lib/sinatra/base.rb:194:in `call'
        /usr/local/bundle/gems/sinatra-2.0.4/lib/sinatra/base.rb:1957:in `call'
        /usr/local/bundle/gems/rack-2.0.8/lib/rack/urlmap.rb:68:in `block in call'
        /usr/local/bundle/gems/rack-2.0.8/lib/rack/urlmap.rb:53:in `each'
        /usr/local/bundle/gems/rack-2.0.8/lib/rack/urlmap.rb:53:in `call'
        /usr/local/bundle/gems/rack-2.0.8/lib/rack/tempfile_reaper.rb:15:in `call'
        /usr/local/bundle/gems/rack-2.0.8/lib/rack/lint.rb:49:in `_call'
        /usr/local/bundle/gems/rack-2.0.8/lib/rack/lint.rb:37:in `call'
        /usr/local/bundle/gems/rack-2.0.8/lib/rack/show_exceptions.rb:23:in `call'
        /usr/local/bundle/gems/rack-2.0.8/lib/rack/common_logger.rb:33:in `call'
        /usr/local/bundle/gems/sinatra-2.0.4/lib/sinatra/base.rb:231:in `call'
        /usr/local/bundle/gems/rack-2.0.8/lib/rack/chunked.rb:54:in `call'
        /usr/local/bundle/gems/rack-2.0.8/lib/rack/content_length.rb:15:in `call'
        /usr/local/bundle/gems/rack-2.0.8/lib/rack/handler/webrick.rb:86:in `service'
        /usr/local/lib/ruby/2.5.0/webrick/httpserver.rb:140:in `service'
        /usr/local/lib/ruby/2.5.0/webrick/httpserver.rb:96:in `run'
        /usr/local/lib/ruby/2.5.0/webrick/server.rb:307:in `block in start_thread'

glensc commented 4 years ago

Background: monitoring a Redis cluster what is filled with jobs by node resque.

x-ref:

https://github.com/actionhero/node-resque/pull/438#issuecomment-688949623

mlh758 commented 4 years ago

It seems the error isn't directly from the library but a complaint from Redis itself. Stack Overflow. It seems like the requested keys (worker ids) requested in the command aren't coming from a single node.

This is the function actually requesting the values. If that is raising this error then it seems the node_for function isn't getting the expected value. The first step would probably be updating the latest version of the redis gem you can if you haven't already.

glensc commented 4 years ago

the same problem persists with redis-4.2.2 gem.

resque / resque-web

Redis::CommandError - CROSSSLOT Keys in request don't hash to the same slot #154