I have my Redis calls instrumented in production with Honeycomb's Ruby beeline, which has pointed to an interesting issue with the reconnect logic.
We're using the synchrony driver, which I can completely believe results in this particular behavior, but I can't be sure. :expressionless: I'm in support of #915, but I'm stuck with EM for the time being.
My code simply does an MGET to fetch an array of items that it converts to a set:
redis.mget(*keys).compact.to_set
From this line, we're seeing errors in production for
NoMethodError: undefined method `compact' for "OK":String
This doesn't happen all the time, but it does happen with some frequency.
On the relevant traces in Honeycomb, we can see that this results from the MGET calls triggering AUTH calls when redis-rb reconnects. Somehow, the AUTH's return value of "OK" is being returned as the output of Redis#mget.
This doesn't always happen on reconnect, though. Many calls return the MGET data just fine, even with an AUTH. But when this error does happen, it always seems to have an AUTH triggered. So I think it might be a strange race condition or something.
The only thing I can think to try is using the block form of Redis#mget to see if the value yielded there is any different from the ultimate return value of Redis::Client#call...? Not really convinced that'll do anything though.
I have my Redis calls instrumented in production with Honeycomb's Ruby beeline, which has pointed to an interesting issue with the reconnect logic.
We're using the synchrony driver, which I can completely believe results in this particular behavior, but I can't be sure. :expressionless: I'm in support of #915, but I'm stuck with EM for the time being.
Relevant versions from my Gemfile.lock:
My code simply does an
MGET
to fetch an array of items that it converts to a set:From this line, we're seeing errors in production for
This doesn't happen all the time, but it does happen with some frequency.
On the relevant traces in Honeycomb, we can see that this results from the
MGET
calls triggeringAUTH
calls when redis-rb reconnects. Somehow, theAUTH
's return value of"OK"
is being returned as the output ofRedis#mget
.This doesn't always happen on reconnect, though. Many calls return the
MGET
data just fine, even with anAUTH
. But when this error does happen, it always seems to have anAUTH
triggered. So I think it might be a strange race condition or something.The only thing I can think to try is using the block form of
Redis#mget
to see if the value yielded there is any different from the ultimate return value ofRedis::Client#call
...? Not really convinced that'll do anything though.