Errno::ETIMEDOUT

[dpep]

I'm seeing Errno::ETIMEDOUT occasionally being raised in production for Redis.evalsha calls (apparently due to openssl timing out). The Redis client catches a bunch of system errors and converts them into Redis::ConnectionError, but does not have similar logic in place for timeouts (it only handles TimeoutError)

stack trace

Errno::ETIMEDOUT: Connection timed out
/usr/local/lib/ruby/3.0.0/openssl/buffering.rb:406:in `syswrite_nonblock': Connection timed out (Errno::ETIMEDOUT)

from /usr/local/lib/ruby/3.0.0/openssl/buffering.rb:406:in `write_nonblock'

from /usr/local/bundle/ruby/3.0.0/gems/redis-4.8.1/lib/redis/connection/ruby.rb:78:in `block in write'

from /usr/local/bundle/ruby/3.0.0/gems/redis-4.8.1/lib/redis/connection/ruby.rb:77:in `loop'

from /usr/local/bundle/ruby/3.0.0/gems/redis-4.8.1/lib/redis/connection/ruby.rb:77:in `write'

from /usr/local/bundle/ruby/3.0.0/gems/redis-4.8.1/lib/redis/connection/ruby.rb:378:in `write'

from /usr/local/bundle/ruby/3.0.0/gems/redis-4.8.1/lib/redis/client.rb:320:in `block in write'

from /usr/local/bundle/ruby/3.0.0/gems/redis-4.8.1/lib/redis/client.rb:299:in `io'

from /usr/local/bundle/ruby/3.0.0/gems/redis-4.8.1/lib/redis/client.rb:318:in `write'

from /usr/local/bundle/ruby/3.0.0/gems/redis-4.8.1/lib/redis/client.rb:276:in `block (3 levels) in process'

from /usr/local/bundle/ruby/3.0.0/gems/redis-4.8.1/lib/redis/client.rb:270:in `each'

from /usr/local/bundle/ruby/3.0.0/gems/redis-4.8.1/lib/redis/client.rb:270:in `block (2 levels) in process'

from /usr/local/bundle/ruby/3.0.0/gems/redis-4.8.1/lib/redis/client.rb:411:in `ensure_connected'

from /usr/local/bundle/ruby/3.0.0/gems/redis-4.8.1/lib/redis/client.rb:269:in `block in process'

from /usr/local/bundle/ruby/3.0.0/gems/redis-4.8.1/lib/redis/client.rb:356:in `logging'

from /usr/local/bundle/ruby/3.0.0/gems/redis-4.8.1/lib/redis/client.rb:268:in `process'

from /usr/local/bundle/ruby/3.0.0/gems/redis-4.8.1/lib/redis/client.rb:161:in `call'

from /usr/local/bundle/ruby/3.0.0/gems/ddtrace-0.54.2/lib/ddtrace/contrib/redis/instrumentation.rb:28:in `block in call'

from /usr/local/bundle/ruby/3.0.0/gems/ddtrace-0.54.2/lib/ddtrace/tracer.rb:283:in `trace'

from /usr/local/bundle/ruby/3.0.0/gems/ddtrace-0.54.2/lib/ddtrace/contrib/redis/instrumentation.rb:22:in `call'

from /usr/local/bundle/ruby/3.0.0/gems/redis-4.8.1/lib/redis.rb:270:in `block in send_command'

from /usr/local/bundle/ruby/3.0.0/gems/redis-4.8.1/lib/redis.rb:269:in `synchronize'

from /usr/local/bundle/ruby/3.0.0/gems/redis-4.8.1/lib/redis.rb:269:in `send_command'

from /usr/local/bundle/ruby/3.0.0/gems/redis-4.8.1/lib/redis/commands/scripting.rb:110:in `_eval'

from /usr/local/bundle/ruby/3.0.0/gems/redis-4.8.1/lib/redis/commands/scripting.rb:97:in `evalsha'

[byroot]

Thanks but redis-4.x is is no longer maintained.

[dpep]

bummer...we are trying to upgrade, but currently blocked by a Sidekiq requirement 😞

[sbiswas-chime]

@byroot would it be possible for you to open up a patch for redis-4.x with this fix ? As @dpep mentioned, our redis upgrade is blocked by a Sidekiq requirement.

[byroot]

Why not just point your Gemfile at your branch?

[dpep]

security doesn't like branches in Gemfiles...we need to be more legit 🤣

[byroot]

Shouldn't be hard to monkey patch.