redis / redis-rb

A Ruby client library for Redis
MIT License
3.96k stars 1.03k forks source link

Max number clients reached using subscribe_with_timeout #1259

Closed gap777 closed 5 months ago

gap777 commented 5 months ago

This is a Rails 7 app using REDIS for pubsub, hosted on Heroku ("Heroku Data for Redis" add on). This also happens locally (when I set the maxclients down to 20, like is set for the Heroku instance)

Our code sets up REDIS like this:

redis = Redis.new(url: ENV.fetch('REDIS_URL', nil), ssl_params: { verify_mode: OpenSSL::SSL::VERIFY_NONE })
pool = ConnectionPool.new(size: 5, timeout: 5) { redis }

We listen for pubsub like this:

  trap(:TERM) { shutdown }
  pool.with do |conn|
     conn.subscribe(@stream) do |listener|
        register_subscription_callback(listener)
        register_message_callback(listener)
        register_unsubscription_callback(listener)
     end
 end

I want to allow the caller to abort the listening, so I'm trying to change it like this:

  pool.with do |conn|
     until abort_signal.set?
        @logger.debug("listening for pubsub events for 5 seconds")
        conn.subscribe_with_timeout(5, @stream) do |listener|
          register_subscription_callback(listener)
          register_message_callback(listener)
          register_unsubscription_callback(listener)
        end
     end
  end

However, after 20 loops (on the 21st attempt), I get an error

RedisClient::CommandError: ERR max number of clients reached (RedisClient::CommandError)

Alternatively, I've tried putting my while loop above the withdrawal from the connection pool:


 until abort_signal.set?
    pool.with do |conn|
        @logger.debug("listening for pubsub events for 5 seconds")
        conn.subscribe_with_timeout(5, @stream) do |listener|
          register_subscription_callback(listener)
          register_message_callback(listener)
          register_unsubscription_callback(listener)
        end
    end
 end

I've also tried removing the connection pool altogether, but still experience the issue.
I've also tried just using the recipe from your README

redis = Redis.new(reconnect_attempts: 0)
while true
  redis.subscribe_with_timeout(5, "news") do |on|
    on.message do |channel, message|
      # ...
    end
  end
end

and still experienced the issue.

Can you help me to get an interruptable subscribe for PUBSUB events? If via subscribe_with_timeout, what is the underlying reason for my error?

Thank you

byroot commented 5 months ago

This was a bug fixed in #1260. You can point your gemfile at the repo and your code should now work as intended.

gap777 commented 5 months ago

Super! Thanks for the quick turnaround. Will you be releasing a new version of the gem anytime soon?