WARN: ActiveRecord::ConnectionTimeoutError:
could not obtain a connection from the pool within 5.000 seconds (waited 5.000 seconds);
all pooled connections were in use
...
WARN: /opt/mastodon/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.8.4/lib/active_record/connection_adapters/abstract/connection_pool/queue.rb:126:in `block in wait_poll'
...
/opt/mastodon/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.8.4/lib/active_record/connection_adapters/abstract/connection_pool/queue.rb:115:in `loop'
...
/opt/mastodon/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.8.4/lib/active_record/connection_adapters/abstract/connection_pool/queue.rb:115:in `wait_poll'
Even though the error seems self explanatory, there aren't a lot of details online except for this. I increased the DB_POOL env to 10 and the issue still persists.
It's especially frustrating because our connection pool is configured to use 85 connections, and I've never seen it use more than, say, 10 in the "current connections" UI of DOcean. It just now gave that error, and there are currently 6 connections.
Even after putting pgbouncer in between the app stack and the database, the web server and sidekiq are still complaining about the connection pool being empty:
Even though the error seems self explanatory, there aren't a lot of details online except for this. I increased the
DB_POOLenv to 10 and the issue still persists.It's especially frustrating because our connection pool is configured to use 85 connections, and I've never seen it use more than, say, 10 in the "current connections" UI of DOcean. It just now gave that error, and there are currently 6 connections.
This is going to require some more debugging.