MagicStack / asyncpg

A fast PostgreSQL Database Client Library for Python/asyncio.
Apache License 2.0
6.88k stars 399 forks source link

issue with connection pooling when using QueuePool (via sqlalchemy) #1123

Closed akhilputhiry closed 6 months ago

akhilputhiry commented 6 months ago

We have a web app using sqlalchemy. If we use NullPool (disable connection pooling) everything works fine. The moment we enable QueuePool the app gets stuck

time=2024-02-29 14:48:01,669 level=INFO request-id=b1de3f74dac04a9799934a671cf9e0d6 user-id=2319885380 message=Permission datalake/roles/user granted for user 2319885380
time=2024-02-29 14:48:01,669 level=INFO request-id=b1de3f74dac04a9799934a671cf9e0d6 user-id=2319885380 message=creating database session
time=2024-02-29 14:48:01,669 level=INFO request-id=b1de3f74dac04a9799934a671cf9e0d6 user-id=2319885380 message=database session created successfully
time=2024-02-29 14:48:01,669 level=INFO request-id=b1de3f74dac04a9799934a671cf9e0d6 user-id=2319885380 message=Fetching datalake environment
2024-02-29 14:48:01,677 DEBUG sqlalchemy.pool.impl.QueuePool Created new connection <AdaptedConnection <asyncpg.connection.Connection object at 0x114136300>>
2024-02-29 14:48:01,680 DEBUG sqlalchemy.pool.impl.QueuePool Created new connection <AdaptedConnection <asyncpg.connection.Connection object at 0x1141365d0>>

[2024-02-29 14:52:27 +0530] [61517] [CRITICAL] WORKER TIMEOUT (pid:61518)
[2024-02-29 14:52:27 +0530] [61517] [ERROR] Worker (pid:61518) was sent SIGABRT!
[2024-02-29 14:52:27 +0530] [62957] [INFO] Booting worker with pid: 62957
[2024-02-29 14:52:42 +0530] [62957] [INFO] Started server process [62957]
[2024-02-29 14:52:42 +0530] [62957] [INFO] Waiting for application startup.
elprans commented 6 months ago

Should this be an SQLAlchemy bug instead?

akhilputhiry commented 6 months ago

Looks we misconfigured the sqlalchemy to use QueuePool instead of AsyncAdaptedQueuePool which was causing the issue