MagicStack / asyncpg

A fast PostgreSQL Database Client Library for Python/asyncio.
Apache License 2.0
6.99k stars 404 forks source link

Asyncpg create_pool doesn't create enough pool connections #983

Closed TimG233 closed 1 year ago

TimG233 commented 1 year ago

So I am using asyncpg as the tool for connecting a discord autosharded-bot and the local database on my ubuntu server. I use bot.pool = await asyncpg.create_pool(user=..., password=..., database=..., host='127.0.0.1', port=..., max_size=10, min_size=10) to create the pool connections (and by default the pool size should be 10). And I use these methods to check the pool connections:

pool.get_size()
pool.get_idle_size()
pool.get_max_size()
pool.get_min_size()
pool._max_inactive_connection_lifetime

And when I test this locally on my windows computer, it works perfectly. The result of pool connections is

Pool size: 10
Idle size: 10
Maximum connection: 10
Minimum connection: 10
Maximum Inactive Connection Lifetime: 300.0

But the strange thing comes when I put the same code on the ubuntu server, it has the following result:

Pool size: 2
Idle size: 2
Maximum connection: 10
Minimum connection: 10
Maximum Inactive Connection Lifetime: 300.0

And sometimes, the pool size and idle size can be both 3 or 4. But it never reaches 10. Also, the maximum and minimum connection in this way is 10. So basically pool connection size is smaller than minimum connection requirement. This really confuses me a lot. These results can be reproduced both locally and on the server side. Any comments or suggestions are welcome. Thank you!

elprans commented 1 year ago

Check if there is anything between the client and the server that could be closing idle connections (e.g a proxy). asyncpg.Pool does not currently actively police pool connections for liveness (though maybe it should).

TimG233 commented 1 year ago

Currently, I didn't find anything (discord.py or discord) that can close idle connections. I didn't set up anything to close idle connections, and I don't use any proxy on this ubuntu server. The max_connections of PostgreSQL shows 100 which means I didn't touch the upperlimit of connections. When checking SELECT * FROM pg_stat_activity (suggested by dpy helper), I found out the number of records that uses the pool's username (postgres) to specific database are 3, which matches the pool_size shown above. But I just wonder why the number is not 10...