MagicStack / asyncpg

A fast PostgreSQL Database Client Library for Python/asyncio.
Apache License 2.0
6.88k stars 399 forks source link

Delay opening of connection pool #1037

Closed interrogator closed 11 months ago

interrogator commented 1 year ago

My app uses rq, which spawns worker processes via os.fork(). I have a fairly typical use-case: I want a connection pool to be shared by all workers and connections acquired by the main app from time to time. But for this to work, the connection needs to be opened after the fork takes place, or else all kinds of errors spring up when two different workers access the same connection.

With psycopg's AsyncConnectionPool, there is an open() async method that can be run after forking. Calling it again subsequently does nothing, which is fine. With asyncpg however there doesn't seem to be such a simple interface. Instead I needed to set _initialized to False, call _initialize() or _async__init_() or similar. The logics for instantiating the pool and opening it are somewhat bound together.

It would be great if the connection pool had the ability to delay opening connection till something like open() is called. Right now, short or just recreating the entire pool in each worker process, I can't find a way of making it work.

Any plans to add something like this? Or at least, an explanation of how this would best be accomplished with the current version?

Thanks for your work on asyncpg!

elprans commented 1 year ago

Asyncpg's equivalent of open is await-ing on it (or using it as an async context manager), so if you want to delay the initialization, simply delay the await part, e.g:

pool_config = asyncpg.Pool(...)

...
# some time after
pool = await pool_config
# or
async with pool_config as pool:
   ...

Does that work for you?