Open thentgesMindee opened 1 year ago
I could not reproduce this on my end using just asyncpg (without SQLAlchemy). Here's the compose file I used:
services:
postgresdb:
image: postgres:12
networks:
proxy_network:
aliases:
- postgresdb_host
env_file:
- db.env
ports:
- 3333:5432
app:
build:
context: .
ports:
- 1234:8383
networks:
proxy_network:
aliases:
- app_two_host
restart: on-failure
entrypoint: bash -c "python /usr/local/bin/test_bug1030.py"
depends_on:
- postgresdb
networks:
proxy_network:
driver: bridge
test_bug1030.py:
import asyncio
import asyncpg
async def main():
con = await asyncpg.connect('postgresql://postgres:foo@postgresdb_host:5432/postgres')
print(await con.fetchval('select 1'))
await con.close()
asyncio.run(main())
It seems that asyncpg is not working correctly when using an host defined using a
bridge
network in docker. I am actually not sure the error comes from asyncpg or docker, any idea here could be really helpful. Opening any new connection using an asyncpg connection string will result in the following errorTypeError: cannot unpack non-iterable NoneType object
, but only when using a network alias from a docker bridge network.More Details + how to reproduce
I have a docker-compose.yml file looking like: (simplified for the sake of this issue)
Where
app_one
is a Flask application using SQLAlchemy and psycopg2 (synchronous session, no asyncpg there), andapp_two
is a fastAPI application using SQLAlchemy and asyncpg.postgresql://postgres:postgres@postgresdb_host:5432/my_db_name
as an URI for the Flask application, everything is working well, meaning myproxy_network
is configured correctly betweenapp_one
andpostgresdb
(and db is properly running)app_two
outside of docker, but still runpostgresdb
container, I can properly usepostgresql+asyncpg://postgres:postgres@127.0.0.1:3333/my_db_name
and it works. (external host / ports), meaning the database container is working correctlyapp_two
container, to an endpoint ofapp_one
, usinghttp://app_one_host:5000
as URI, it works correctly, meaningapp_two
andapp_one
are also properly linked to the bridge network.However, using
postgresql+asyncpg://postgres:postgres@postgresdb_host:5432/my_db_name
as my database URI from within the docker container ofapp_two
, here is the error I get when trying to open any new connection:see full error stacktrace:
``` File "/app/app/db/session.py", line 85, in check_dbs async with self.engines[bind].begin() as connection: File "/usr/local/lib/python3.8/site-packages/sqlalchemy/ext/asyncio/base.py", line 66, in __aenter__ return await self.start(is_ctxmanager=True) File "/usr/local/lib/python3.8/site-packages/sqlalchemy/ext/asyncio/engine.py", line 599, in start await self.conn.start(is_ctxmanager=is_ctxmanager) File "/usr/local/lib/python3.8/site-packages/sqlalchemy/ext/asyncio/engine.py", line 157, in start await (greenlet_spawn(self.sync_engine.connect)) File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 126, in greenlet_spawn result = context.throw(*sys.exc_info()) File "/usr/local/lib/python3.8/site-packages/sqlalchemy/future/engine.py", line 406, in connect return super(Engine, self).connect() File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 3320, in connect return self._connection_cls(self, close_with_result=close_with_result) File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 96, in __init__ else engine.raw_connection() File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 3399, in raw_connection return self._wrap_pool_connect(self.pool.connect, _connection) File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 3366, in _wrap_pool_connect return fn() File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 327, in connect return _ConnectionFairy._checkout(self) File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 894, in _checkout fairy = _ConnectionRecord.checkout(pool) File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 493, in checkout rec = pool._do_get() File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/impl.py", line 146, in _do_get self._dec_overflow() File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ compat.raise_( File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 211, in raise_ raise exception File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/impl.py", line 143, in _do_get return self._create_connection() File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 273, in _create_connection return _ConnectionRecord(self) File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 388, in __init__ self.__connect() File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 691, in __connect pool.logger.debug("Error on connect(): %s", e) File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ compat.raise_( File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 211, in raise_ raise exception File "/usr/local/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 686, in __connect self.dbapi_connection = connection = pool._invoke_creator(self) File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/create.py", line 574, in connect return dialect.connect(*cargs, **cparams) File "/usr/local/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 598, in connect return self.dbapi.connect(*cargs, **cparams) File "/usr/local/lib/python3.8/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 780, in connect await_only(self.asyncpg.connect(*arg, **kw)), File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 68, in await_only return current.driver.switch(awaitable) File "/usr/local/lib/python3.8/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 121, in greenlet_spawn value = await result File "/usr/local/lib/python3.8/site-packages/asyncpg/connection.py", line 2092, in connect return await connect_utils._connect( File "/usr/local/lib/python3.8/site-packages/asyncpg/connect_utils.py", line 881, in _connect return await _connect_addr( File "/usr/local/lib/python3.8/site-packages/asyncpg/connect_utils.py", line 773, in _connect_addr return await __connect_addr(params, timeout, True, *args) File "/usr/local/lib/python3.8/site-packages/asyncpg/connect_utils.py", line 825, in __connect_addr tr, pr = await compat.wait_for(connector, timeout=timeout) TypeError: cannot unpack non-iterable NoneType object ```With all those experiments I did, I cannot find what is the issue, except that it happens only when using asyncpg + bridge network. Bridge network with
psycopg2
works fine, as well asasyncpg
without bridge network hosts. Combining both fails.I tried to add as much info as possible on my issue and how to reproduce, but feel free to ask anything I may have forgotten!