DB issues

[apolkingg8]

Got DB connection error when testing the api-dispute app:

ERROR:    Exception in ASGI application

Traceback (most recent call last):

  File "/usr/src/apps/api-dispute/.venv/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 3240, in _wrap_pool_connect

    return fn()

  File "/usr/src/apps/api-dispute/.venv/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 310, in connect

    return _ConnectionFairy._checkout(self)

  File "/usr/src/apps/api-dispute/.venv/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 868, in _checkout

    fairy = _ConnectionRecord.checkout(pool)

  File "/usr/src/apps/api-dispute/.venv/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 476, in checkout

    rec = pool._do_get()

  File "/usr/src/apps/api-dispute/.venv/lib/python3.9/site-packages/sqlalchemy/pool/impl.py", line 146, in _do_get

    self._dec_overflow()

  File "/usr/src/apps/api-dispute/.venv/lib/python3.9/site-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__

    compat.raise_(

  File "/usr/src/apps/api-dispute/.venv/lib/python3.9/site-packages/sqlalchemy/util/compat.py", line 207, in raise_

    raise exception

  File "/usr/src/apps/api-dispute/.venv/lib/python3.9/site-packages/sqlalchemy/pool/impl.py", line 143, in _do_get

    return self._create_connection()

  File "/usr/src/apps/api-dispute/.venv/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 256, in _create_connection

    return _ConnectionRecord(self)

  File "/usr/src/apps/api-dispute/.venv/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 371, in __init__

    self.__connect()

  File "/usr/src/apps/api-dispute/.venv/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 666, in __connect

    pool.logger.debug("Error on connect(): %s", e)

  File "/usr/src/apps/api-dispute/.venv/lib/python3.9/site-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__

    compat.raise_(

  File "/usr/src/apps/api-dispute/.venv/lib/python3.9/site-packages/sqlalchemy/util/compat.py", line 207, in raise_

    raise exception

  File "/usr/src/apps/api-dispute/.venv/lib/python3.9/site-packages/sqlalchemy/pool/base.py", line 661, in __connect

    self.dbapi_connection = connection = pool._invoke_creator(self)

  File "/usr/src/apps/api-dispute/.venv/lib/python3.9/site-packages/sqlalchemy/engine/create.py", line 590, in connect

    return dialect.connect(*cargs, **cparams)

  File "/usr/src/apps/api-dispute/.venv/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 584, in connect

    return self.dbapi.connect(*cargs, **cparams)

  File "/usr/src/apps/api-dispute/.venv/lib/python3.9/site-packages/psycopg2/__init__.py", line 122, in connect

    conn = _connect(dsn, connection_factory=connection_factory, **kwasync)

psycopg2.OperationalError: could not connect to server: Connection refused

Is the server running on host "localhost" (127.0.0.1) and accepting

TCP/IP connections on port 5432?

could not connect to server: Address not available

Is the server running on host "localhost" (::1) and accepting

TCP/IP connections on port 5432?

I'm pretty sure the 127.0.0.1:5432 works fine because I can connect it with other applications. If I replace the POSTGRES_SERVER=localhost with POSTGRES_SERVER=host.docker.internal the connection would be fixed, but it causes another issue about DB init:

Traceback (most recent call last):

  File "/usr/src/apps/api-dispute/.venv/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1799, in _execute_context

    self.dialect.do_execute(

  File "/usr/src/apps/api-dispute/.venv/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 717, in do_execute

    cursor.execute(statement, parameters)

psycopg2.errors.UndefinedTable: relation "events" does not exist

LINE 1: INSERT INTO events (message_id, type, party1, party2, contra...

                    ^

The above exception was the direct cause of the following exception:

Traceback (most recent call last):

  File "/usr/src/apps/api-dispute/.venv/lib/python3.9/site-packages/uvicorn/protocols/http/h11_impl.py", line 373, in run_asgi

    result = await app(self.scope, self.receive, self.send)

  File "/usr/src/apps/api-dispute/.venv/lib/python3.9/site-packages/uvicorn/middleware/proxy_headers.py", line 75, in __call__

    return await self.app(scope, receive, send)

  File "/usr/src/apps/api-dispute/.venv/lib/python3.9/site-packages/fastapi/applications.py", line 208, in __call__

    await super().__call__(scope, receive, send)

  File "/usr/src/apps/api-dispute/.venv/lib/python3.9/site-packages/starlette/applications.py", line 112, in __call__

    await self.middleware_stack(scope, receive, send)

  File "/usr/src/apps/api-dispute/.venv/lib/python3.9/site-packages/starlette/middleware/errors.py", line 181, in __call__

    raise exc

  File "/usr/src/apps/api-dispute/.venv/lib/python3.9/site-packages/starlette/middleware/errors.py", line 159, in __call__

    await self.app(scope, receive, _send)

  File "/usr/src/apps/api-dispute/.venv/lib/python3.9/site-packages/starlette/exceptions.py", line 82, in __call__

    raise exc

  File "/usr/src/apps/api-dispute/.venv/lib/python3.9/site-packages/starlette/exceptions.py", line 71, in __call__

    await self.app(scope, receive, sender)

  File "/usr/src/apps/api-dispute/.venv/lib/python3.9/site-packages/starlette/routing.py", line 656, in __call__

    await route.handle(scope, receive, send)

  File "/usr/src/apps/api-dispute/.venv/lib/python3.9/site-packages/starlette/routing.py", line 259, in handle

    await self.app(scope, receive, send)

  File "/usr/src/apps/api-dispute/.venv/lib/python3.9/site-packages/starlette/routing.py", line 61, in app

    response = await func(request)

  File "/usr/src/apps/api-dispute/.venv/lib/python3.9/site-packages/fastapi/routing.py", line 226, in app

    raw_response = await run_endpoint_function(

  File "/usr/src/apps/api-dispute/.venv/lib/python3.9/site-packages/fastapi/routing.py", line 161, in run_endpoint_function

    return await run_in_threadpool(dependant.call, **values)

  File "/usr/src/apps/api-dispute/.venv/lib/python3.9/site-packages/starlette/concurrency.py", line 39, in run_in_threadpool

    return await anyio.to_thread.run_sync(func, *args)

  File "/usr/src/apps/api-dispute/.venv/lib/python3.9/site-packages/anyio/to_thread.py", line 28, in run_sync

    return await get_asynclib().run_sync_in_worker_thread(func, *args, cancellable=cancellable,

  File "/usr/src/apps/api-dispute/.venv/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 805, in run_sync_in_worker_thread

    return await future

  File "/usr/src/apps/api-dispute/.venv/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 743, in run

    result = func(*args)

  File "/usr/src/apps/api-dispute/src/./app/api/events.py", line 41, in handle_event

    models.Event.create(db, data=event)

  File "/usr/src/apps/api-dispute/src/./app/models/event.py", line 30, in create

    db.commit()

  File "/usr/src/apps/api-dispute/.venv/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 1428, in commit

    self._transaction.commit(_to_root=self.future)

  File "/usr/src/apps/api-dispute/.venv/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 829, in commit

    self._prepare_impl()

  File "/usr/src/apps/api-dispute/.venv/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl

    self.session.flush()

  File "/usr/src/apps/api-dispute/.venv/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 3340, in flush

    self._flush(objects)

  File "/usr/src/apps/api-dispute/.venv/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 3480, in _flush

    transaction.rollback(_capture_exception=True)

  File "/usr/src/apps/api-dispute/.venv/lib/python3.9/site-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__

    compat.raise_(

  File "/usr/src/apps/api-dispute/.venv/lib/python3.9/site-packages/sqlalchemy/util/compat.py", line 207, in raise_

    raise exception

  File "/usr/src/apps/api-dispute/.venv/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 3440, in _flush

    flush_context.execute()

  File "/usr/src/apps/api-dispute/.venv/lib/python3.9/site-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute

    rec.execute(self)

  File "/usr/src/apps/api-dispute/.venv/lib/python3.9/site-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute

    util.preloaded.orm_persistence.save_obj(

  File "/usr/src/apps/api-dispute/.venv/lib/python3.9/site-packages/sqlalchemy/orm/persistence.py", line 244, in save_obj

    _emit_insert_statements(

  File "/usr/src/apps/api-dispute/.venv/lib/python3.9/site-packages/sqlalchemy/orm/persistence.py", line 1221, in _emit_insert_statements

    result = connection._execute_20(

  File "/usr/src/apps/api-dispute/.venv/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1611, in _execute_20

    return meth(self, args_10style, kwargs_10style, execution_options)

  File "/usr/src/apps/api-dispute/.venv/lib/python3.9/site-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection

    return connection._execute_clauseelement(

  File "/usr/src/apps/api-dispute/.venv/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1478, in _execute_clauseelement

    ret = self._execute_context(

  File "/usr/src/apps/api-dispute/.venv/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1842, in _execute_context

    self._handle_dbapi_exception(

  File "/usr/src/apps/api-dispute/.venv/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 2023, in _handle_dbapi_exception

    util.raise_(

  File "/usr/src/apps/api-dispute/.venv/lib/python3.9/site-packages/sqlalchemy/util/compat.py", line 207, in raise_

    raise exception

  File "/usr/src/apps/api-dispute/.venv/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1799, in _execute_context

    self.dialect.do_execute(

  File "/usr/src/apps/api-dispute/.venv/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 717, in do_execute

    cursor.execute(statement, parameters)

sqlalchemy.exc.ProgrammingError: (psycopg2.errors.UndefinedTable) relation "events" does not exist

LINE 1: INSERT INTO events (message_id, type, party1, party2, contra...

[solomondefi-dev]

👀

@apolkingg8 can you provide the commands you're using to run the tests, and information about the environment?

It looks like the database initialization scripts (e.g. migrations) have not been run.

[apolkingg8]

Sure, please try branch 161_deploy-contract and the test case is in apps/app-e2e/src/api-dispute.spec.ts/events. Before the DB migration scripts, I think the POSTGRES_SERVER=localhost should works fine, there is no reason to change to POSTGRES_SERVER=host.docker.internal. May you test this part?

[apolkingg8]

Some note:

1. The connection issue happened on both Windows and Mac
2. If I set POSTGRES_SERVER=host.docker.internal and upgrade nx-python-poetry, got new error:

   [api-dispute] ERROR [alembic.util.messaging] Can't locate revision identified by '3543c7aaeaae'
   [api-dispute] FAILED: Can't locate revision identified by '3543c7aaeaae'
   [api-dispute] /usr/src
   [api-dispute] Poetry command failed:
   [api-dispute] > poetry run alembic upgrade head

[kelvin-wong]

I'm looking into it. For the hostname, as kerbernetes create and manage a separate network for the whole cluster. Some configuration for the applications should be updated accordingly.

[solomondefi-dev]

Let me know if there's something I can take a look at, I have been doing a bit of k8s configuration lately.

[kelvin-wong]

@apolkingg8 The database initialization should work now.

[apolkingg8]

The test passed! Thanks a lot :)