MagicStack / asyncpg

A fast PostgreSQL Database Client Library for Python/asyncio.
Apache License 2.0
6.99k stars 404 forks source link

concurrent.futures._base.CancelledError in prepared_stmt.fetch from random queries in sqlalchemy #992

Closed pankajkumar229 closed 1 year ago

pankajkumar229 commented 1 year ago

I am getting a cancellederror and not sure how to debug it. I tried a number of suggestions outlined in https://github.com/sqlalchemy/sqlalchemy/issues/6467 but cannot overcome the issue.

The stack for the error looks like this: File "/product/labgrader/nbgrader/api.py", line 2093, in upsert_student_to_course course_student_association = await self.find_association_by_ids(course_id, student_id) File "/product/labgrader/nbgrader/api.py", line 1819, in find_association_by_ids .where(Course_user_association_table.course_id == course_id)) File "/usr/local/lib/python3.7/site-packages/sqlalchemy/ext/asyncio/session.py", line 217, in execute *kw File "/usr/local/lib/python3.7/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 136, in greenlet_spawn result = context.switch(value) File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/session.py", line 1692, in execute result = conn._execute_20(statement, params or {}, execution_options) File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1614, in _execute_20 return meth(self, args_10style, kwargs_10style, execution_options) File "/usr/local/lib/python3.7/site-packages/sqlalchemy/sql/elements.py", line 326, in _execute_on_connection self, multiparams, params, execution_options File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1491, in _execute_clauseelement cache_hit=cache_hit, File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1846, in _execute_context e, statement, parameters, cursor, context File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 2030, in _handle_dbapiexception util.raise(exc_info[1], with_traceback=excinfo[2]) File "/usr/local/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 207, in raise raise exception File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1803, in _execute_context cursor, statement, parameters, context File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 732, in do_execute cursor.execute(statement, parameters) File "/usr/local/lib/python3.7/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 461, in execute self._prepare_and_execute(operation, parameters) File "/usr/local/lib/python3.7/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 76, in await_only return current.driver.switch(awaitable) File "/usr/local/lib/python3.7/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 129, in greenlet_spawn value = await result File "/usr/local/lib/python3.7/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 435, in _prepare_and_execute self._handle_exception(error) File "/usr/local/lib/python3.7/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 370, in _handle_exception self._adapt_connection._handle_exception(error) File "/usr/local/lib/python3.7/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 665, in _handle_exception raise error File "/usr/local/lib/python3.7/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 423, in _prepare_and_execute self._rows = await prepared_stmt.fetch(parameters) File "/usr/local/lib/python3.7/site-packages/asyncpg/prepared_stmt.py", line 176, in fetch data = await self.bind_execute(args, 0, timeout) File "/usr/local/lib/python3.7/site-packages/asyncpg/prepared_stmt.py", line 242, in bind_execute lambda protocol: protocol.bind_execute( File "/usr/local/lib/python3.7/site-packages/asyncpg/prepared_stmt.py", line 230, in __do_execute return await executor(protocol) File "asyncpg/protocol/protocol.pyx", line 201, in bind_execute concurrent.futures._base.CancelledError

I have tried setting prepared_cache_size , prepared_statement_cache_size to 0 or using a different pool type but nothing seems to solve the error.

pankajkumar229 commented 1 year ago

With some debugging, I could overcome this bug. The main problem was that I was running from within the tornado test framework which was calling timeout before waiting for the test method to complete. I moved all logic to the setup phase of the test and things started to work