search

vijfhoek / telematrix

Bridge between Telegram and Matrix
MIT License
98 stars 19 forks source link

telematrix hangs when loosing db connection #58

Open mtippmann opened 6 years ago

mtippmann commented 6 years ago

I'm running current master + https://github.com/SijmenSchoon/telematrix/pull/56 - but the issue also seems to appear without the pull request.

the likely cause is an sqlalchemy error - I've used postgresql as database - I've now switched to sqlite3 - I hope this will work.

The issue seems to be common when using sqlalchemy with mysql/postgres... I don't know how to work around the issue.

Restarting seems to work - probably due to the working reconnect? I had no time to debug further: 

Mar 26 17:25:46 matrix python[113]:     conn = self._revalidate_connection()
Mar 26 17:25:46 matrix python[113]:   File "/home/telematrix/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 424, in _revalidate_connection
Mar 26 17:25:46 matrix python[113]:     "Can't reconnect until invalid "
Mar 26 17:25:46 matrix python[113]: sqlalchemy.exc.InvalidRequestError: Can't reconnect until invalid transaction is rolled back
Mar 26 17:25:46 matrix python[113]: The above exception was the direct cause of the following exception:
Mar 26 17:25:46 matrix python[113]: Traceback (most recent call last):
Mar 26 17:25:46 matrix python[113]:   File "/home/telematrix/venv/lib/python3.6/site-packages/aiohttp/server.py", line 261, in start
Mar 26 17:25:46 matrix python[113]:     yield from self.handle_request(message, payload)
Mar 26 17:25:46 matrix python[113]:   File "/home/telematrix/venv/lib/python3.6/site-packages/aiohttp/web.py", line 88, in handle_request
Mar 26 17:25:46 matrix python[113]:     resp = yield from handler(request)
Mar 26 17:25:46 matrix python[113]:   File "/home/telematrix/telematrix/telematrix/__init__.py", line 200, in matrix_transaction
Mar 26 17:25:46 matrix python[113]:     .filter_by(matrix_room=event['room_id']).first()
Mar 26 17:25:46 matrix python[113]:   File "/home/telematrix/venv/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 2695, in first
Mar 26 17:25:46 matrix python[113]:     ret = list(self[0:1])
Mar 26 17:25:46 matrix python[113]:   File "/home/telematrix/venv/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 2487, in __getitem__
Mar 26 17:25:46 matrix python[113]:     return list(res)
Mar 26 17:25:46 matrix python[113]:   File "/home/telematrix/venv/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 2795, in __iter__
Mar 26 17:25:46 matrix python[113]:     return self._execute_and_instances(context)
Mar 26 17:25:46 matrix python[113]:   File "/home/telematrix/venv/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 2818, in _execute_and_instances
Mar 26 17:25:46 matrix python[113]:     result = conn.execute(querycontext.statement, self._params)
Mar 26 17:25:46 matrix python[113]:   File "/home/telematrix/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 945, in execute
Mar 26 17:25:46 matrix python[113]:     return meth(self, multiparams, params)
Mar 26 17:25:46 matrix python[113]:   File "/home/telematrix/venv/lib/python3.6/site-packages/sqlalchemy/sql/elements.py", line 263, in _execute_on_connection
Mar 26 17:25:46 matrix python[113]:     return connection._execute_clauseelement(self, multiparams, params)
Mar 26 17:25:46 matrix python[113]:   File "/home/telematrix/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1053, in _execute_clauseelement
Mar 26 17:25:46 matrix python[113]:     compiled_sql, distilled_params
Mar 26 17:25:46 matrix python[113]:   File "/home/telematrix/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1121, in _execute_context
Mar 26 17:25:46 matrix python[113]:     None, None)
Mar 26 17:25:46 matrix python[113]:   File "/home/telematrix/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1393, in _handle_dbapi_exception
Mar 26 17:25:46 matrix python[113]:     exc_info
Mar 26 17:25:46 matrix python[113]:   File "/home/telematrix/venv/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 202, in raise_from_cause
Mar 26 17:25:46 matrix python[113]:     reraise(type(exception), exception, tb=exc_tb, cause=cause)
Mar 26 17:25:46 matrix python[113]:   File "/home/telematrix/venv/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 185, in reraise
Mar 26 17:25:46 matrix python[113]:     raise value.with_traceback(tb)
Mar 26 17:25:46 matrix python[113]:   File "/home/telematrix/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1114, in _execute_context
Mar 26 17:25:46 matrix python[113]:     conn = self._revalidate_connection()
Mar 26 17:25:46 matrix python[113]:   File "/home/telematrix/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 424, in _revalidate_connection
Mar 26 17:25:46 matrix python[113]:     "Can't reconnect until invalid "
Mar 26 17:25:46 matrix python[113]: sqlalchemy.exc.StatementError: (sqlalchemy.exc.InvalidRequestError) Can't reconnect until invalid transaction is rolled back [SQL: 'SELECT 
Mar 26 17:34:21 matrix python[113]: ERROR:aiohttp.web:Error handling request
Mar 26 17:34:21 matrix python[113]: Traceback (most recent call last):
Mar 26 17:34:21 matrix python[113]:   File "/home/telematrix/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 346, in connection
Mar 26 17:34:21 matrix python[113]:     return self.__connection
Mar 26 17:34:21 matrix python[113]: AttributeError: 'Connection' object has no attribute '_Connection__connection'
Mar 26 17:34:21 matrix python[113]: During handling of the above exception, another exception occurred:
Mar 26 17:34:21 matrix python[113]: Traceback (most recent call last):
Mar 26 17:34:21 matrix python[113]:   File "/home/telematrix/venv/lib/python3.6/site-packages/aiohttp/server.py", line 261, in start
Mar 26 17:34:21 matrix python[113]:     yield from self.handle_request(message, payload)
Mar 26 17:34:21 matrix python[113]:   File "/home/telematrix/venv/lib/python3.6/site-packages/aiohttp/web.py", line 88, in handle_request
Mar 26 17:34:21 matrix python[113]:     resp = yield from handler(request)
Mar 26 17:34:21 matrix python[113]:   File "/home/telematrix/telematrix/telematrix/__init__.py", line 358, in matrix_transaction
Mar 26 17:34:21 matrix python[113]:     db.session.commit()
Mar 26 17:34:21 matrix python[113]:   File "/home/telematrix/venv/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 874, in commit
Mar 26 17:34:21 matrix python[113]:     self.transaction.commit()
Mar 26 17:34:21 matrix python[113]:   File "/home/telematrix/venv/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 465, in commit
Mar 26 17:34:21 matrix python[113]:     t[1].commit()
Mar 26 17:34:21 matrix python[113]:   File "/home/telematrix/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1626, in commit
Mar 26 17:34:21 matrix python[113]:     self._do_commit()
Mar 26 17:34:21 matrix python[113]:   File "/home/telematrix/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1657, in _do_commit
Mar 26 17:34:21 matrix python[113]:     self.connection._commit_impl()
Mar 26 17:34:21 matrix python[113]:   File "/home/telematrix/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 723, in _commit_impl
Mar 26 17:34:21 matrix python[113]:     self._handle_dbapi_exception(e, None, None, None, None)
Mar 26 17:34:21 matrix python[113]:   File "/home/telematrix/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1396, in _handle_dbapi_exception
Mar 26 17:34:21 matrix python[113]:     util.reraise(*exc_info)
Mar 26 17:34:21 matrix python[113]:   File "/home/telematrix/venv/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 186, in reraise
Mar 26 17:34:21 matrix python[113]:     raise value
Mar 26 17:34:21 matrix python[113]:   File "/home/telematrix/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 721, in _commit_impl
Mar 26 17:34:21 matrix python[113]:     self.engine.dialect.do_commit(self.connection)
Mar 26 17:34:21 matrix python[113]:   File "/home/telematrix/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 351, in connection
Mar 26 17:34:21 matrix python[113]:     self._handle_dbapi_exception(e, None, None, None, None)
Mar 26 17:34:21 matrix python[113]:   File "/home/telematrix/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1396, in _handle_dbapi_exception
Mar 26 17:34:21 matrix python[113]:     util.reraise(*exc_info)
Mar 26 17:34:21 matrix python[113]:   File "/home/telematrix/venv/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 186, in reraise
Mar 26 17:34:21 matrix python[113]:     raise value
Mar 26 17:34:21 matrix python[113]:   File "/home/telematrix/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 349, in connection
Mar 26 17:34:21 matrix python[113]:     return self._revalidate_connection()
Mar 26 17:34:21 matrix python[113]:   File "/home/telematrix/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 424, in _revalidate_connection
Mar 26 17:34:21 matrix python[113]:     "Can't reconnect until invalid "
Mar 26 17:34:21 matrix python[113]: sqlalchemy.exc.InvalidRequestError: Can't reconnect until invalid transaction is rolled back