akrentsel
commented
4 years ago Have the same issue.
Open nmweizi opened 6 years ago
akrentsel
commented
4 years ago Have the same issue.
Prometheus3375
commented
4 years ago It seems this issue is occurring when db worker is launched too early. Wait till spider will print in logs INFO: Starting frontier and then start db worker.
sibiryakov
commented
4 years ago Is there any other process using the same sqlite file? strategy worker may be?
psdon
commented
3 years ago I'm also having this issue
[db-worker] batches_after_start=32 INFO:db-worker:batches_after_start=32 [db-worker.batchgen] Getting new batches for partition 0 INFO:db-worker.batchgen:Getting new batches for partition 0 ERROR:sqlalchemy.queue:(sqlite3.ProgrammingError) SQLite objects created in a thread can only be used in that same thread.The object was created in thread id 140553838536448 and this is thread id 140554174347072 (Background on this error at: http://sqlalche.me/e/f405) Traceback (most recent call last): File "/usr/local/lib64/python3.6/site-packages/sqlalchemy/engine/base.py", line 724, in _commit_impl self.engine.dialect.do_commit(self.connection) File "/usr/local/lib64/python3.6/site-packages/sqlalchemy/engine/default.py", line 462, in do_commit dbapi_connection.commit() sqlite3.ProgrammingError: SQLite objects created in a thread can only be used in that same thread.The object was created in thread id 140553838536448 and this is thread id 140554174347072
The above exception was the direct cause of the following exception:
Traceback (most recent call last): File "/usr/local/lib/python3.6/site-packages/frontera/contrib/backends/sqlalchemy/components.py", line 26, in func_wrapper return func(self, *args, **kwargs) File "/usr/local/lib/python3.6/site-packages/frontera/contrib/backends/sqlalchemy/components.py", line 213, in schedule self.session.commit() File "/usr/local/lib64/python3.6/site-packages/sqlalchemy/orm/session.py", line 943, in commit self.transaction.commit() File "/usr/local/lib64/python3.6/site-packages/sqlalchemy/orm/session.py", line 471, in commit t[1].commit() File "/usr/local/lib64/python3.6/site-packages/sqlalchemy/engine/base.py", line 1643, in commit self._do_commit() File "/usr/local/lib64/python3.6/site-packages/sqlalchemy/engine/base.py", line 1674, in _do_commit self.connection._commit_impl() File "/usr/local/lib64/python3.6/site-packages/sqlalchemy/engine/base.py", line 726, in _commit_impl self._handle_dbapi_exception(e, None, None, None, None) File "/usr/local/lib64/python3.6/site-packages/sqlalchemy/engine/base.py", line 1413, in _handle_dbapi_exception exc_info File "/usr/local/lib64/python3.6/site-packages/sqlalchemy/util/compat.py", line 265, in raise_from_cause reraise(type(exception), exception, tb=exc_tb, cause=cause) File "/usr/local/lib64/python3.6/site-packages/sqlalchemy/util/compat.py", line 248, in reraise raise value.with_traceback(tb) File "/usr/local/lib64/python3.6/site-packages/sqlalchemy/engine/base.py", line 724, in _commit_impl self.engine.dialect.do_commit(self.connection) File "/usr/local/lib64/python3.6/site-packages/sqlalchemy/engine/default.py", line 462, in do_commit dbapi_connection.commit() sqlalchemy.exc.ProgrammingError: (sqlite3.ProgrammingError) SQLite objects created in a thread can only be used in that same thread.The object was created in thread id 140553838536448 and this is thread id 140554174347072 (Background on this error at: http://sqlalche.me/e/f405) ERROR:sqlalchemy.pool.NullPool:Exception during reset or similar Traceback (most recent call last): File "/usr/local/lib64/python3.6/site-packages/sqlalchemy/engine/base.py", line 724, in _commit_impl self.engine.dialect.do_commit(self.connection) File "/usr/local/lib64/python3.6/site-packages/sqlalchemy/engine/default.py", line 462, in do_commit dbapi_connection.commit() sqlite3.ProgrammingError: SQLite objects created in a thread can only be used in that same thread.The object was created in thread id 140553838536448 and this is thread id 140554174347072
The above exception was the direct cause of the following exception:
Traceback (most recent call last): File "/usr/local/lib/python3.6/site-packages/frontera/contrib/backends/sqlalchemy/components.py", line 26, in func_wrapper return func(self, *args, **kwargs) File "/usr/local/lib/python3.6/site-packages/frontera/contrib/backends/sqlalchemy/components.py", line 213, in schedule self.session.commit() File "/usr/local/lib64/python3.6/site-packages/sqlalchemy/orm/session.py", line 943, in commit self.transaction.commit() File "/usr/local/lib64/python3.6/site-packages/sqlalchemy/orm/session.py", line 471, in commit t[1].commit() File "/usr/local/lib64/python3.6/site-packages/sqlalchemy/engine/base.py", line 1643, in commit self._do_commit() File "/usr/local/lib64/python3.6/site-packages/sqlalchemy/engine/base.py", line 1674, in _do_commit self.connection._commit_impl() File "/usr/local/lib64/python3.6/site-packages/sqlalchemy/engine/base.py", line 726, in _commit_impl self._handle_dbapi_exception(e, None, None, None, None) File "/usr/local/lib64/python3.6/site-packages/sqlalchemy/engine/base.py", line 1413, in _handle_dbapi_exception exc_info File "/usr/local/lib64/python3.6/site-packages/sqlalchemy/util/compat.py", line 265, in raise_from_cause reraise(type(exception), exception, tb=exc_tb, cause=cause) File "/usr/local/lib64/python3.6/site-packages/sqlalchemy/util/compat.py", line 248, in reraise raise value.with_traceback(tb) File "/usr/local/lib64/python3.6/site-packages/sqlalchemy/engine/base.py", line 724, in _commit_impl self.engine.dialect.do_commit(self.connection) File "/usr/local/lib64/python3.6/site-packages/sqlalchemy/engine/default.py", line 462, in do_commit dbapi_connection.commit() sqlalchemy.exc.ProgrammingError: (sqlite3.ProgrammingError) SQLite objects created in a thread can only be used in that same thread.The object was created in thread id 140553838536448 and this is thread id 140554174347072 (Background on this error at: http://sqlalche.me/e/f405)
During handling of the above exception, another exception occurred: