sqlite3.ProgrammingError

[almico]

I looked at scan.log from scan.py (using the latest available Monocle source code) and found multiple errors like this. This is one of them:

[2017-08-15 20:55:36][   ERROR][notifier] An exception occurred while trying to estimate remaining time.
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 721, in _commit_impl
    self.engine.dialect.do_commit(self.connection)
  File "/usr/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 443, in do_commit
    dbapi_connection.commit()
sqlite3.ProgrammingError: SQLite objects created in a thread can only be used in that same thread.The object was created in thread id 139928004450048 and this is thread id 139931380840256

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/root/pogo/Monocle/monocle/db.py", line 328, in session_scope
    session.commit()
  File "/usr/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 906, in commit
    self.transaction.commit()
  File "/usr/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 465, in commit
    t[1].commit()
  File "/usr/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1632, in commit
    self._do_commit()
  File "/usr/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1663, in _do_commit
    self.connection._commit_impl()
  File "/usr/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 723, in _commit_impl
    self._handle_dbapi_exception(e, None, None, None, None)
  File "/usr/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1402, in _handle_dbapi_exception
    exc_info
  File "/usr/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/usr/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 186, in reraise
    raise value.with_traceback(tb)
  File "/usr/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 721, in _commit_impl
    self.engine.dialect.do_commit(self.connection)
  File "/usr/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 443, in do_commit
    dbapi_connection.commit()
sqlalchemy.exc.ProgrammingError: (sqlite3.ProgrammingError) SQLite objects created in a thread can only be used in that same thread.The object was created in thread id 139928004450

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 701, in _rollback_impl
    self.engine.dialect.do_rollback(self.connection)
  File "/usr/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 440, in do_rollback
    dbapi_connection.rollback()
sqlite3.ProgrammingError: SQLite objects created in a thread can only be used in that same thread.The object was created in thread id 139928004450048 and this is thread id 139931380840256

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 701, in _rollback_impl
    self.engine.dialect.do_rollback(self.connection)
  File "/usr/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 440, in do_rollback
    dbapi_connection.rollback()
sqlite3.ProgrammingError: SQLite objects created in a thread can only be used in that same thread.The object was created in thread id 139928004450048 and this is thread id 139931380840256

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/root/pogo/Monocle/monocle/notification.py", line 724, in notify
    tth = await run_threaded(estimate_remaining_time, session, pokemon['spawn_id'], seen)
  File "/usr/lib64/python3.6/contextlib.py", line 88, in __exit__
    next(self.gen)
  File "/root/pogo/Monocle/monocle/db.py", line 330, in session_scope
    session.rollback()
  File "/usr/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 870, in rollback
    self.transaction.rollback()
  File "/usr/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 518, in rollback
    util.reraise(*rollback_err)
  File "/usr/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 187, in reraise
    raise value
  File "/usr/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 490, in rollback
    transaction._rollback_impl()
  File "/usr/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 527, in _rollback_impl
    t[1].rollback()
  File "/usr/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1621, in rollback
    self._do_rollback()
  File "/usr/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1659, in _do_rollback
    self.connection._rollback_impl()
  File "/usr/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 703, in _rollback_impl
    self._handle_dbapi_exception(e, None, None, None, None)
  File "/usr/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1395, in _handle_dbapi_exception
    self._autorollback()
  File "/usr/lib/python3.6/site-packages/sqlalchemy/util/langhelpers.py", line 76, in __exit__
    compat.reraise(type_, value, traceback)
  File "/usr/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 187, in reraise
    raise value
  File "/usr/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1395, in _handle_dbapi_exception
    self._autorollback()
  File "/usr/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 824, in _autorollback
    self._root._rollback_impl()
  File "/usr/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 703, in _rollback_impl
    self._handle_dbapi_exception(e, None, None, None, None)
  File "/usr/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1323, in _handle_dbapi_exception
    exc_info
  File "/usr/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/usr/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 186, in reraise
    raise value.with_traceback(tb)
  File "/usr/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 701, in _rollback_impl
    self.engine.dialect.do_rollback(self.connection)
  File "/usr/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 440, in do_rollback
    dbapi_connection.rollback()
sqlalchemy.exc.ProgrammingError: (sqlite3.ProgrammingError) SQLite objects created in a thread can only be used in that same thread.The object was created in thread id 139928004450048 and this is thread id 139931380840256

[MiguelGall]

I have the same problem, people told me sqlite problems, use another database

[almico]

Interesting. Thank you. I will look for some information about how to migrate from sqlite3 to mysql.

[ghost]

It is caused by incorrect use of database API objects in notification.py. My notification system rewrite (#214) will fix this problem, but if you're in a hurry for a short-term fix, you can apply this patch to your monocle instance:

diff --git a/monocle/notification.py b/monocle/notification.py
index 754ccf4..3a2ba95 100755
--- a/monocle/notification.py
+++ b/monocle/notification.py
@@ -720,8 +720,10 @@ class Notifier:
             seen = pokemon['seen'] % 3600
             cache_handle = self.cache.store.add(pokemon['encounter_id'])
             try:
-                with session_scope() as session:
-                    tth = await run_threaded(estimate_remaining_time, session, pokemon['spawn_id'], seen)
+                def xxx_estimate_remaining_time_with_session():
+                    with session_scope() as session:
+                        return estimate_remaining_time(session, pokemon['spawn_id'], seen)
+                tth = await run_threaded(xxx_estimate_remaining_time_with_session)
             except Exception:
                 self.log.exception('An exception occurred while trying to estimate remaining time.')
                 now_epoch = time()

[almico]

Thank you @mewio 😃 I took the opportunity to move to MySQL (with some fiddling I managed to port everything), but I think your fix will be much appreciated by a lot of users.

[DavePlater]

Notifications use their own DB that is different than where sightings are kept?