search

g2p / bedup

Btrfs deduplication
http://pypi.python.org/pypi/bedup
GNU General Public License v2.0
322 stars 50 forks source link

sqlite3.ProgrammingError: SQLite objects created in a thread can only be used in that same thread. #38

Closed nefelim4ag closed 9 years ago

nefelim4ag commented 10 years ago

uname -rv: 3.13.6-1-ARCH #1 SMP PREEMPT Fri Mar 7 22:47:48 CET 2014 python --version: Python 3.3.5

Distribution: Arch Linux

00.00 Committing tracking stateException closing connection <sqlite3.Connection object at 0x7ff5bef498f0> Traceback (most recent call last): File "/usr/lib/python3.3/site-packages/sqlalchemy/pool.py", line 536, in _finalize_fairy fairy._reset(pool, echo) File "/usr/lib/python3.3/site-packages/sqlalchemy/pool.py", line 666, in _reset pool._dialect.do_rollback(self) File "/usr/lib/python3.3/site-packages/sqlalchemy/engine/default.py", line 395, in do_rollback dbapi_connection.rollback() sqlite3.ProgrammingError: SQLite objects created in a thread can only be used in that same thread.The object was created in thread id 140693399103232 and this is thread id 140693533701888

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/lib/python3.3/site-packages/sqlalchemy/pool.py", line 243, in _close_connection self._dialect.do_close(connection) File "/usr/lib/python3.3/site-packages/sqlalchemy/engine/default.py", line 401, in do_close dbapi_connection.close() sqlite3.ProgrammingError: SQLite objects created in a thread can only be used in that same thread.The object was created in thread id 140693399103232 and this is thread id 140693533701888 00.03 Committing tracking state

nefelim4ag commented 10 years ago

Maybe, i can provide more information?

andrwp commented 10 years ago

I'm getting the same error. Distribution: Arch Linux. uname -rv: 3.14.1-1-ARCH #1 SMP PREEMPT Mon Apr 14 20:40:47 CEST 2014 python --version: 3.4.0 SQLAlchemy: 0.9.4 sqlite3 --version: 3.8.4.3 2014-04-03 16:53:12 a611fa96c4a848614efe899130359c9f6fb889c3

marcin-github commented 10 years ago

I'm, adding famous "me too". I'm using Gentoo, kernel: 3.14.2-hardened-r1, python --version Python 3.3.3 dev-python/sqlalchemy-0.9.4 sqlite3 --version 3.8.2

bedup dedup

Skipping filesystem {893334c4-57fc-4df7-84b9-40af3616c977}, not mounted Not scanning /tmp/src, generation is still 74 Not scanning /usr/src, generation is still 111 Scanning volume /var/tmp generations from 583 to 595, with size cutoff 8388608 00.00 Scanned retained 0/usr/lib64/python3.3/site-packages/cffi/vengine_cpy.py:166: UserWarning: reimporting '_cffi__x8e7359e8x51fe3b45' might overwrite older defini tions % (self.verifier.get_module_name())) 00.02 Scanned 316 retained 0 Not scanning /usr/portage, generation is still 557 Scanning volume /home/farmbuild generations from 595 to 602, with size cutoff 8388608 00.64 Scanned 14301 retained 0 Scanning volume /dane/coredumps generations from 562 to 601, with size cutoff 8388608 00.06 Scanned 333 retained 1 Deduplicating filesystem {7dfcf227-2c63-466d-8e4f-f8ccb4e8bb26} Deduplicated:

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/lib64/python3.3/site-packages/sqlalchemy/pool.py", line 244, in _close_connection self._dialect.do_close(connection) File "/usr/lib64/python3.3/site-packages/sqlalchemy/engine/default.py", line 411, in do_close dbapi_connection.close() sqlite3.ProgrammingError: SQLite objects created in a thread can only be used in that same thread.The object was created in thread id 3287727789824 and this is thread id 3287778469632 00.08 Committing tracking state

voidzero commented 10 years ago

Add me to the list. Perhaps the following link is useful for someone with actual python-fu: https://stackoverflow.com/questions/393554/python-sqlite3-and-concurrency

The following suggestion seems to be useful: sqlite.connect(":memory:", check_same_thread=False)

voidzero commented 10 years ago

For sqlalchemy, it seems that one solution is to add connect_args={'check_same_thread':False}, to line 100 of __main__.py.

I'm not sure if it is without risk though.

The documentation at http://docs.sqlalchemy.org/en/rel_0_7/dialects/sqlite.html will provide more info.

voidzero commented 10 years ago

@g2p Any suggestions, please?

SjonHortensius commented 10 years ago

I had the same issue; right after the dedup seemed to be completed. Reran with the above workaround added and it completed fine. Not sure about the reproducability of this issue, but the workaround seems to work for me.

alphaonex86 commented 10 years ago

Same here: odroid-u3-1 ~ # bedup dedup / Scanning volume / generations from 0 to 28, with size cutoff 8388608 00.00 Scanned retained 0/usr/lib/python3.3/site-packages/cffi/vengine_cpy.py:177: UserWarning: reimporting '_cffi__x7b97d593x3f1476d8' might overwrite older definitions % (self.verifier.get_module_name())) 04:06.0 Scanned 1464740 retained 38 Deduplicating filesystem 14.14 Size group 1/1 (75537008) sampled 2 hashed 2 freed 0 00.00 Committing tracking stateException closing connection <sqlite3.Connection object at 0xb5dbfb48> Traceback (most recent call last): File "/usr/lib/python3.3/site-packages/sqlalchemy/pool.py", line 536, in _finalize_fairy fairy._reset(pool, echo) File "/usr/lib/python3.3/site-packages/sqlalchemy/pool.py", line 666, in _reset pool._dialect.do_rollback(self) File "/usr/lib/python3.3/site-packages/sqlalchemy/engine/default.py", line 395, in do_rollback dbapi_connection.rollback() sqlite3.ProgrammingError: SQLite objects created in a thread can only be used in that same thread.The object was created in thread id -1246186400 and this is thread id -1229721600

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/lib/python3.3/site-packages/sqlalchemy/pool.py", line 243, in _close_connection
self._dialect.do_close(connection)
File "/usr/lib/python3.3/site-packages/sqlalchemy/engine/default.py", line 401, in do_close
dbapi_connection.close()
sqlite3.ProgrammingError: SQLite objects created in a thread can only be used in that same thread.The object was created in thread id -1246186400 and this is thread id -1229721600

voidzero commented 9 years ago

Nice. Well done.

mahdidavoodi7 commented 7 years ago

me tooooo

tarek421995 commented 5 years ago

thank you