search

ethereum / trinity

The Trinity client for the Ethereum network
https://trinity.ethereum.org
MIT License
474 stars 145 forks source link

Unknown sqlite error #1348

Closed carver closed 3 years ago

carver commented 4 years ago 
aiohttp==3.6.0
alabaster==0.7.12
apipkg==1.5
argcomplete==1.10.0
argh==0.26.2
asn1crypto==1.2.0
async-exit-stack==1.0.1
async-generator==1.10
async-lru==0.1.0
async-timeout==3.0.1
asyncio-cancel-token==0.2.0
atomicwrites==1.3.0
attrdict==2.0.1
attrs==19.3.0
Babel==2.7.0
backcall==0.1.0
base58==1.0.3
blake2b-py==0.1.3
bleach==3.1.0
bloom-filter==1.3
bumpversion==0.5.3
cached-property==1.5.1
cachetools==3.1.1
certifi==2019.9.11
cffi==1.13.0
chardet==3.0.4
Click==7.0
coincurve==10.0.0
colorama==0.4.1
contextvars==2.4
coverage==4.5.4
cryptography==2.6.1
cytoolz==0.10.1
dataclasses==0.6
decorator==4.4.0
docopt==0.6.2
docutils==0.15.2
eth-abi==1.3.0
eth-account==0.2.3
eth-bloom==1.0.3
eth-hash==0.2.0
eth-keyfile==0.5.1
eth-keys==0.2.4
eth-rlp==0.1.2
eth-tester==0.2.0b3
eth-typing==2.2.0
eth-utils==1.8.0
execnet==1.7.1
factory-boy==2.12.0
Faker==2.0.3
fastecdsa==1.7.4
flake8==3.5.0
flake8-bugbear==18.8.0
hexbytes==0.2.0
hypothesis==4.41.2
idna==2.7
idna-ssl==1.1.0
ifaddr==0.1.6
imagesize==1.1.0
immutables==0.11
incremental==17.5.0
ipfshttpclient==0.4.12
ipython==7.8.0
ipython-genutils==0.2.0
jedi==0.15.1
Jinja2==2.10.3
jsonschema==3.0.1
lahja==0.14.5
libp2p==0.1.1
lru-dict==1.1.6
lxml==4.4.1
MarkupSafe==1.1.1
mccabe==0.6.1
milagro-bls-binding==0.1.3
more-itertools==7.2.0
multiaddr==0.0.8
multidict==4.5.2
mypy==0.730
mypy-extensions==0.4.3
netaddr==0.7.19
netdisco==2.6.0
netifaces==0.10.9
outcome==1.0.1
packaging==19.2
parsimonious==0.8.1
parso==0.5.1
pathtools==0.1.2
pexpect==4.7.0
pickleshare==0.7.5
pkg-resources==0.0.0
pkginfo==1.5.0.1
pluggy==0.7.1
plyvel==1.1.0
prompt-toolkit==2.0.10
protobuf==3.9.0
ptyprocess==0.6.0
py==1.8.0
py-ecc==1.7.1
-e git+git@github.com:ethereum/py-evm.git@10df2a6501f93be3f2308c779c76d3095a1da582#egg=py_evm
pycodestyle==2.3.1
pycparser==2.19
pycryptodome==3.9.0
pyethash==0.1.27
pyflakes==1.6.0
Pygments==2.4.2
pymultihash==0.8.2
pyparsing==2.4.2
pyrsistent==0.15.4
pysha3==1.0.2
pytest==3.6.4                                                                                                                                                                                                                                 
pytest-cov==2.5.1
pytest-mock==1.10.4
pytest-randomly==3.0.0
pytest-watch==4.2.0
pytest-xdist==1.18.1
python-dateutil==2.8.0
python-snappy==0.5.4
pytz==2019.3
PyYAML==5.1.2
readme-renderer==24.0
requests==2.22.0
requests-toolbelt==0.9.1
rlp==1.1.0
rpcudp==3.0.0
ruamel.yaml==0.15.98
semantic-version==2.8.2
six==1.12.0
sniffio==1.1.0
snowballstemmer==2.0.0
sortedcontainers==2.1.0
Sphinx==1.7.9
sphinx-rtd-theme==0.4.3
sphinxcontrib-asyncio==0.2.0
sphinxcontrib-websupport==1.1.2
SQLAlchemy==1.3.10
sqlalchemy-stubs==0.1
ssz==0.1.5
termcolor==1.1.0
text-unidecode==1.3
toml==0.10.0
toolz==0.10.0
towncrier==19.2.0
tox==2.7.0
tqdm==4.36.1
traitlets==4.3.3
trie==1.4.0
-e git+git@github.com:carver/trinity.git@85a60a683eb5b02f16f89a7895cfcbad6774eab3#egg=trinity
trio==0.11.0
trio-typing==0.2.0
twine==2.0.0
typed-ast==1.4.0
typing-extensions==3.7.4
u-msgpack-python==2.5.2
uPnPClient==0.0.8
urllib3==1.25.6
uvloop==0.11.2
varint==1.0.2
virtualenv==16.7.6
watchdog==0.9.0
wcwidth==0.1.7
web3==4.4.1
webencodings==0.5.1
websockets==5.0.1
yarl==1.3.0
zeroconf==0.23.0

What is wrong?

Logs:

(venv) jcarver@carver-tower-2019:~/code/trinity$ PYTHONWARNINGS=ignore trinity --max-peers=20 --beam-from-checkpoint="eth://block/byhash/0x01b33227c79cb00b7918067eb10e8c4f502e9063260e806f016d80196d80627e?score=12,435,926,
049,655,753,781,418" --enable-experimental-eth1-peer-tracking
    INFO  2019-12-03 10:39:32,909               trinity
      ______     _       _ __
     /_  __/____(_)___  (_) /___  __
      / / / ___/ / __ \/ / __/ / / /
     / / / /  / / / / / / /_/ /_/ /
    /_/ /_/  /_/_/ /_/_/\__/\__, /
                           /____/
    INFO  2019-12-03 10:39:32,909               trinity  Started main process (pid=3190)
    INFO  2019-12-03 10:39:32,911               trinity  Trinity/v0.1.0a30/linux/cpython3.6.8
    INFO  2019-12-03 10:39:32,911               trinity  Trinity DEBUG log file is created at /home/jcarver/.local/share/trinity/mainnet/logs-eth1/trinity.log
    INFO  2019-12-03 10:39:33,934                  root  Started DB server process (pid=3198)
    INFO  2019-12-03 10:39:35,120  component(#JsonRpcServerComponent)  Component started: JSON-RPC API (pid=3207)
    INFO  2019-12-03 10:39:35,339             IPCServer  IPC started at: /home/jcarver/.local/share/trinity/mainnet/ipcs-eth1/jsonrpc.ipc
    INFO  2019-12-03 10:39:36,169  component(#NetworkDBComponent)  Component started: Network Database (pid=3214)
    INFO  2019-12-03 10:39:37,197  component(#PeerDiscoveryComponent)  Component started: Discovery (pid=3222)
    INFO  2019-12-03 10:39:38,226  component(#UpnpComponent)  Component started: Upnp (pid=3229)
    INFO  2019-12-03 10:39:38,232           UPnPService  Setting up NAT portmap...
    INFO  2019-12-03 10:39:39,259  component(#BeamChainExecutionComponent)  Component started: Beam Sync Chain Execution (pid=3237)
   ERROR  2019-12-03 10:39:39,283              NullPool  Exception during reset or similar
Traceback (most recent call last):
  File "/home/jcarver/code/trinity/venv/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 680, in _finalize_fairy
    fairy._reset(pool)
  File "/home/jcarver/code/trinity/venv/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 867, in _reset
    pool._dialect.do_rollback(self)
  File "/home/jcarver/code/trinity/venv/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 530, in do_rollback
    dbapi_connection.rollback()
sqlite3.ProgrammingError: SQLite objects created in a thread can only be used in that same thread. The object was created in thread id 139629487785792 and this is thread id 139629191157504.
--- Logging error ---
Traceback (most recent call last):
  File "/home/jcarver/code/trinity/venv/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 680, in _finalize_fairy
    fairy._reset(pool)
  File "/home/jcarver/code/trinity/venv/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 867, in _reset
    pool._dialect.do_rollback(self)
  File "/home/jcarver/code/trinity/venv/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 530, in do_rollback
    dbapi_connection.rollback()
sqlite3.ProgrammingError: SQLite objects created in a thread can only be used in that same thread. The object was created in thread id 139629487785792 and this is thread id 139629191157504.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.6/logging/handlers.py", line 71, in emit
    if self.shouldRollover(record):
  File "/usr/lib/python3.6/logging/handlers.py", line 188, in shouldRollover
    self.stream.seek(0, 2)  #due to non-posix-compliant Windows feature
RuntimeError: reentrant call inside <_io.BufferedWriter name='/home/jcarver/.local/share/trinity/mainnet/logs-eth1/trinity.log'>
Call stack:
  File "/usr/lib/python3.6/threading.py", line 884, in _bootstrap
    self._bootstrap_inner()
  File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.6/threading.py", line 864, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/lib/python3.6/logging/handlers.py", line 1479, in _monitor
    self.handle(record)
  File "/usr/lib/python3.6/logging/handlers.py", line 1462, in handle
    handler.handle(record)
  File "/usr/lib/python3.6/logging/__init__.py", line 865, in handle
    self.emit(record)
  File "/usr/lib/python3.6/logging/handlers.py", line 73, in emit
    logging.FileHandler.emit(self, record)
  File "/usr/lib/python3.6/logging/__init__.py", line 1072, in emit
    StreamHandler.emit(self, record)
  File "/usr/lib/python3.6/logging/__init__.py", line 998, in emit
    self.flush()
  File "/usr/lib/python3.6/logging/__init__.py", line 978, in flush
    self.stream.flush()
  File "/home/jcarver/code/trinity/venv/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 503, in <lambda>
    and _finalize_fairy(None, rec, pool, ref, echo),
  File "/home/jcarver/code/trinity/venv/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 689, in _finalize_fairy
    "Exception during reset or similar", exc_info=True
Message: 'Exception during reset or similar'
Arguments: ()
   ERROR  2019-12-03 10:39:39,290              NullPool  Exception closing connection <sqlite3.Connection object at 0x7efdf3790490>
Traceback (most recent call last):
  File "/home/jcarver/code/trinity/venv/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 680, in _finalize_fairy
    fairy._reset(pool)
  File "/home/jcarver/code/trinity/venv/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 867, in _reset
    pool._dialect.do_rollback(self)
  File "/home/jcarver/code/trinity/venv/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 530, in do_rollback
    dbapi_connection.rollback()
sqlite3.ProgrammingError: SQLite objects created in a thread can only be used in that same thread. The object was created in thread id 139629487785792 and this is thread id 139629191157504.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/jcarver/code/trinity/venv/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 270, in _close_connection
    self._dialect.do_close(connection)
  File "/home/jcarver/code/trinity/venv/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 536, in do_close
    dbapi_connection.close()
sqlite3.ProgrammingError: SQLite objects created in a thread can only be used in that same thread. The object was created in thread id 139629487785792 and this is thread id 139629191157504.
--- Logging error ---
Traceback (most recent call last):
  File "/home/jcarver/code/trinity/venv/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 680, in _finalize_fairy
    fairy._reset(pool)
  File "/home/jcarver/code/trinity/venv/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 867, in _reset
    pool._dialect.do_rollback(self)
  File "/home/jcarver/code/trinity/venv/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 530, in do_rollback
    dbapi_connection.rollback()
sqlite3.ProgrammingError: SQLite objects created in a thread can only be used in that same thread. The object was created in thread id 139629487785792 and this is thread id 139629191157504.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/jcarver/code/trinity/venv/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 270, in _close_connection
    self._dialect.do_close(connection)
  File "/home/jcarver/code/trinity/venv/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 536, in do_close
    dbapi_connection.close()
sqlite3.ProgrammingError: SQLite objects created in a thread can only be used in that same thread. The object was created in thread id 139629487785792 and this is thread id 139629191157504.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.6/logging/handlers.py", line 71, in emit
    if self.shouldRollover(record):
  File "/usr/lib/python3.6/logging/handlers.py", line 188, in shouldRollover
    self.stream.seek(0, 2)  #due to non-posix-compliant Windows feature
RuntimeError: reentrant call inside <_io.BufferedWriter name='/home/jcarver/.local/share/trinity/mainnet/logs-eth1/trinity.log'>
Call stack:
  File "/usr/lib/python3.6/threading.py", line 884, in _bootstrap
    self._bootstrap_inner()
  File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.6/threading.py", line 864, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/lib/python3.6/logging/handlers.py", line 1479, in _monitor
    self.handle(record)
  File "/usr/lib/python3.6/logging/handlers.py", line 1462, in handle
    handler.handle(record)
  File "/usr/lib/python3.6/logging/__init__.py", line 865, in handle
    self.emit(record)
  File "/usr/lib/python3.6/logging/handlers.py", line 73, in emit
    logging.FileHandler.emit(self, record)
  File "/usr/lib/python3.6/logging/__init__.py", line 1072, in emit
    StreamHandler.emit(self, record)
  File "/usr/lib/python3.6/logging/__init__.py", line 998, in emit
    self.flush()
  File "/usr/lib/python3.6/logging/__init__.py", line 978, in flush
    self.stream.flush()
  File "/home/jcarver/code/trinity/venv/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 503, in <lambda>
    and _finalize_fairy(None, rec, pool, ref, echo),
  File "/home/jcarver/code/trinity/venv/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 692, in _finalize_fairy
    connection_record.invalidate(e=e)
  File "/home/jcarver/code/trinity/venv/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 586, in invalidate
    self.__close()
  File "/home/jcarver/code/trinity/venv/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 628, in __close
    self.__pool._close_connection(self.connection)
  File "/home/jcarver/code/trinity/venv/lib/python3.6/site-packages/sqlalchemy/pool/base.py", line 273, in _close_connection
    "Exception closing connection %r", connection, exc_info=True
Message: 'Exception closing connection %r'
Arguments: (<sqlite3.Connection object at 0x7efdf3790490>,)

How can it be fixed

Fill this in if you know how to fix it.

carver commented 4 years ago

The issue is not reliably reproducible yet.

pipermerriam commented 4 years ago

Guessing this has something to do with how the network-db component does validation. It tries to acquire a session with the database. That's where I would start looking.