Describe the bug
When running code to follow the head of ethereum chain we encounter a timeout error.
This is mostly because I am not closing sqlalchemy connections properly. WIll need to investigate further.
After this bug appears we see another bug i.e. ValueError: {'code': -32000, 'message': 'Filter not found'}, the new block filter somehow gets vanished and the process completely breaks down.
Expected behavior
A clear and concise description of what you expected to happen.
Everything should run smoothly.
Logs
Timeout error log
[2018-08-30 09:08:16,594][PID:25589][ERROR][ForkPoolWorker-2] Task ether_sql.tasks.filters.push_blocks_in_queue[d50b141d-4fe6-4ce9-a474-9971da4d5a24] raised unexpected: TimeoutError('QueuePool limit of size 5 overflow 10 reached, connection timed out, timeout 30',)
Traceback (most recent call last):
File "/home/ankit/ether_sql/venv/lib/python3.6/site-packages/celery/app/trace.py", line 375, in trace_task
R = retval = fun(*args, **kwargs)
File "/home/ankit/ether_sql/venv/lib/python3.6/site-packages/celery/app/trace.py", line 632, in __protected_call__
return self.run(*args, **kwargs)
File "/home/ankit/ether_sql/ether_sql/tasks/filters.py", line 42, in push_blocks_in_queue
for blocks in blocks_in_waiting:
File "/home/ankit/ether_sql/venv/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 2878, in __iter__
return self._execute_and_instances(context)
File "/home/ankit/ether_sql/venv/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 2899, in _execute_and_instances
close_with_result=True)
File "/home/ankit/ether_sql/venv/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 2908, in _get_bind_args
**kw
File "/home/ankit/ether_sql/venv/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 2890, in _connection_from_session
conn = self.session.connection(**kw)
File "/home/ankit/ether_sql/venv/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 1035, in connection
execution_options=execution_options)
File "/home/ankit/ether_sql/venv/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 1040, in _connection_for_bind
engine, execution_options)
File "/home/ankit/ether_sql/venv/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 409, in _connection_for_bind
conn = bind.contextual_connect()
File "/home/ankit/ether_sql/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2123, in contextual_connect
self._wrap_pool_connect(self.pool.connect, None),
File "/home/ankit/ether_sql/venv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 2158, in _wrap_pool_connect
return fn()
File "/home/ankit/ether_sql/venv/lib/python3.6/site-packages/sqlalchemy/pool.py", line 403, in connect
return _ConnectionFairy._checkout(self)
File "/home/ankit/ether_sql/venv/lib/python3.6/site-packages/sqlalchemy/pool.py", line 784, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "/home/ankit/ether_sql/venv/lib/python3.6/site-packages/sqlalchemy/pool.py", line 532, in checkout
rec = pool._do_get()
File "/home/ankit/ether_sql/venv/lib/python3.6/site-packages/sqlalchemy/pool.py", line 1182, in _do_get
(self.size(), self.overflow(), self._timeout), code="3o7r")
sqlalchemy.exc.TimeoutError: QueuePool limit of size 5 overflow 10 reached, connection timed out, timeout 30 (Background on this error at: http://sqlalche.me/e/3o7r)
No filter log
[2018-08-30 09:09:46,634][PID:25589][ERROR][ForkPoolWorker-2] Task ether_sql.tasks.filters.new_blocks[4abe0390-b89d-4f44-9fb8-879089941119] raised unexpected: ValueError({'code': -32000, 'message': 'Filter not found'},)
Traceback (most recent call last):
File "/home/ankit/ether_sql/venv/lib/python3.6/site-packages/celery/app/trace.py", line 375, in trace_task
R = retval = fun(*args, **kwargs)
File "/home/ankit/ether_sql/venv/lib/python3.6/site-packages/celery/app/trace.py", line 632, in __protected_call__
return self.run(*args, **kwargs)
File "/home/ankit/ether_sql/ether_sql/tasks/filters.py", line 22, in new_blocks
block_hashes = current_session.block_filter.get_new_entries()
File "/home/ankit/ether_sql/venv/lib/python3.6/site-packages/web3/utils/filters.py", line 105, in get_new_entries
log_entries = self._filter_valid_entries(self.web3.eth.getFilterChanges(self.filter_id))
File "/home/ankit/ether_sql/venv/lib/python3.6/site-packages/web3/eth.py", line 344, in getFilterChanges
"eth_getFilterChanges", [filter_id],
File "/home/ankit/ether_sql/venv/lib/python3.6/site-packages/web3/manager.py", line 110, in request_blocking
raise ValueError(response["error"])
ValueError: {'code': -32000, 'message': 'Filter not found'}
Additional context
Possible debugging solutions include:
Describe the bug When running code to follow the head of ethereum chain we encounter a timeout error. This is mostly because I am not closing sqlalchemy connections properly. WIll need to investigate further. After this bug appears we see another bug i.e.
ValueError: {'code': -32000, 'message': 'Filter not found'}
, the new block filter somehow gets vanished and the process completely breaks down.To Reproduce Steps to reproduce the behavior: Do not know a reproducible method to recreate this error but if we run https://ether-sql.readthedocs.io/en/latest/guides/syncing-blockchain.html#following-the-block-chain-head after ~150 blocks we start seeing this error.
Expected behavior A clear and concise description of what you expected to happen. Everything should run smoothly.
Logs
Timeout error log
No filter log
Additional context Possible debugging solutions include: