ethereum / py-evm

A Python implementation of the Ethereum Virtual Machine
https://py-evm.readthedocs.io/en/latest/
MIT License
2.27k stars 650 forks source link

Unhandled Timeout on startup #874

Closed cburgdorf closed 6 years ago

cburgdorf commented 6 years ago
$ pip freeze
aiohttp==2.3.10
alabaster==0.7.10
apipkg==1.4
argh==0.26.2
asn1crypto==0.24.0
async-lru==0.1.0
async-timeout==3.0.0
atomicwrites==1.1.5
attrdict==2.0.0
attrs==18.1.0
Babel==2.6.0
backcall==0.1.0
bumpversion==0.5.3
certifi==2018.4.16
cffi==1.11.5
chardet==3.0.4
coincurve==7.1.0
colorama==0.3.9
coverage==4.5.1
cryptography==2.2.2
cytoolz==0.9.0.1
decorator==4.3.0
docopt==0.6.2
docutils==0.14
eth-abi==1.1.1
eth-account==0.2.3
eth-bloom==1.0.0
eth-hash==0.1.4
eth-keyfile==0.5.1
eth-keys==0.2.0b3
eth-rlp==0.1.2
eth-utils==1.0.3
execnet==1.5.0
flake8==3.5.0
hexbytes==0.1.0
hypothesis==3.44.26
idna==2.6
idna-ssl==1.0.1
imagesize==1.0.0
ipython==6.4.0
ipython-genutils==0.2.0
jedi==0.12.0
Jinja2==2.10
lru-dict==1.1.6
lxml==4.2.1
MarkupSafe==1.0
mccabe==0.6.1
more-itertools==4.2.0
multidict==4.3.1
mypy==0.590
netdisco==1.4.1
netifaces==0.10.7
packaging==17.1
parsimonious==0.8.0
parso==0.2.1
pathtools==0.1.2
pexpect==4.6.0
pickleshare==0.7.4
pluggy==0.6.0
plyvel==1.0.4
prompt-toolkit==1.0.15
ptyprocess==0.5.2
py==1.5.3
py-ecc==1.4.2
-e git+https://github.com/ethereum/py-evm.git@bd89504c66bc0275c0ed21f54edad59453261585#egg=py_evm
pycodestyle==2.3.1
pycparser==2.18
pycryptodome==3.6.1
pyethash==0.1.27
pyflakes==1.6.0
Pygments==2.2.0
pyparsing==2.2.0
pysha3==1.0.2
pytest==3.6.0
pytest-asyncio==0.8.0
pytest-cov==2.5.1
pytest-watch==4.2.0
pytest-xdist==1.18.1
python-dateutil==2.7.3
pytz==2018.4
PyYAML==3.12
requests==2.18.4
rlp==1.0.1
rope==0.10.7
simplegeneric==0.8.1
six==1.11.0
snowballstemmer==1.2.1
Sphinx==1.7.5
sphinx-rtd-theme==0.3.1
sphinxcontrib-asyncio==0.2.0
sphinxcontrib-websupport==1.0.1
termcolor==1.1.0
toolz==0.9.0
tox==2.7.0
traitlets==4.3.2
trie==1.3.7
typed-ast==1.1.0
uPnPClient==0.0.8
urllib3==1.22
virtualenv==16.0.0
watchdog==0.8.3
wcwidth==0.1.7
web3==4.2.1
websockets==5.0.1
yarl==1.2.4
zeroconf==0.20.0

What is wrong?

I ran into what appears to be an unhandled timeout at startup

ERROR  06-06 11:29:43     service  Unexpected error in <p2p.server.Server object at 0x7fb0a4a818d0>, exiting
Traceback (most recent call last):
  File "/home/cburgdorf/Documents/hacking/ef/py-evm/p2p/service.py", line 68, in run
    await self._run()
  File "/home/cburgdorf/Documents/hacking/ef/py-evm/p2p/server.py", line 245, in _run
    upnp_dev = await self._discover_upnp_device()
  File "/home/cburgdorf/Documents/hacking/ef/py-evm/p2p/server.py", line 184, in _discover_upnp_device
    timeout=discover_timeout)
  File "/home/cburgdorf/Documents/hacking/ef/py-evm/p2p/service.py", line 51, in wait_first
    return await wait_with_token(*awaitables, token=token_chain, timeout=timeout)
  File "/home/cburgdorf/Documents/hacking/ef/py-evm/p2p/cancel_token.py", line 111, in wait_with_token
    raise TimeoutError()
TimeoutError
    INFO  06-06 11:29:43      server  Closing server...
   ERROR  06-06 11:29:43     service  Unexpected error in <trinity.nodes.mainnet.MainnetFullNode object at 0x7fb0a6c82f98>, exiting
Traceback (most recent call last):
  File "/home/cburgdorf/Documents/hacking/ef/py-evm/p2p/service.py", line 68, in run
    await self._run()
  File "/home/cburgdorf/Documents/hacking/ef/py-evm/trinity/nodes/base.py", line 107, in _run
    await self.get_p2p_server().run()
  File "/home/cburgdorf/Documents/hacking/ef/py-evm/p2p/service.py", line 80, in run
    await self.cleanup()
  File "/home/cburgdorf/Documents/hacking/ef/py-evm/p2p/service.py", line 87, in cleanup
    await self._cleanup()
  File "/home/cburgdorf/Documents/hacking/ef/py-evm/p2p/server.py", line 271, in _cleanup
    await asyncio.gather(self.peer_pool.cancel(), self.discovery.stop())
AttributeError: 'NoneType' object has no attribute 'cancel'
   ERROR  06-06 11:29:48        ssdp  Error 'HTTPConnectionPool(host='192.168.2.1', port=5438): Max retries exceeded with url: /tr64_igd_wdcm.xml (Caused by ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x7fb0a415b160>, 'Connection to 192.168.2.1 timed out. (connect timeout=10)'))' for http://192.168.2.1:5438/tr64_igd.xml

How can it be fixed

:hear_no_evil:

pipermerriam commented 6 years ago

@cburgdorf can you look higher up in your stack trace and see if this is actually a secondary error due to the port being in use?

cburgdorf commented 6 years ago

@pipermerriam it wasn't the address being in use and I can not reproduce that right now (except by setting the timeout to something ridiculously low) but I think there were some connection issues around that time.

pipermerriam commented 6 years ago

This strikes me as further evidence that we need to make the peer pool be avialable earlier in the process rather than having it be magically created by the server.

cburgdorf commented 6 years ago

Oh, I just realized that when I commented I thought this was commenting on this ticket https://github.com/ethereum/py-evm/pull/875

But yes, you are right that having the PeerPool available earlier seems related here (#877)