shsmith
commented
6 years ago Same here. Peer discovery seems to stop at some point, leaving many peers in the "unknown" state and "never" for last attempt.
Closed SomberNight closed 6 years ago
shsmith
commented
6 years ago Same here. Peer discovery seems to stop at some point, leaving many peers in the "unknown" state and "never" for last attempt.
kyuupichan
commented
6 years ago It's normal for my BCH node I'm running. It seems it's never even attempted the peers above? I'll take a look for a clue; as you know I rewrote the peer disovery loop to remove the callback tangle but I might have lost something along the way. If there are any clues in the logs about the "never" peers let me know (like failed network connection attempts, etc)
shsmith
commented
6 years ago Is this related?
INFO:ElectrumX:[709] SSL 146.185.34.240:48998, 118 total
INFO:ElectrumX:[710] SSL 37.168.74.225:25788, 117 total
ERROR:Tasks:uncaught task exception: connection not allowed by ruleset
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/electrumX-1.7-py3.6.egg/electrumx/lib/tasks.py", line 60, in _check_task_exception
task.result()
File "/usr/local/lib/python3.6/dist-packages/electrumX-1.7-py3.6.egg/electrumx/server/peers.py", line 229, in _retry_peer
async with PeerSession(peer.host, port, **kwargs) as session:
File "/usr/local/lib/python3.6/dist-packages/aiorpcx/session.py", line 328, in __aenter__
await self.create_connection()
File "/usr/local/lib/python3.6/dist-packages/aiorpcx/session.py", line 322, in create_connection
self_func, self.host, self.port, loop=self.loop, **self.kwargs)
File "/usr/local/lib/python3.6/dist-packages/aiorpcx/socks.py", line 379, in create_connection
sock, address = await self._connect(addresses, loop, timeout)
File "/usr/local/lib/python3.6/dist-packages/aiorpcx/socks.py", line 292, in _connect
OSError(f'multiple exceptions: {", ".join(strings)}'))
File "/usr/local/lib/python3.6/dist-packages/aiorpcx/socks.py", line 267, in _connect_one
self.auth, loop))
File "/usr/local/lib/python3.6/dist-packages/aiorpcx/util.py", line 161, in run
return await self.task
File "/usr/local/lib/python3.6/dist-packages/aiorpcx/socks.py", line 222, in handshake
data[1], f'unknown SOCKS5 error code: {data[1]}'))
aiorpcx.socks.SOCKSFailure: connection not allowed by ruleset
ERROR:Tasks:uncaught task exception: connection not allowed by ruleset
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/electrumX-1.7-py3.6.egg/electrumx/lib/tasks.py", line 60, in _check_task_exception
task.result()
File "/usr/local/lib/python3.6/dist-packages/electrumX-1.7-py3.6.egg/electrumx/server/peers.py", line 175, in _discover_peers
await self._retry_peers()
File "/usr/local/lib/python3.6/dist-packages/electrumX-1.7-py3.6.egg/electrumx/server/peers.py", line 201, in _retry_peers
await asyncio.gather(*tasks)
File "/usr/local/lib/python3.6/dist-packages/electrumX-1.7-py3.6.egg/electrumx/lib/tasks.py", line 60, in _check_task_exception
task.result()
File "/usr/local/lib/python3.6/dist-packages/electrumX-1.7-py3.6.egg/electrumx/server/peers.py", line 229, in _retry_peer
async with PeerSession(peer.host, port, **kwargs) as session:
File "/usr/local/lib/python3.6/dist-packages/aiorpcx/session.py", line 328, in __aenter__
await self.create_connection()
File "/usr/local/lib/python3.6/dist-packages/aiorpcx/session.py", line 322, in create_connection
self_func, self.host, self.port, loop=self.loop, **self.kwargs)
File "/usr/local/lib/python3.6/dist-packages/aiorpcx/socks.py", line 379, in create_connection
sock, address = await self._connect(addresses, loop, timeout)
File "/usr/local/lib/python3.6/dist-packages/aiorpcx/socks.py", line 292, in _connect
OSError(f'multiple exceptions: {", ".join(strings)}'))
File "/usr/local/lib/python3.6/dist-packages/aiorpcx/socks.py", line 267, in _connect_one
self.auth, loop))
File "/usr/local/lib/python3.6/dist-packages/aiorpcx/util.py", line 161, in run
return await self.task
File "/usr/local/lib/python3.6/dist-packages/aiorpcx/socks.py", line 222, in handshake
data[1], f'unknown SOCKS5 error code: {data[1]}'))
aiorpcx.socks.SOCKSFailure: connection not allowed by ruleset
INFO:PeerManager:verified s7clinmo4cazmhul.onion via TCP over Tor in 24.4s
INFO:PeerManager:verified hsmiths5mjk6uijs.onion via SSL over Tor in 24.8s
Perhaps. Any uncaught task exception is a bug. Never seen that socks failure before. @SomberNight do you have something similar?
kyuupichan
commented
6 years ago @shsmith thanks, I was missing an exception type. I think that should fix it.
SomberNight
commented
6 years ago It works now, thanks.
I've just noticed while testing that it takes around 10-11 minutes to find ~all peers; which previously took <1 minute. It needs 2-3 rounds of _retry_peers (which are 5 mins apart).
So actually, maybe it would have worked before, I was just expecting it to be as fast as with the previous version. :)
kyuupichan
commented
6 years ago @SomberNight good. Yes there's a slight bottleneck in that the asyncio.gather waits for all current probes to complete before looping again. I planned to put each peer in its own task but it was lower priority. As well as being the ultimate for concurrency there it's also cleaner. I'm about to push some code to aiorpcX that makes it easier to express too. As you mention it here, I'll try and get those changes done in the next day or so.
kyuupichan
commented
6 years ago @SomberNight can you test 1.7.1 please? It should be fully concurrent now
SomberNight
commented
6 years ago tested with 1.7.1. it's fast again :)
kyuupichan
commented
6 years ago Thanks for the confirmation. Now to fix that one "async" that seems to break Python 3.7...
e-x ~1.7 finds significantly fewer peers than ~1.6
BTC mainnet.
On commit 023e463748eeb0ccde87b0315cea7818a479a5a5
On commit 3cdf4536636f01d9d63a99ff96c1bfdef3d9216a