Improve add_peer RPC behaviour (was: Server not getting any sessions)

[mmouse-]

I'm sorry, but this is more a question than a bugreport. Or you could say: the bug is ElectrumX not generating a proper error message despite something being wrong apparently.

I'm running an ElectrumX server 1.0.15 for Bitcoin Cash.

My server is up for at least a week now, reachable at elecash.bcc.nummi.it, Port 50002 (SSL). It has two connections to BCC fullnodes. And some peers from the BCC chain shown as "good". I can connect my Electron Cash client to it, and everything works fine.

But it has exactly one client session all the time, which is my own client that I did manually connect to it. Why are there no other client connections showing up?

I'm also running an ElectrumX for the legacy Bitcoin chain, which normally has 600-700 connections, so my general setup should be ok.

Any help appreciated...

[shsmith]

I do not see your server in the peers list on my abc node.
I was able to manually connect to your host, so your port is open, dns works and ssl cert works. Do you have peer discovery disabled?

PEER_DISCOVERY=yes PEER_ANNOUNCE=yes

[mmouse-]

Do you have peer discovery disabled?

I checked. Both variables were not set at all. And default is to announce it.
Nevertheless I set both to yes and restarted. No change...

[shsmith]

What PeerManager messages were output after the restart? If self verification fails I think it prevents announcement to peers.

[mmouse-]

Verification may be a point, as I didn't find anything in the logs at first.
So I explicitely set REPORT_HOST and REPORT_SSL_PORT and restarted again.
Log shows now about PeerManager:

...
INFO:BlockProcessor:reorg limit is 200 blocks
INFO:PeerManager:accepted new peer 1/1 elecash.bcc.nummi.it from env 
INFO:PeerManager:accepted new peer 1/26 electron.ueo.ch from peers file 
INFO:PeerManager:accepted new peer 2/26 electrum.rofl.cat from peers file 
INFO:PeerManager:accepted new peer 3/26 bcc.arihanc.com from peers file 
...
INFO:PeerManager:accepted new peer 26/26 bccarihace4jdcnt.onion from peers file 
INFO:Controller:RPC server listening on None:8004
INFO:Prefetcher:caught up to daemon height 479,723
INFO:Controller:block processor has caught up
INFO:PeerManager:IRC is disabled
INFO:Controller:max session count: 1,000
INFO:Controller:session timeout: 200 seconds
INFO:Controller:session bandwidth limit 2,000,000 bytes
INFO:Controller:max response size 1,000,000 bytes
INFO:Controller:max subscriptions across all sessions: 250,000
INFO:Controller:max subscriptions per session: 50,000
INFO:Controller:bands: [7812, 31250, 125000, 500000, 2000000, 3000000, 4500000, 6750000, 10125000, 15187500]
INFO:Controller:SSL server listening on elecash.bcc.nummi.it:50002
...
INFO:PeerManager:beginning peer discovery; force use of proxy: False
INFO:ElectrumX:[0] SSL 80.241.216.62:47648, 1 total
INFO:PeerManager:verified elecash.bcc.nummi.it via SSL at 80.241.216.62 in 0.1s
...

I let it run then for around an hour, but still no client sessions showing up...

[shsmith]

INFO:Controller:SSL server listening on elecash.bcc.nummi.it:50002

Try just

HOST=

Don't put your FQDN in the HOST setting.

[mmouse-]

Ok, did that. Had to change the port to 50012 for that to work (BTC electrumx on same machine, same port, other IP). It now says

INFO:Controller:SSL server listening on :50012
INFO:PeerManager:verified elecash.bcc.nummi.it via SSL at 80.241.216.62 in 0.1s

But still no client sessions after more than 20 minutes uptime.
This is a really strange issue. Is it possible that the subdomain hostname containing three dots (instead of the usual two) is a problem somehow?

Edit: Changed the REPORT_HOST from hostname to IP address. Same result, so the particular hostname isn't the problem.

[kyuupichan]

It sounds like your port settings and REPORT_HOST were initially wrong. If now correct you will gradually become known over 24 hours. To speed it up, shut down the server, remove the peers file from the DB directory, and restart.

[kyuupichan]

Your setup is still bad. This is from my logs:

rejected add_peer request from 80.241.211.40 for elecash.bcc.nummi.it (source-destination mismatch)

Your DNS looks badly set up.

[mmouse-]

Ok, thanks for the hint. I finally got to a proper bug report, and it's not my DNS :)

If a machine has more than one IP, ElectrumX binds its listening port correctly to the IP/Hostname given in the HOST= variable.

But for outgoing connections it always uses the first available system IP, which is not necessarily the one it was bound to. This leads to "source-destination mismatch" errors, because we're advertizing a different IP than we're connecting from. Additionally these errors get logged on the remote host, but not on the local machine.

For me I got this fixed by a small hack of peers.py. See #226

Having 40+ clients after 10 minutes with the fix.

[kyuupichan]

This is great. I requested a couple of cosmetic changes. Thanks for looking into this and submitting a PR.

[aseedb]

Hi @kyuupichan, this thread helped me get my electrumx server to run properly. The trick was not to set HOST to my FQDN. On another electrumx server that I set up earlier, I had HOST set to that other server FQDN and it was working fine. Why is it sometimes a problem to set HOST? Note that the server where I had the problem has no fix IP but uses noip.com to get an FQDN.

[kyuupichan]

I assume you were not getting any sessions because you didn't get on a peer list. Best to ask someone for their logs which would say why you were rejected

[kyuupichan]

Perhaps I should have the remote side not respond until deciding whether to accept the peer request, and return the reason for rejection

[kyuupichan]

Let's get a server to respond to an add_peer request with it's actual success or failure, so both ends can log it. Much easier to debug on the side of the one making the request.

This would mean add_peer could take several seconds to reply but that's not a big issue