Whitelist for public servers?

[peleion]

No longer effective

My solution below worked until 12 hours ago (2/10/19) but the IP list has exploded and there are just too many varied IPs now. I'm afraid that the only solution will be to refuse to propagate servers on the peer list unless they run a version > 1.8.12 with some way to verify it so the version string can't just be spoofed.

Original idea (before 2/10/19)

Just upgraded my electrumx server to 1.9.5 and noticed it pulls in your https://electrum.org/blacklist.json regularly. I had already started using ipsets to blacklist known malicious servers and noticed many of them come from relatively small IP blocks but the blacklist only lists DNS names/wildcards and individual IPs. I also notice many of the ever-changing malicious domain wildcard servers still show up as GOOD while blacklisting the CIDR ranges at the firewall the server never even sees them.

Is is possible CIDR formated ranges would be more effective? Most are 255 addresses or smaller - the risk of excluding legitimate users would be outweighed by catching multiple, quickly changing IPs for malicious servers and relying on name wildcard matching which doesn't seem to be effective. This might help make maintaining the blacklist easier as well.

Here's my list: 111.90.128.0/19 (Malaysia) 85.206.160.0/20 (Lithuania) 185.8.177.120/29 (Netherlands) 91.211.88.0/24 (Ukraine) 185.25.48.0/22 (Lithuania) 46.166.160.0/24 (Lithuania) 27.102.0.0/16 (South Korea)

Just by blacklisting these CIDR blocks it drops my peer list down to about 100 servers which is about what I would expect.

[shsmith]

Seems like we need a whitelist, not a blacklist. Seed it with the 98 good servers from 90 days ago and allow new servers to be added via github issues or pull requests. That should not be a burden for legit server operators and will give you server op contact info via github. I know it is all just temporary, but we are really building an 'immune system' for peers, which will likely be needed forever.

[peleion]

Yeah, this might be the way to help secure the future of Electrum and BTC by proxy. Too much bad press with over 500 BTC having been stolen so far.

My server running electrumx (electrum.peleion.com) shows 199 bad addresses to 87 good ones this morning - over a 2:1 ratio. I have a script that runs hourly, simply blocks IPs at the firewall for servers running 1.8.12 that don't resolve a DNS name along with the known 4 malicious IPs that have many, many DNS aliases pointing to different ports and the occasional hand edit. Not perfect or complete but at least I know the 350ish users at any given time have a far lower risk of being ripped off. I don't know if my peer list helps decrease the bad peers propagated to other servers - iptables logs show the blacklisted IPs continually try to rejoin the peer list.

Only downside to this, and I can understand the reasons people would object, is that it "centralizes" the decision making for who can run a public server and might be a barrier to new entry like everyone is at some point.

Another possibility I've mulled: have an online forum for public server admins where we can a) share information like this (closed membership) - even the 2 holdouts still running elsrv or any new server software that may be writen and b) an open subforum for new user/server requests like you suggest where they could be vetted before whitelisting. Then only one person with commit access would be needed for a whitelist. I would certainly be willing to host it.

[ecdsa]

It is perfectly normal for your server to show blacklisted servers in its list of peers. Blacklisted peers are allowed to connect to other peers. The blacklist is not used for the peer discovery, but only when sending the peers list to clients.

The blacklist published on our website is the union of two internal lists: a blacklist (for phishing servers) and a graylist (servers who report peers in the blacklist). This is similar to your suggestion of refusing to propagate servers running old software, but much more secure, because we do not trust servers with the version number they publish, we look at how they behave.

[peleion]

Thanks for the information on the internals of the black/greylist. 2 issues I am unclear about:

a graylist (servers who report peers in the blacklist)

Does the second statement mean "servers who report peers in the blacklist to clients? I must not be propagating them (i.e. your list is working in electrumx) since I'm not on the graylist and there are some on your union list that aren't on mine. Sounds like my script/firewall block is just belt and suspenders then - I should just parse your blacklist.json hourly and block those instead.

Blacklisted peers are allowed to connect to other peers.

My iptables logs show IPs on my blacklist constantly trying to reconnect to me - approximately 20K packets blocked daily. What is the purpose of letting servers on your black/greylist servers connect to any other peers (me) and list them as "good" (or is that properly an electrumx issue)?

Thanks for the great software.

[shsmith]

What is the purpose of letting servers on your black/greylist servers connect to any peers (me) and list them as "good" (or is that properly an electrumx issue)?

This is part of the Sybil detection. If you block the bad peers you won't report them to other servers. Those other servers will flag you as a possible Sybil because you are returning less than half the number of known peers (blacklisted or not).

Here is a tiny shell script that takes the output of 'electrumx_rpc peers' and tells you which are blacklisted.

#!/usr/bin/python3.7
# pipe output of 'electrumx_rpc peers' to classify as ok or blacklisted

import sys, json, requests

blacklist = json.loads(requests.get("https://electrum.org/blacklist.json").text)

for peer in sys.stdin:
        host = peer.split(" ",1)[0]
        if any(item in blacklist for item in (host, '*.' + '.'.join(host.split('.')[-2:]))):
                print("blacklisted: ", host)
        else:
                print("ok:          ", host)

[peleion]

Awesome, thanks. I'll stop the firewall block and run your script.

[ecdsa]

@peleion your server is now in the greylist, for returning phishing servers.

[peleion]

Thanks for the heads up. Puzzling - I run ElectrumX 1.9.5 and Bitcoin core 17.1 compiled from github source in python-alpine dockers - absolutely nothing that isn't 100% stock.

After hsmith's comment above I dropped my iptables block and just let it run past 3 days - this morning had 497 "good" hosts on ElectrumX. Couldn't get hsmith's script to run (maybe a problem with docker output?) to verify but obviously (my) stock ElectrumX isn't blocking from your blacklist (hence this thread).

I've reinstated the firewall block from my "peer" list - shows only 18 discrete bad IPs but it is catching them all at the firewall and the host list is still empty after restarting. Also will recompile the docker - it pulls fresh from github every build - and get on IRC after letting it run a few hours like this.