status-im / nimbus-eth2

Nim implementation of the Ethereum Beacon Chain
https://nimbus.guide
Other
514 stars 221 forks source link

Discovery send failed, Invalid argument when using localhost as listen address #4841

Open sealer3 opened 1 year ago

sealer3 commented 1 year ago

Describe the bug I am not able to discover any peers and get "Discovery send failed" warning message in the logs for each attempt. The logs also say there is an Invalid argument error message, which could be a real bug.

It looks like the same set of peer nodes is being attempted over and over, but nothing happens.

To Reproduce Steps to reproduce the behavior:

  1. Platform details (OS, architecture): linux amd64. Unsynced geth as EL client
  2. Branch/commit used: Latest in apt, v23.3.2 maybe
  3. Commands being executed: nimbus_beacon_node --data-dir=/srv/nimbus --el=http://127.0.0.1:8551 --jwt-secret=/tmp/jwtsecret --listen-address=127.0.0.1
  4. Relevant log lines:
    Discovery send failed                      topics="eth p2p discv5" msg="(22) Invalid argument" address=139.162.196.49:9000
    Discovery send failed                      topics="eth p2p discv5" msg="(22) Invalid argument" address=172.105.173.25:9000
    Discovery send failed                      topics="eth p2p discv5" msg="(22) Invalid argument" address=3.64.117.223:9100

    These logs continue forever with maybe 5-6 more IPs, trying to connect to them.

Additional context I didn't want to listen on all interfaces (listen address of 0.0.0.0). I thought setting listen-address to localhost would prevent nimbus from listening on all interfaces.

I could connect to peers once I removed --listen-address=127.0.0.1 from the arguments.

kdeme commented 1 year ago

127.0.0.1 is the loopback address (= localhost). You cannot use this as listen address if you want to connect to networks on the Internet, this would only be useful for testing locally on 1 machine.

If you want to use a specific --listen-address to avoid listening on all interfaces, then you must provide the IP address of the interface you want to listen on AND that has a route to the Internet (and is going over the public interface of which you have its IP set in the ENR). If you are behind a NAT, this is basically the interface that is connected to your router/gateway.

edit: To be more clear, the --listen-address option is the listen address used for p2p traffic, that is discv5 and libp2p. It doesn't have anything to do with for example el <-> cl traffic, which can be only local.

sealer3 commented 1 year ago

Usually when I think of the "listen address" I think of the IP address I am hosting a server on so other peers can connect to me, and not the one I use to connect to them, so I think that is what confused me.

Because the IP on my network interface changes often, I think I am looking for a way to bind to an interface by name and not by address. Ideally, there would be a way to set the socket option SO_BINDTODEVICE, as described here: https://unix.stackexchange.com/a/648721 SO_BINDTODEVICE binds a socket to a particular device like “eth0” that is passed into it.

But I don't know if Nim supports this option in its networking module, so maybe it's not possible for the moment to include it. It appears only boolean socket options are settable: https://nim-lang.org/docs/net.html#setSockOpt%2CSocket%2CSOBool%2Cbool