status-im / nimbus-eth2

Nim implementation of the Ethereum Beacon Chain
https://nimbus.guide
Other
546 stars 233 forks source link

Public ip6 not shown in `last_seen_p2p_address` of Nimbus peer; nor in `p2p_addresses` of node identity #6591

Open yorickdowne opened 1 month ago

yorickdowne commented 1 month ago

Describe the bug

A dual-stack Nimbus with confirmed IPv6 connectivity from inside its container, running on Linux and Docker 27.3.1. Connectivity to the P2P port on the public ipv6 from outside has been tested and is working; tested with https://port.tools/port-checker-ipv6/

Its identity shows the ip6 ULA but not its own public ip6 in p2p_addresses. Should it be learning its ip6 from peers, like it might on ip4?

The public ip4 is visible in discovery_addresses. Example of curl http://localhost:5052/eth/v1/node/identity with actual public ip4 and enr censored:

{"data":{"peer_id":"16Uiu2HAkuovqjmjMNLQWURJ3yha381Xi3MgA4P5b5Wobx9BonAcs","enr":"enr:-myENR","p2p_addresses":["/ip6/::1/tcp/9000/p2p/16Uiu2HAkuovqjmjMNLQWURJ3yha381Xi3MgA4P5b5Wobx9BonAcs","/ip6/fdd9:ce96:f94c::9/tcp/9000/p2p/16Uiu2HAkuovqjmjMNLQWURJ3yha381Xi3MgA4P5b5Wobx9BonAcs","/ip6/fe80::42:acff:fe12:9/tcp/9000/p2p/16Uiu2HAkuovqjmjMNLQWURJ3yha381Xi3MgA4P5b5Wobx9BonAcs","/ip4/127.0.0.1/tcp/9000/p2p/16Uiu2HAkuovqjmjMNLQWURJ3yha381Xi3MgA4P5b5Wobx9BonAcs","/ip4/172.18.0.9/tcp/9000/p2p/16Uiu2HAkuovqjmjMNLQWURJ3yha381Xi3MgA4P5b5Wobx9BonAcs"],"discovery_addresses":["/ip4/123.245.123.245/udp/9000/p2p/16Uiu2HAkuovqjmjMNLQWURJ3yha381Xi3MgA4P5b5Wobx9BonAcs"],"metadata":{"seq_number":"2","syncnets":"0x00","attnets":"0x0000000003000000"}}}

Likewise, in the peer list, Nimbus peers show up without their public ip6, examples of curl http://localhost:5052/eth/v1/node/peers | jq . :

    {
      "peer_id": "16Uiu2HAmRvSPcHjJ2MJsrukgDtJVGNRzwNaMgwBNoL4ZWHqmBHsp",
      "enr": "",
      "last_seen_p2p_address": "/ip6/::/tcp/9002",
      "state": "disconnected",
      "direction": "inbound",
      "agent": "nimbus",
      "proto": "ipfs/0.1.0"
    },
    {
      "peer_id": "16Uiu2HAmQ5D6om2jkgksLQ3vRJGTzWeXVgJ7D4bkkKuCYefeuda1",
      "enr": "",
      "last_seen_p2p_address": "/ip6/::/tcp/9000",
      "state": "connected",
      "direction": "inbound",
      "agent": "nimbus",
      "proto": "ipfs/0.1.0"
    },

Note this second example shows the state connected - I am unsure whether this is an actual issue or just cosmetic. Or whether it connects and then goes to disconnected because it can't see the public ip6.

To Reproduce Run a Nimbus v24.9.0-f54a03-stateofus on an IPv6-capable host, so it uses dual-stack.

It'd be interesting to see behavior in systemd. I will only be testing this inside Docker containers, with Docker >= 27.3.1. Docker uses ULA addresses and then has ip6tables do a NAT66 for outgoing and host-mapped incoming ports.

cheatfate commented 1 month ago

I have tried to fix it in https://github.com/status-im/nimbus-eth2/pull/6595 , but core of the problem is in nim-libp2p implementation https://github.com/vacp2p/nim-libp2p/issues/1210