search

libp2p / specs

Technical specifications for the libp2p networking stack
https://libp2p.io
1.53k stars 272 forks source link

NAT traversal tracking issue #312

Open mxinden opened 3 years ago

mxinden commented 3 years ago

This GitHub issue tracks the status of NAT traversal capabilities across libp2p implementations and platforms.

See Hole Punching document for greater picture.

Please comment below to suggest additions or corrections.

Projects

The following projects combine various protocols from the table below to achieve NAT traversal:

robertkiel commented 3 years ago

Hi @mxinden,

thanks for the overview.

Remarks concerning hopr-connect:

I think the description misses one of the main issues when reasoning about NAT traversal: there is a bunch of smaller protocols such as STUN, TURN, UPnP, TCP, and QUIC. Each of them is able to solve their main purpose for example STUN allows the discovery of public IPv4 addresses.

But the main issue is that a combination of these protocols is required to successfully bypass NATs, and this makes handling unsuccessful attempts and automatic fallbacks necessary. Otherwise you end up with a transport module that always fails on certain internet setups. Unfortunately, this is the most tricky part.

mxinden commented 3 years ago

Updated the description:

I think the description misses one of the main issues when reasoning about NAT traversal: there is a bunch of smaller protocols such as STUN, TURN, UPnP, TCP, and QUIC. Each of them is able to solve their main purpose for example STUN allows the discovery of public IPv4 addresses.

I am not quite sure I follow. Each of these are listed in the table. An individual protocol in itself can not achieve NAT traversal by itself. Instead Projects, like HOPR connect, are combining various protocols to overcome a NAT.

But the main issue is that a combination of these protocols is required to successfully bypass NATs, and this makes handling unsuccessful attempts and automatic fallbacks necessary. Otherwise you end up with a transport module that always fails on certain internet setups. Unfortunately, this is the most tricky part.

Correct. This should be handled by the individual libp2p implementations. Today, if none of the direct connection approaches succeed, the fallback would be the circuit relay v1 protocol.

mxinden commented 3 years ago

Changelog

mxinden commented 3 years ago

Changelog

mxinden commented 3 years ago

Changelog

mxinden commented 3 years ago

Changelog

mxinden commented 3 years ago

Changelog

mxinden commented 3 years ago

Changelog

0xjjpa commented 3 years ago

Thanks for the update @mxinden!

@andreykiryushkin worth for you to take a dive on this now that you are exploring hopr-connect

mxinden commented 3 years ago

Below sequence diagram depicts the process of Hole Punching with Project Flare. I have yet to find the right place for this to live long-term. Still I am posting it here in case others are looking for a high level overview already today.

image

To reproduce via plantuml.com ``` @startuml participant Initiator participant Relay participant Responder collections Other_Peers mainframe Goal: The **Initiator** peer establishes a direct connection to the non-dialable **Responder** peer. == Responder determining whether it is dialable\n\nAutoNAT Protocol == Responder -> Other_Peers: Dial Request alt Dialable. No need for Hole Punching. Don't continue. Other_Peers -> Responder: Dial Response "Ok" else Not dialable. Hole Punching needed. Continue. Other_Peers -> Responder: Dial Response "Error" end == Responder finding closest public Relay nodes\n\n E.g. via Kademlia Protocol == Responder -> Other_Peers: Find nodes closest to Responder Peer ID Other_Peers -> Responder: List of nodes closest to Responder Peer ID == Responder listening for incoming connections via closest Relays\n\nCircuit Relay v2 Protocol == group For each closest Relay Responder -> Relay: Establish connection Responder -> Relay: Request reservation Relay -> Responder: Accept reservation end == Initiator establish relayed connection to Responder\n\nCircuit Relay v2 Protocol == Initiator -> Relay: Establish Connection Initiator -> Relay: Request connection to Responder Relay -> Responder: Request connection from Initiator Responder -> Relay: Accept connection request Relay -> Initiator: Accept connection request skinparam sequenceMessageAlign center Initiator <-> Responder: **Relayed Connection established** == Initiator and Responder coordinate simultaneous Hole Punch\n\nDirect Connection Upgrade through Relay (DCUtR) Protocol == loop until direct connection established Initiator -> Responder: Sync message hnote over Initiator: Measure round-trip time (RTT) Responder -> Initiator: Sync message Initiator -> Responder: Connect message hnote over Initiator, Responder: Simultaneously establish connection\nInitiator after 1/2 RTT, Responder when receiving Connect. Initiator <-> Responder: **Direct Connection established** end @enduml ```
mxinden commented 2 years ago

Changelog

cheako commented 2 years ago

https://discuss.libp2p.io/t/nat-traversal-with-libp2p/295 https://discuss.libp2p.io/t/how-to-construct-relay-peer/1112

cheako commented 2 years ago

https://gitlab.torproject.org/tpo/core/torspec/-/issues/64

cheako commented 2 years ago

There is also a transport that "talks" to a running instance of tor and configures it: https://github.com/berty/go-libp2p-tor-transport

mxinden commented 2 years ago

Changelog

mxinden commented 2 years ago

Changelog

Thus all protocols for basic hole punching in rust-libp2p are in master now :tada: If you want to punch holes yourself, check out https://github.com/libp2p/rust-libp2p/pull/2460.

0xjjpa commented 2 years ago

Hey @mxinden, seems like a lot of work has been done lately around the rust implementation. Would it be safe to say that teams looking to implement hole-punching from scratch using libp2p API would be better off doing it via Rust?

vyzo commented 2 years ago

go-libp2p is further ahead and supports QUIC which has better penetration succeess rate, so no this isnt an accurate statement.

mxinden commented 2 years ago

a lot of work has been done lately around the rust implementation.

Yes, a lot of progress in regards to hole punching.

Would it be safe to say that teams looking to implement hole-punching from scratch using libp2p API would be better off doing it via Rust?

rust-libp2p does not yet have support for QUIC (see https://github.com/libp2p/rust-libp2p/pull/2289) which bears higher hole punchings success rates given that it is based on UDP. For now, rust-libp2p only supports TCP hole punching. As mentioned above by @vyzo, go-libp2p is further ahead with QUIC support and a more battle tested hole punching stack.

TheRook commented 1 year ago

IPFS developed Delegate routers which can help connect nodes behind NAT as well as browser-based clients: https://github.com/ipfs/js-ipfs/blob/master/docs/DELEGATE_ROUTERS.md

Something like delegates could be ported to the libp2p core and a side-channel like DNS, or another bootstrap can be used to identify delegate nodes which can facilitate connectivity. IPFS Delegates do not support the full range of libp2p features like pub/sub.