libp2p / specs

Technical specifications for the libp2p networking stack
https://libp2p.io
1.55k stars 273 forks source link

Extend gossipsub with support for peer exchange #391

Open nazar-pc opened 2 years ago

nazar-pc commented 2 years ago

This is created based on comments in https://github.com/libp2p/rust-libp2p/issues/2398

Basically the idea is that when 2 peers are subscribed to the same topic and a third peer joins one of those 2, everyone should eventually discover everyone, in particular, when number of peers in a given topic mesh is below D_low.

This is not a replacement for "Ambient Peer Discovery", rather an additional mechanism that improves connectivity within topic mesh by using information from relevant peers.

vyzo commented 2 years ago

there is already support for PX on prune, so this would be very easy to add.

There are security considerations however (peer leaking, sybil poisoning, and so on) so this has to be carefully restricted to trusted nodes and gated by the score function.

Frederik-Baetens commented 2 years ago

I initially expected gossipsub to include peer discovery because I wasn't aware that Kademlia and Identify were required for peer discovery.

Now that I've learned how Kademlia & Identify are supposed to be used for peer discovery, I still think that this feature would be useful for those with simpler use cases. My use case doesn't even require any kind of byzantine fault tolerance, so I wouldn't even need functionality which ensures that these nodes introduced through gossipsub PX are "less trusted". But that functionality would be nice for a broader use case. It's nice to see that libp2p already has a score system to account for such things.

Could this peer exchange mechanism also ensure that the connection graph is at least k-connected (with k being configurable) so that the network is resistant against failure of any k-1 nodes? Or would that be too complicated to implement? I'm also not sure how often real world p2p networks actually encounter such problems.

vyzo commented 2 years ago

Yes, I agree, it would be useful for simple cases. The sole reason it was not included was security considerations, but we can do it with the gossipsub v1.1 mechanisms; we just have to be careful :)

I don't think you can make connectedness guarantees without any side protocol, and it doesn't look like it is worth the complexity.

nazar-pc commented 2 years ago

My concern is finding peers relatively rare topics on a huge network without resorting to some centralized rendezvous nodes or something like that. Doing random walking might take way too much time to be practical.

Connecting to one or a few peers that are part of the topic mesh should be sufficient to join the rest, this is similar to joining the network in general when you connect to bootstrap nodes that tell you about other nodes that are on DHT.