Libp2p transport or P2P Hub?

[adlrocha]

Libp2p is a networking stack and library for p2p protocols. It supports several transport, security, pubsub, and routing protocols. Grav could really benefit from supporting libp2p as a transport as it offers a lot of nice features from scratch that could help Grav scale in certain decentralized environments.

Leveraging a libp2p transport in Grav could also add service mesh discovery (#32) and a pubsub transport (#15) by design.

Despite all this, for the best of my knowledge libp2p has not been extensively deployed in cloud environments (traditionally less aggressive than p2p networks). Thus, my plan is to build a basic go-libp2p transport plugin for grav, evaluate how it performs compared to the existing transports, and understand if it is worth it supporting additional integration and embedding with libp2p protocols.

This could maybe also open the door to this? :)

[adlrocha]

After doing a first exploration of libp2p as a transport, I realize libp2p may not suit a Grav transport but an alternative Grav hub. The fact that you can already use several transports within the same libp2p node, that transport and discovery are really couple in libp2p (due to its p2p nature), and that you can't run raw libp2p connections without creating a libp2p node (all connections are encrypted, so nodes need to be uniquely identified with a key pair), led me to think that it'd maybe make sense to add an alternative implementation of the hub with p2p capabilities (so the user can choose which one to use), or to implement a p2phub so both alternatives can be used in parallel.

Libp2p already does handshakes, endpoint discovery and dialing, etc. and I was seeing myself re-implementing many of the things already in libp2p and including a lot of glue code to make it work within Grav. I will explore this new approach and report back with results.

PS: This commit doesn't include all the things I tried for libp2p in Grav. I didn't push the rest of the code because it was really dirty after so many testing, and I chose not to clean it, if someone is interested in seeing how I reached the "libp2p can't be a transport in Grav" dead end let me know"

[cohix]

@adlrocha interesting, so you're suggesting that the hub should be an interface that the caller can provide as a plug-in?

Hmm I could certainly see that being possible. We could provide a DefaultHub which is basically the current implementation (you give it your Transport and Discovery of choice), or allow you to load your own hub with Transport and Discovery built in.

What do you think?

[adlrocha]

That's it! The DefaultHub accepts Transport and Discovery plugins, but then there is a Hub plugin so that Grav can accommodate transports and technologies with the discovery already embedded (i.e. where is hard to decouple the protocols). I think the DefaultHub is key though, because it aggregates disjoint protocols together. I am also starting to think that having a PubSub plugin in the hub instead of implementing it as a transport (as suggested here) could also offer a lot of flexibility, as developers could come up with their own PubSub algorithms built upon existing transports, or just plug-in existing PubSub protocols as a transport (this needs a bit more of thinking though).

In any case, I will try to allocate some time this week to have hub as an interface and a P2P Hub, and share the code see what you think.

[cohix]

@adlrocha sounds awesome! Feel free to book time with me to go over it when you have a prototype :)