feat: rendezvous protocol initial implementation

[vasco-santos]

This PR adds the rendezvous protocol server implementation using the latest js-libp2p.

Spec: https://github.com/libp2p/specs/tree/master/rendezvous

This PR contains the server implementations, as well as some hardening and performance considerations. Take into account that this was a full rewrite of the module, so you do not need to have into account the older code.

Github actions CI only allows services to operate in linux. Other environments are not being used

[vasco-santos]

@jacobheun this should be ready for an initial review on the structure, API and rpc parts. I also left a few questions that would like to discuss ( @vyzo would like your feedback on them as well)

[carsonfarmer]

How likely is this to be merged any time soon? Looking really good!

[vasco-santos]

How likely is this to be merged any time soon? Looking really good!

Thanks for reaching out. Apologies for the lack of visibility on the state of this. This is currently blocked on a couple of things:

• Message signing https://github.com/libp2p/specs/issues/303
• Get benchmarking like https://github.com/libp2p/js-libp2p-rendezvous/pull/8 to a more reliable scenario using testground (js sdk is in development)
• Reviews/Deployments

It will likely not happen in the near future, unless we have more specific requirements from the community that push this work. The priority for this is being tracked in this project PR: https://github.com/protocol/web3-dev-team/pull/67

[vasco-santos]

As a status update on the Rendezvous Protocol, there is a new breaking change to the protocol that needs to be accounted for this PR https://github.com/libp2p/specs/pull/338

[nonchip]

unless we have more specific requirements from the community that push this work

@vasco-santos well one of those requirements is the fact https://github.com/libp2p/js-libp2p/issues/385 exists, the -stars are all deprecated and only work half the time anyway, this project is supposed to replace them, while js-libp2p claims to wanna work in a browser...

not sure how "web3" (:vomiting_face:) relates to this is or what other additional requirements have to be tracked to give "the only transport that actually works in a browser" priority, but actually being able to run libp2p and connect to enough nodes more than 5% of the time to even get a single pubsub message through on the same host even would be kinda nice.

[p-shahi]

@nonchip good news

the -stars are all deprecated and only work half the time anyway, this project is supposed to replace them, while js-libp2p claims to wanna work in a browser...

there are two solutions for js-libp2p to work in the browser. The first is js-libp2p-webtransport. Here's an example of using webtransport to fetch a file from Kubo https://github.com/libp2p/js-libp2p-webtransport/tree/main/examples/fetch-file-from-kubo

the other solution is native support for WebRTC. There are two parts to this. The first is browser-to-server connectivity via js-libp2p-webrtc which enables connecting to server nodes (i.e. a rust-libp2p or go-libp2p node running WebRTC). It's implemented per this specification: https://github.com/libp2p/specs/tree/master/webrtc#browser-to-public-server. This work is largely completed in js-libp2p and rust-libp2p. The go-libp2p pull request is currently in progress:

The second part of WebRTC is browser-to-browser: https://github.com/libp2p/js-libp2p/issues/1462 ; work on this is in progress and we hope to deliver by the end of February. Here's the specification: https://github.com/libp2p/specs/pull/497

So rest assured, there are plenty of ways of getting js-libp2p to work in the browser with support for native WebTransport and WebRTC.

[nonchip]

@p-shahi thanks!

The second part of WebRTC is browser-to-browser: https://github.com/libp2p/js-libp2p/issues/1462 ; work on this is in progress and we hope to deliver by the end of February. Here's the specification: https://github.com/libp2p/specs/pull/497

that's the bit i personally am relying on most because i'm building a 100% selfcontained single-file html "app" which can't expect a server ideally.

so far I'm using webrtc-star for both transport and discovery (the latter being "enhanced" by pubsub-peer-discovery since i'm heavily relying on pubsub anyway), but it's sadly very unreliable as soon as it gets "in the wild", apparently having issues with connectivity in general (dialling all kinds of invalid addresses), both gossip- and floodsub being unable to get anything through, and on top of all that apparently chrome has a bug where it just never deletes any closed webrtc connections so after like 3 minutes of seeing all kinds of things like ipfs nodes it simply refuses to let libp2p open more connections.

after weeks of careful tweaking it works like 50%ish reliably if i'm putting 4-6 nodes on the same non-public webrtc-star server, but any more or less than that, or if i just let them talk to each other for longer than like half an hour, and it still fails.

and with the various deprecations and stuff (and also the official webrtc-star server just crashing every few hours) i doubt i can improve my current codebase more with those dependencies, and the news of a better interoperable and specified implementation to come soon sounds definitely great!

---

Edit: i hope you forgive my frustration above (also @vasco-santos), and that you'll see why this project was total chaos for me, and then seeing a statement like "nah p2p in the browser won't happen for a while in js-libp2p" when it seemingly just never did anything (the 5% i mentioned weren't an understatement, took me weeks and careful tweaking of the "network size" to even be able to test it, and i can't rely on a fixed network size anyway, i wanna deploy that to users :P) was rather disheartening.