Browser-based WebRTC support

[sbazerque]

Hello! Any chances of supporting WebRTC in the browser?

Hyperswarm seems great, and supporting in-browser WebRTC swarming would add a lot of interesting use cases.

I have some experience doing swarm networking in the browser and could volunteer programming a transport implementation if there's interest.

[mafintosh]

Hi @sbazerque 👋

Biggest issue is that the discovery stack is UDP based so not really any APIs for that in the browser. Having said that, I’m def to to experimenting with browser transports.

How big swarms were you running over WebRTC? It used to quite resource intensive but mb that’s changed :)

[RangerMauve]

There's https://github.com/RangerMauve/hyperswarm-web which uses WebRTC for browser peers and relies on a proxy to connect to node peers which might be relevant.

[tinchoz49]

Hey @sbazerque, in discovery-swarm-webrtc we found a way to build a distributed swarm with optimal connections thanks to the @RangerMauve work on mmst

[sbazerque]

Hi @mafintosh !

Small ad-hoc networks, got a demo chat app for 2018 dweb camp, it's still up here: https://hyperhyper.space (the UI is... terse). It worked well on Chrome & Firefox but it's very light on the network (it's just text messaging).

I'm working on making that into a general p2p database library and was considering swapping my network code for hyperswarm.

So from what you say it would be hard, at least, to make the web-based peers interoperate with peers on real TCP/UDP. You could branch the discovery completely for web based peers, but that seems... inelegant, right?

Or have some peers operate as bridges, but I see your point.

[mafintosh]

@sbazerque If you ask me then I think you should experiment on

a) how many peers can the browser can the browser support today? b) what kind of throughput can webrtc support? c) whats the resource consumption in regards to o/ (mem, cpu etc) d) whats the interop story these days with node for webrtc?

If we had good answers to those it be much easier to frame this conversation. /cc @martinheidegger who I know is interested in this as well.

[sbazerque]

@mafintosh yeah! I don't have answers for those concerns, but I think the only way to move forward is by experimenting. I'll play a bit more with the implementation I have (and hopefully the ones mentioned by @RangerMauve and @tinchoz49 above) and maybe this will make more sense later on. Thanks for the quick replies!

[mafintosh]

Sounds good. This comes up a lot so would be good to share the findings here

[KyleMaas]

I'd be really interested to see something like this as well. I've been struggling trying to find a way to do this for DHT-based Secure Scuttlebutt connections in ssb-browser-demo, since it runs in the browser.

[sbazerque]

Hi @KyleMaas and all!

I've done a bit of thinking and testing since asking this question back in May. Here are my conclusions (this is still work-in-progress):

• Eventually I realized I wanted to have both browser-based and server-based peers, for both kinds to interoperate well, and I didn't want browser-based peers to be second-class citizens.
• WebRTC being so limiting requires to adapt all the transport layer (even how servers communicate) to make it browser-friendly, so I didn't end up using HyperSwarm but designing something from scratch.
• Since browser-based peers need a signalling server for WebRTC connection establishment, I refined signalling thus:
  • Two browsers don't need to use the same signalling server to establish a connection. Each peer just needs a signalling mechanism for incoming connection negotiation messages. Then signalling can be done by independent servers (in extremis, each peer may have her own signalling server!). Each peer still needs to know which signalling server(s) is being used by the one he's trying to connect to. This can be part of account discovery, or the signalling servers (that have no state besides who is listening on them) can work as a federated service and share this information using a DHT.
  • Since we need this signalling mechanism to be in place, we may as well piggyback on the signalling to do peer discovery.
• Server-based peers should communicate using WebSockets, but they should also use signalling servers sometimes. Then, connection establishment works as follows:
  • server -> server: just open a WebSocket to the destination server's WebSocket listening address.
  • browser -> browser: open a WebRTC connection, aided by the initiator and the receiver signalling servers.
  • browser -> server: open a WebSocket connection from the browser to the destination server's WebSocket listener.
  • server -> browser: through the destination browser's signalling server, send a special message asking the browser to initiate a WebSocket connection to the server's listening address.

I've implemented this from scratch in what has become the transport layer for Hyper Hyper Space's core library. It uses a very basic signalling server, it's basically a single Python function.

I've only done small scale testing, but it seems to work well.

[draeder]

Could a libraries like Bugout, or P2PT help with this? Both libraries operate in a similar way to each other using WebTorrent & DHT to establish a WebRTC swarm and both work in the browser, although P2PT's CDN script tag isn't working right now, so that would need to be browserified from the module.

[gfodor]

Bugout + P2PT still rely upon webtorrent trackers to act as signaling servers, so the robustness falls back onto relying upon the tracker network existing and being able to support the capacity of maintaining many live O(N^2) websocket connections, which is a fairly sizable requirement.

One track I've been working on recently is another angle: can you get signaling working entirely within the free tier of any cloud providers without any VMs? (to reduce outage or capacity bottlenecks) The answer is: yes, you can build a HTTP polling based signaling server on Cloudflare using workers + R2 for strongly consistent storage, which has a very generous free capacity. Holding open websockets right now is prohibitively expensive. Write-wise can support 500k-ish peer joins per month.

The same approach will work with eg lambda + s3 (in read-after-write regions) or any other situation where a provider offers a HTTP req/response FaaS + a strongly consistent durable KV store. If a large network of these were stood up it would basically be fairly decentralized, backstopped by multiple cloud providers and futher basically reliant on the legal entities that have the credit card holds for the accounts involved remaining in good standing. GCP, AWS, Cloudflare would all work probably several others.

These obviously can also gossip among themselves for providing service discovery. (You'd need to connect to the same endpoint for a given room to signal, ofc.) The services themselves could track usage and basically start throwing 429's if they're about to tip out of the free tiers.

[KrishnaPG]

webTorrent is much better alternative than webRTC.

[RangerMauve]

webTorrent is much better alternative than webRTC.

Webtorrent also uses WebRTC. 😅

[draeder]

@gfodor Just saw this: https://github.com/gfodor/p2pcf. Nice work!

[pubkey]

@draeder thank you, that is what I was looking for. Also I found this: https://github.com/subins2000/p2pt which might be related.