Peer Address Discovery & Relay

[innerop]

Hi @sinclairzx81 :-)

Cool stuff.

I went over the whole Readme and somehow I couldn't see how to get a list of all Nodes the are connected to a given Hub, or at least have some kind of DNS like functionality where I can give each Node a name and be able to access them from other Nodes that are connected to the same Hub.

The examples in the Readme all use Page Hubs for explanation purpose and so accessing them by reference is all I see in the docs. How do you access other Nodes if they are not on the same page, or even not in the same browser?

I can modify the Hub to act as a DNS so it can be queried like nmap to find all Nodes on the network and I can also make it associate a unique name with each Node where the name is chosen by the client and approved by the Hub (to eliminate conflicts)

My Scenario:

The Hub will be on a public IP address. The Nodes will be behind different NATs. Most of the Nodes are behind a normal NAT but a small percentage of them are behind symmetric NATs. Can I use the Hub or a Node (in a headless browser) that is on a public IP address in the cloud as a relay (like a TURN server)? If so, which of the two options? I need to access a given Node from other Nodes and the Node that will be accessed is almost always going to be behind a normal NAT.

Looking forward to your response.

[sinclairzx81]

@innerop Hi

Peer Discovery

Yes, i get asked this a lot. As of yet, smoke does not provide any peer discovery mechanism and places this burden on the end user. The reasons being there are a number of ways to handle peer discovery based on the application type, and I've been somewhat hesitant opting into one particular approach. However, you can do something like the following..

import { Node } from 'smoke-node'

const node = new Node({...})
const address = await node.address() 
const hostname = 'my-hostname'
const headers = { 'Content-Type': 'application/json' }

// Register hostname and address with external DNS like service.
await fetch('/register', {
    method: 'POST',
    headers.
    body: JSON.stringify({
       hostname,
       address
   })
}

Additionally, there are ways to implement peer discovery directly on the Hub, but it does involve making code updates to the smoke-hub to facilitate this (recommended to fork the project in these cases). In my experience, its generally more pragmatic for this functionality to exist in the hub, but it hasn't been implemented as part of this library yet. I am generally more open to expressing this functionality in the hub than I was when I initially wrote this project, but needs a proper design and weighing the centralising DNS services in the Hub (which is another reason this functionality was omitted).

Scenario

The Hub will be on a public IP address. The Nodes will be behind different NATs. Most of the Nodes are behind a normal NAT but a small percentage of them are behind symmetric NATs. Can I use the Hub or a Node (in a headless browser) that is on a public IP address in the cloud as a relay (like a TURN server)? If so, which of the two options? I need to access a given Node from other Nodes and the Node that will be accessed is almost always going to be behind a normal NAT.

This should work fine. Smoke doesn't diverge from standard WebRTC infrastructure requirements, so for users behind symmetric NAT, you will want to run a TURN server of some kind (so not necessarily a headless chrome instance).

As of writing, my preference is to leverage coturn, however the pion project also seems quite promising, but I haven't had any experience using it yet. The minimal services required to run Smoke with problematic NAT are as follows.

     +---------+             +----------+
     |  TURN   | <---------- |   hub    |        SERVER REALM
     +---------+    config   +----------+
      (coturn) \
         |      \
---------|--(a)--\--------------------------------
         |        \
   +---------+    +---------+     +---------+
   |  Node   |    |  Node   |-(b)-|  Node   |     BROWSER REALM
   +---------+    +---------+     +---------+         
     x.x.x.0        x.x.x.1         x.x.x.2                                     

(a) - symmetric NAT connection through TURN relay
(b) - p2p connection for open NAT

Note: the ICEConfiguration is configured on the Smoke-Hub. You can find information configuring the iceServers here

Hope that helps. S

[innerop]

Yes on all fronts.

My last question for now is about MediaStream fan out scenario.

It's actually a question about WebRTC. When you have multiple peer connections or data channels from one node to many others, does WebRTC perform compute intensive encoding for each connection/channel? or does it encode just once then send the encoded stream to all nodes? Do you happen to know?

[sinclairzx81]

@innerop My assumption is that the encoding is happening once when one obtains a MediaStream. In the fan out example, there is 1 MediaStream and N RTCPeerConnection. My guess is that the encoding happens behind the MediaStream (so it happens once) and the result of that encoding is piped over many RTCPeerConnection (so no additional encoding required). Intermediate Nodes would be piping through the encoding until such time as one of the nodes attempts to play it. In which case the decoders would kick in (via the <video> element)

There might be additional information you can find on the internal processes going on located here https://www.w3.org/TR/mediacapture-streams/ if its helpful.

Cheers

[innerop]

@sinclairzx81

I just found in one Chromium issue from 2015 that due to the constraints being applied in the context of each RTCPeerConnection, the browser encodes the media stream track separately for each connection. The issue offers a suggestion to use SVC to apply the constraints after the video is encoded, in the same way the bitrate is adapted separately in SVC after encoding, but that would not include codec-level constraints, so I'm not sure that they've solved this problem.

I could not find the follow up to the issue. The person who had commented on that issue referenced the wrong issue number. I can't assume that they've made progress on it.

In addition, I found out that each RTCPeerConnection instantiates its own copy of the WebRTC media stack. DataChannels don't have that overhead but then you'd have to send the raw uncompressed media track, which won't be practical for UHD, unless you use a WASM based codec first.

I could be wrong about some of this, but it seems that using an SFU with Simulcast is the current best option for scalable high quality realtime broadcast. There is one SFU that supports VP9 SVC, but in the end we'd be defeating the whole purpose of WebRTC if we go to a centralized scheme.

I love what you've built and it's a really wonderful abstraction on top of WebRTC, esp once you have the DNS sorted out. But I think I'm going to give up on WebRTC for high quality P2P broadcast unless I can determine for sure that we can have multiple connections served while encoding only once. Otherwise, it's ridiculously expensive on the client, and not practical for more than 4 peers or so.

[innerop]

Update:

I've confirmed with a member of the WebRTC project that sadly enough they never got to implement that idea, so WebRTC's multi-connection model is still very abusive to the CPU...

:/

I may look into compiling H264 software encoder to WASM

[innerop]

One more update:

Tried a WASM VP9 codec and needless to say it introduces lag and stutter on my MacBook Pro (3 year old model)

I could try VP8 in WASM but either way I’d lose SVC if I go with my own implementation over data channel.

I’ll need to think more about this but for now

I encourage you to consider adding DNS like capability You can use a WebRTC based DHT for DNS type queries.

Happy to collaborate on that.

Should I close this for now?

[sinclairzx81]

Hi, Thanks for the info on the media codec side of things. Interesting issue you linked here, might do a bit more digging into that.

Yes, will certainly be reviewing a number of options for peer discovery when I next get around to updating this project. On the DNS side, would be interested in a DHT backed DNS implementation. There are a variety of DHT implementations (Kademlia, Chord, etc) with varying performance characteristics, Im not entire sure which would be appropriate for this project. If you have insights into them, any info / implementation details you could provide would be great.

Feel free to close off this issue. If you're interested however, there is a very small channel on the freenode IRC network (irc.freenode.org), channel #smoke if you would like discuss technical details in a more free flow format.

Cheers S

[innerop]

I think a TTL feature is important for any DHT when it comes to Dynamic DNS. My assumption is DHT is used to synchronize records across Hubs but each Hub ought to have authority over the records it creates. Problem is once a given Hub creates an entry, another Hub can overwrite it, so entries must be write-protected and allow only the Hub that created the entry to update it or purge it (super user/admin should be able to purge it.) Nodes should only have read access to the DHT.

Would that work? Sure, I'll jump on IRC when I get some time and if/when I have some ideas to share.

[innerop]

I think maybe CRDTs like Automerge would be closer to what I’m describing than DHTs. In a DHT the data is scattered redundantly across many nodes but not fully replicated across all nodes. That wouldn’t be feasible without a merge protocol given nodes can be ephemeral and come on and off the network.

So what I was thinking about would be closer to using AutomergeJS at every Hub to keep the Hubs in sync. But then you can’t say which Hub has authority over which keys.

[sinclairzx81]

@innerop Thanks for the insights. Might close off this issue though. There are no immediate plans to incorporate a peer discovery mechanism into smoke (leaving this task for implementors). But may review a change in the hub api / protocol / signalling to make incorporating discovery mechanisms a little easier.

Thanks Again S