[Question] Is it possible to create a communication between 3 devices with MRTK-WebRTC?

[alxfigueiredo]

Hello to all. I am trying to create a communication between a PC, an Android Smartphone and a Hololens 2 using MRTK-WebRTC on Unity. These three devices need to communicate simultaneously using their Camera and Microphone. Unfortunately I understand that MRTK-WebRTC only allows a 1 to 1 connection (Tell me if I'm wrong and if can do it).

Does anyone know how I can establish such a communication please?

[aldenpang]

I am also very curious on this.

[spacecheeserocks]

WebRTC PeerConnections are all 1-to-1 only, but you can create more PeerConnections!

This works well for groups of 3 or 4, but that's about the reasonable limit. Larger groups begin to fail for many reasons (limits on simultaneous hardware decodes, and just the burden of sending/receiving so much data to different peers).

To set this up, it's the same idea as a 1-to-1 call, you just need to make another PeerConnection.

You must use the same AudioSource and VideoSource for all PeerConnections - otherwise it tries to capture the webcam twice and will fail.

Depending on your signalling solution, you will also need to make sure that you can reliably signal these connections multi-way. The Node-DSS sample wouldn't work out-of-the-box because it would mix together offers from 2 different peers. You could however just use a different LocalID for each peerconnection (e.g. PC1 <-> Holo1, PC2 <-> Android1 etc).

Some words of advice

1. You should investigate an SFU of some kind, such as MeetEcho Janus - specifically the "Meeting Room" module for this scenario. SFUs act as a centralised servers, where you can either negotiate all the peerconnections to a single server, or you can stick to just one peerconnection but ask the server to "switch" which video you're watching.
   SFUs reduce both the bandwidth and encoder burdens, and also Janus can work as your "signalling server".
   There will be a code overhead to using an SFU like Janus since you will have to handle Janus' way of signalling, SDP offers/answers etc, but if you want to support larger groups, it's worth it.
2. You will need to lower your performance expectations when handling multiple calls. For whatever reason, the HoloLens handles this particularly badly, even in 2D apps.
   When in a group call, you will need to drop the video resolutions (both sending and receiving).
   Many devices are only capable of decoding 2-4 simultaneous H264 streams in hardware, so if you have too many videos, they will either show up blank, or the performance will tank further if you're decoding high-res video on the CPU.
3. The UWP-side library really doesn't handle multiple calls very well, especially in Unity.
   If you're doing some small proof of concepts or very restricted scenarios, then proceed (with caution). Things generally work fine "the first time" and in "perfect scenarios", but unusual things happen when some peers leave or join the call.
   Unfortunately, if someone disconnects from the call, you pretty much have to completely stop and restart the UWP app to become stable again. I'm sure this is fixable but the bugs here are far deeper in the C++ library than I was willing to debug at the time.
   I would generally recommend a different library if you wish to do multi-person calls, but to my knowledge, there is no viable/working WebRTC library for UWP other than this one. WinRTC sounds like it's heading in the right direction, but was still in development and not usable when I last checked on it (about 6 months ago). WinRTC is also aimed at Windows/UWP, and does not include a Unity layer.

[alxfigueiredo]

Hello @spacecheeserocks,

Thank you for your valuable post! Your advice is very well received. Indeed, changing the IDs as you mentioned, made my calls work. I am indeed making a prototype, so it fits me well for now. For future development, I'll note your valuable information. Thanks a lot!