microsoft / MixedReality-WebRTC

MixedReality-WebRTC is a collection of components to help mixed reality app developers integrate audio and video real-time communication into their application and improve their collaborative experience
https://microsoft.github.io/MixedReality-WebRTC/
MIT License
913 stars 283 forks source link

Provide additional information on how to setup Desktop Server/Peer to communicate with Unity Server/Peer #765

Open ARGs-code opened 3 years ago

ARGs-code commented 3 years ago

Describe the problem This repository does a good job of setting up a basic Web RTC project and explaining the basics. But it does not provide any useful information on how to use the different platforms in tandem, only how to use the platforms with each other. Because the Unity solution uses Node-dss and the Desktop solution does not, naive users cannot figure out how to bridge the gap between the two and hook these two services together.

Describe the solution you'd like I would the documentation to explain how node-dss can be configured to connect to the Desktop solution detailed in the existing documentation and vice versa.

Describe alternatives you've considered I have tried to research this on my own and am continuing to do so, but as a naive user when it comes to server / client programming it is hard to know how to resolve this current issue and bridge this gap.

spacecheeserocks commented 3 years ago

I don't have a concrete example of this off the top of my head, but having done something like this recently...

The "Signaler" class is specific only to the Unity library, and is supplied as a convenience. The underlying C# library doesn't have any sort of explicit "Signaler" interface. However, it's (arguably) simpler to implement the same idea in C#.

In short, the functionality you have to implement in Desktop/Non-Unity is: 1) For every 'PeerConnection' object, subscribe to the LocalSdpReadyToSend and IceCandidateReadyToSend events.
When these events are fired, you should transmit the SDP Offer/Answer message or ICE message (in the same way that your unity Signaler class does with the methods you override). 2) When you receive incoming ICE messages from your signaling server, you should call AddIceCandidate on the appropriate PeerConnection object. 3) When you receive incoming SDP Offer/Answers from your signaling server, you should call SetRemoteDescriptionAsync on the appropriate PeerConnection object.

If you are handling more than a 1-to-1 call, you need to put in a little bit of extra work to identify which messages should be used with each PeerConnection object.

If you like, you could copy over your Unity Signaler script (there's no base class to inherit though, so remove the override keywords and the inherited class), and call the methods in that to handle this abstraction a little.

If you dig in the Unity library/scripts a bit, you can probably find how they use the Signaler classes. They definitely do some more work before handling Ice/RemoteDescription etc, which isn't all necessary outside Unity, but if you are having problems, that is a good place to start.