[BUG]: Many to many broadcast setup not working for more than two users

TdXYZ commented 2 years ago

Package version

3.1.0-exp.3

Environment

* OS: macOS Monterey 12.3.1
* Unity version: 2020.3.30f1
* Signaling: Websocket (Default settings in Render Streaming component)

Setup
* Local User who broadcasts audio and transform data to many clients.
* Local User has Custom Broadcast component similar to the Multiplay sample that OnOffer spawns remote users carrying receivers for audio and transform data whenever another client connects.
* Each person who connects is a local user and has remote users for any other person that connects and has connected. When a local user moves, the corresponding remote user moves as well.
* Essentially everyone broadcasts something to everyone else. It is NOT One to Many like in the Multiplay sample.

Steps To Reproduce

Client A connects to webserver (Enter Play Mode in Unity or start .exe)
Client B connects to webserver
Client C connects to webserver

Current Behavior

When client A and B connect, one remote user appears on each of them and when one client moves, the other can see that movement on the remote user, just as expected
When client C connects however, remote users for A and B do appear and a remote user for C appears on the other two clients, but they each cannot see the remote user move
Client A and B can no longer see their respective remote user move anymore
In Unity you can see this error in the console: "Failed to set remote answer sdp: Failed to apply the description for m= section with mid='0' Failed to set SSL role for the transport"

Expected Behavior

The expected outcome as seen in bullet point 1 in Current Behavior should work for more than two users

Anything else?

While debugging I think I managed to trace the origin back to OnSetSessionDescFailure inside RTCPeerConnection.cs
Is this sort of many to many setup even possible or can I only do a one to many setup like in the multiplay sample? Do I specifically need a SFU for this? Can one local data channel send their data to many remote channels or only to one?

karasusan commented 2 years ago

@TdXYZ

It is NOT One to Many like in the Multiplay sample.

Do you want to make the mesh network using p2p? I guess the web server provided by ours can not work for the mesh network.

TdXYZ commented 2 years ago

@karasusan The plan was to first test things out with a mesh network and then move on to an SFU. Are there any guidelines on how to set up an SFU with the render streaming package? We're currently trying to use Janus for that. Are there perhaps more suitable alternatives?

karasusan commented 2 years ago

@TdXYZ We have not tested it with SFU yet.

TdXYZ commented 2 years ago

If there are no plans to add many to many broadcast support in the future, then this issue can be closed,

karasusan commented 2 years ago

@TdXYZ We want to make the sample for integrating with SFU but I cannot promise when it is provided.

karasusan commented 2 years ago

memo: URS-107

karasusan commented 2 years ago

This thread in the Unity forum is interesting related to the issue. https://forum.unity.com/threads/providing-sfu-for-unity-renderstreaming.1315635/

eulersson commented 2 years ago

This is also related to this: https://github.com/Unity-Technologies/UnityRenderStreaming/issues/799

The possibility of connecting this wonderful plugin with existing media servers would be great. To perhaps create a new kind of signaling class that would sit inbetween URS and Janus VideoRoom.

karasusan commented 2 years ago

@docwhite Sure. SFU integraion is a good solution for users.

Unity-Technologies / UnityRenderStreaming