matrix-org / waterfall

A cascading stream forwarding unit for scalable, distributed voice and video conferencing over Matrix
Apache License 2.0
97 stars 5 forks source link

Screenshare transceiver reused for incoming screenshare without renegotiation #125

Open dbkr opened 1 year ago

dbkr commented 1 year ago

If you start screensharing in EC and then someone else starts swcreensharing too, you won't see their screenshare. This is because waterfall tries to re-use the transceiver created by the client for the screenshare publish to the SFU for the new screenshare down to the client. Waterfall calls AddTrack and the transceiver is in the recvonly state (from the SFU's PoV) so it's not unreasonable for pion to re-use it, but we'd expect a renegotiation from the SFU to change the direction to sendrecv (the client doesn't get one).

Options to fix:

The spec probably isn't super explicit on this, but the second option seems like it would keep things simpler.

dbkr commented 1 year ago

Having checked this in the debugger, actually this isn't quite true - we don't even seem to be calling AddTrack at all for the second screenshare. @EnricoSchw says this looks like pion mixing up the mids of the transceivers.