matrix-org / waterfall

A cascading stream forwarding unit for scalable, distributed voice and video conferencing over Matrix
Apache License 2.0
97 stars 5 forks source link

simulcast: properly check the type of a track #104

Closed daniel-abramov closed 1 year ago

daniel-abramov commented 1 year ago

A minor thing that I noticed during the testing yesterday, namely that sometimes the SFU subscribed to a video track simulcast layer SimulcastLayerNone as if there was no simulcast on a video track (according to log there were 3 layers). I checked when this could happen. And found that only in 2 cases this is the case:

  1. There are no published simulcast layers (not the case).
  2. If metadata for this track has a width of height equal to 0 (this was a check for audio vs video track).

I replaced the (2) with a more reliable check for the type of media (we actually have always had this information) since it seems to be a better (reliable) way to check if it's a video track or an audio track.

Though we must later investigate if the client can send 0 as width or height for the video track by mistake.