search

matrix-org / waterfall

A cascading stream forwarding unit for scalable, distributed voice and video conferencing over Matrix
Apache License 2.0
97 stars 5 forks source link

Lower res streams sent for screenshares #112

Closed dbkr closed 1 year ago

dbkr commented 1 year ago

The SFU seems to be sending me lower-then-expected resolution streams for screen shares meaning that I get a very pixellated screenshare feed where I can't read anything, unless I full-screen the screenshare.

The screen-sharer in this case was sending resolutions: 432x264 864x528 and 1728x1056 and EC was requesting 878x799 (which was the the pixel size the spotlit feed was ending up) but I was getting the lowest resolution: I'd expect to get the medium res one since thr requested size was greater than the size of the medium stream in both height and width.

daniel-abramov commented 1 year ago

I think I might have found the cause of this issue. The problem is the way SFU handles qualities, it expects:

And it calculates resolution as width * height. However, as a user, you probably expect medium (i.e. half resolution) to be half of its width and half of its height and that's where the bug in the SFU is 🙂 It assumed half resolution as (maxWidth * maxHeight) / 2 which was incorrect since the resolution of a tile is an area of a rectangle with sides equal to width and height. Halving each of the sides results in quartering the area (a/2 * b/2 = area/4).

So in reality, half resolution (as expected by the user) is (a * b) / 4, while the quarter resolution is (a * b) / 16.

I'll fix it right away and add some unit tests to cover possible cases.

SimonBrandner commented 1 year ago

Oh, of course :man_facepalming: