Feature Request: Option to Increase Screen Sharing Quality

[advil0]

It would be awesome to have some kind of setting to control screenshare quality, as right now when sharing a full desktop small text is kind of hard to read for some. I see in the docs that screenshares are limited to 1mbps, if we have more bandwidth available, is there a way to force it higher?

[streamer45]

Hi @advil0, thanks for your feedback.

This is a known limitation we purposely set since we don't yet have measures in place to send scaled down resolutions/bitrates to clients (simulcast/svc). Letting the sender set the bitrate would likely mean some users not being able to receive the video due to bandwidth constraints on the client side.

We have plans to address this in the coming months so keeping this open and will post an update when possible.

[Wh1t3Rose]

any updates with this?

[streamer45]

any updates with this?

We have a series of improvements planned, some of which are likely to get included in the next couple of releases.

[douglasparker]

any updates with this?

We have a series of improvements planned, some of which are likely to get included in the next couple of releases.

Any updates on this? It's almost been a year since you said there were plans to address this in the coming months and six months since your last update.

[streamer45]

@douglasparker We introduced an experimental option to enable simulcast in the latest release (v0.16) which should result in higher screen sharing bitrate and quality as a consequence. It can enabled from the admin console.

[douglasparker]

@douglasparker We introduced an experimental option to enable simulcast in the latest release (v0.16) which should result in higher screen sharing bitrate and quality as a consequence. It can enabled from the admin console.

Is there still no way to change the bitrate for self hosted instances? I prefer to be able to configure this setting myself.

[streamer45]

Nope. and to be honest I wouldn't recommend to manually set the bitrate since there's no guarantee other clients will be able to sustain it. Also web browsers don't really keep the bitrate constant so it's only possible to provide a maximum value.

Were you looking for a knob at the client level or globally?

[douglasparker]

Nope. and to be honest I wouldn't recommend to manually set the bitrate since there's no guarantee other clients will be able to sustain it. Also web browsers don't really keep the bitrate constant so it's only possible to provide a maximum value.

Were you looking for a knob at the client level or globally?

client side control is fine, but 1 Mbps in 2023 is pretty bad.

Whoever is screen sharing should be able adjust their bitrate and clients watching the stream should be able to transcode the feed down based on their own bandwidth constraints. Typically these days it's the upload bandwidth that is limited, but most people can easily stream higher than 1 Mbps.

I've seen a lot of companies try and limit bitrate and not give control to the people consuming the service and it's really off putting. Discord has this problem too, where even Nitro users screen share at less than 1 Mbps. If we're self hosting on our own infrastructure, it's likely we have at minimum a 1 Gbps pipe going both ways. Give us more freedom and control for the quality of our streams.

[streamer45]

I think you highlighted the problem very well there. In order to allow for that kind of control we'd need real-time transcoding in place which is not as easy as it sounds as it can become quite expensive, so we tend to play safe.

But in all fairness, I think the main problem with the quality here is not much the bitrate but the 15 years old video codec used (VP8). The best solution in my mind would be to move to state of the art video compression algorithms such as AV1 but it's still not widely supported (e.g. Apple device) and ultimately would need some engineering resources to make it happen, something we unfortunately lack these days. That said, project is open source so you never know.

[douglasparker]

I think you highlighted the problem very well there. In order to allow for that kind of control we'd need real-time transcoding in place which is not as easy as it sounds as it can become quite expensive, so we tend to play safe.

I can understand and respect this being expensive at scale, but I think it should be an option for self hosted Mattermost.

But in all fairness, I think the main problem with the quality here is not much the bitrate but the 15 years old video codec used (VP8). The best solution in my mind would be to move to state of the art video compression algorithms such as AV1 but it's still not widely supported (e.g. Apple device) and ultimately would need some engineering resources to make it happen, something we unfortunately lack these days. That said, project is open source so you never know.

The AV1 codec sounds great but definitely needs a fallback.

I'm personally looking for a Discord alternative that I can run on my own servers.

Higher bitrate streams (12 Mbps minimum) is something that I am actively seeking.

[kaksikanaa]

Just found this open issue in a situation where we have also seen some difficulties on low internet connections with the screensharing. We just wonder that the (jes quite old) SpreedME app on Nextcloud did it quite a bit better in similar conditions with screensharing. It uses the open sourced libde265 HEVC/H.265 video codec:

https://www.libde265.org/ https://github.com/strukturag/libde265

Would it be worth while to investigate as on option at least for real mattermost clients?

[streamer45]

The main problem is that H265 is not a royalty free codec which means there are licensing concerns. Also there's no widespread support on most systems as it's mostly used by Apple. I still think our best bet would be to add AV1 support as mentioned above but currently we can't find the resources to put much work on this sort of improvements.

[hbouttev]

There is another possible option to increase screen sharing quality for at least one usage.

In my team we use screen sharing of full desktop a lot for pair programming, so having clear text is the most important for us (reading for long time on heavy compressed video is tiring and annoying). At the moment we use Discord because, while it has poor default screen sharing quality, it also has a very nice feature: there is a "Better Text Readability" option to share at 5 FPS with the same bitrate (I suppose). It results in a crisp image thanks to the decrease of per image compression, at the cost of video frame rate, which for us is an acceptable compromise in this use case.

Could this be a possible solution to address part of the problem with the current architecture? Allow for an option to decrease the frame rate of the VP8 encoding while staying at a bitrate of 1 Mbps? Maybe a little more than 5, like 8 FPS or so, would be a good compromise too.

Screen sharing quality for pair programming is really the last thing preventing us to use Mattermost.

[streamer45]

Thanks for your feedback @hbouttev. More customizability on this front is definitely an option that wouldn't require massive effort. The only little blocker would be to give a practical way to users to tweak calls related settings, something we are looking into to cover other functionalities as well (e.g. custom shortcuts, volume levels and such). /cc @cpoile

[streamer45]

Not the ultimate solution to this problem, but the upcoming release comes with experimental AV1 support for screen sharing, which should give a noticeable improvement in terms of visual clarity.