Add frame-rate and resolution chooser for each screenshare

[Bleuzen]

Is your feature request related to a problem you are facing?

Currently everyone in a room uses the same settings for screen shares. It is always at original resolution and always the frame rate of the room. However, often in rooms people share different content, which imposes different requirements for the screen share settings. For example:

Person 1 shares a coding session or text document. For this, frame rate is not important, so the default of 5 FPS is fine. However the resolution should be somewhat high to ensure text readability.

Person 2 wants to show a video or stream a video game. In this case, frame rate has a much higher priority while a little lower resolution is better in most cases to improve performance.

Both would need individual settings tuned for their screenshares content. Which is currently not possible and makes screenshares sub-optimal and unpleasant to watch.

Describe the solution you'd like

I think Discord is a good example of how to solve this problem. Every time someone wants to start a screenshare, they can select the resolution and framerate, to best fit the content. Jitsi could implement something similar. Example, taken from Discord:

Describe alternatives you've considered

None, I think the solution described above would be best in our use case. Currently don't have alternative ideas, but feel free to suggest something.

[damencho]

There is already such a setting

[Bleuzen]

@damencho

There is already such a setting

For frame rate, right, thanks, I missed that. Just tested it and it works... limited. In my first test, with just two participants, it didn't work at all. Figured out this settings only takes effect if p2p is disabled. Workarounds: a) Have more than two participants b) add url parameter config.p2p.enabled=false

But even after having p2p disabled, I set it to 30 fps, the video does not look like 30 fps. At times it is somewhat smoother compared to the default 5 fps, but it has a lot of frame drops, sometimes even freezes for 2 - 3 seconds... which makes it hardly better, probably even worse than the default 5 fps. At least in my first tests I just did, could be unlucky.

A setting for the resolution however is completely missing.

So, things left todo: 1) Add a setting for the screenshare resolution 2) Add 60 FPS to frame rate options 3) Find the cause of the frame drops (could be on my side, will test again over the next days) 4) Make it easier to change those settings (currently it is somewhat hidden, would prefer a pop up every time when starting a screenshare - if you think this could be annoying for some users, it could probably be made into a setting, something like config.screenshare.always-prompt-settings as url parameter or an Ask every time option in the frame rate drop down)

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[Bleuzen]

I still consider this as an very useful feature and valid request. Do not close yet.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[Bleuzen]

still important request, since without this feature Jitsi is very much unusable in some situations

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[IlijazM]

I don't understand, why this issue don't get more attention. I think this is a very important feature for jitsi and a mayor step forward.

[saghul]

Browser's screen sharing implementation was not really designed for high fps, so at this point we cannot really provide good results there.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[Bleuzen]

Browser's screen sharing implementation was not really designed for high fps, so at this point we cannot really provide good results there.

@saghul High FPS sharing is only a part of this issue. For example making the current options (5 - 30 FPS for now) more accessible would already be an improvement. I also suggested an option to select the resolution. This should help reducing CPU and network usage.

When it comes to high FPS sharing, there might be a way outside of the browser: OBS in its latest version is able to stream via WebRTC/WHIP. It is also able to encode video on the GPU, which makes high fps + high resolution video encoding possible without taking too much of a performance hit. Jitsi would have to provide a WHIP endpoint for this to work. Do you think this can be a suitable solution?

[saghul]

@saghul High FPS sharing is only a part of this issue. For example making the current options (5 - 30 FPS for now) more accessible would already be an improvement. I also suggested an option to select the resolution. This should help reducing CPU and network usage.

Adding setting won't magically make things work. If there is a way to indeed improve the status quo there, we'd consider it. Perhaps @jallamsetty1 can shed some light on the current state of affairs there.

When it comes to high FPS sharing, there might be a way outside of the browser: OBS in its latest version is able to stream via WebRTC/WHIP. It is also able to encode video on the GPU, which makes high fps + high resolution video encoding possible without taking too much of a performance hit. Jitsi would have to provide a WHIP endpoint for this to work. Do you think this can be a suitable solution?

That's a very heavy approach because it would require a WHEP endpoint on our end. If you are using OBS already, why not use the virtual camera option?

[jallamsetty1]

also suggested an option to select the resolution. This should help reducing CPU and network usage.

I do not think an option to select a resolution for screenshare is a good idea. The only way a particular resolution be enforced on a screenshare source is by applying constraints on the MediaStreamTrack that the browser returns. In the past we have seen tab crashes when the shared window (after resolution constraint is applied) is resized and also this doesn't guarantee a reduction in CPU or network usage as optimizations for encoding fewer bits for screenshare have already been implemented at the browser level. Adding or prompting for an fps value everytime someone starts a share will be an overkill for the generic cases. We already provide an API and a config.js setting for overriding the defaults on your deployment.

[jallamsetty1]

Moreover, the browsers are not designed to handle high resolution high fps streams properly. Please see https://bugs.chromium.org/p/chromium/issues/detail?id=1512472. So providing the options to capture the streams at high resolution/fps doesn't seem to add much value. Therefore, closing this issue.

[saghul]

@jallamsetty1 That issue is still open and there is a fix made 3 days ago rolling out in Chromium.

I don't know what that means in the grand scheme of things, but it doesn't seem to be a good indicator for the lack of support (or willingness to have it).