Aspect-ratio for video-tracks is not considered

[Antreesy]

How to use GitHub

• Please use the 👍 reaction to show that you are affected by the same issue.
• Please don't comment if you have no relevant information to add. It's just extra noise for everyone subscribed to this issue.
• Subscribe to receive notifications on status change and new comments.

---

Steps to reproduce

1. Join a call with an enabled camera. Device checker shows a video stream 'as-is', without any constraints (16:9 on example)

2. Video is cropped, because function _getConstraintsForQuality in VideoConstrainer.js fits only for 4:3 videos.

3. Switch to the grid view and back to the speaker view / Leave and rejoin / Disable and enable video device reinitialize video stream, and it is going without constraints:

Originally posted by @ShGKme in https://github.com/nextcloud/spreed/issues/8772#issuecomment-1437625786

Expected behaviour

Video should not be cropped (see step 3)

Actual behaviour

16:9 video is cropped by 4:3 constrain rules (see step 2)

Talk app

Talk app version: 16.0.0 and below

[danxuliu]

16:9 video is cropped by 4:3 constrain rules (see step 2)

This is actually the expected behaviour. Even if the camera supports a higher resolution in calls the resolution is requested to be 720x540 (the actual resolution can be a different one, depending on the browser, camera, operating system... although it should be something similar to that). This is a default value chosen for performance reasons, both for CPU load when encoding the video and for network bandwidth when sending it.

Showing the video in one resolution and sending it in another one is not a solution either; on one hand if I recall correctly there were issues depending on the browser when applying different constraints to a video track from the same source, and on the other hand the user should see what is being sent to others (even if the receiver client could crop part of the image, but at least as a sender you should see what you are sending). Additionally applying the blur background to two different videos would be too taxing.

Having said all that, Switch to the grid view and back to the speaker view / Leave and rejoin / Disable and enable video device should not cause the constraints to be lost, they should be applied again automatically, so something is not working as expected either in the SentVideoQualityThrottler, the VideoConstrainer or the media pipeline.