marcoscaceres
commented
11 months ago Discussed with colleagues internally. We think what is being proposed is suitable, provided the hint definitions are not too vague and it is clear how to use them to tune encoders.
Unless we hear otherwise over the next week, we are inclined to label this with "support".
WebKittens
@youennf
Title of the spec
VideoEncoderConfig's contentHint
URL to the spec
https://www.w3.org/TR/webcodecs/#dom-videoencoderconfig-contenthint
URL to the spec's repository
https://github.com/w3c/webcodecs
Issue Tracker URL
No response
Explainer URL
https://gist.github.com/Djuffin/c3742404b7c53ada227849c8b2b76b4c
TAG Design Review URL
No response
Mozilla standards-positions issue URL
No response
WebKit Bugzilla URL
No response
Radar URL
No response
Description
Problem and Motivation
Webcam content often require denoising and is often intelligible even when downscaled or with high quantization levels. Screencast content of presentations or webpages with a lot of text content is completely unintelligible if the quantization levels are too high or if the content is downscaled or otherwise blurry.
Video encoding libraries (like libvpx, OpenH264) provide a way for API users to inform the encoder about type of the encoded content. In some cases the web application can make a more-educated guess or take user input, but currently there is no way to inform
VideoEncoderwhat kind of content is being encoded.Proposed solutions
Add a new field to VideoEncoderConfig
Let's add a
contentHintfield toVideoEncoderConfig. It will take video content hint values that are already used forMediaStreamTrack: "motion", "text", "detail".This gives web developers a way to communicate to
VideoEncoderthe expected type of the content they intent to encode. This setting is not intended to replace encoder-level settings completely but rather complement them with a simpler hint that does not require broad knowledge of video encoders.