search

w3c / mst-content-hint

This is the specification for the content-hint attribute. This optional hint permits MediaStreamTrack consumers such as PeerConnection or MediaRecorder to encode or process track media with methods more appropriate to the type of content that is being consumed.
https://w3c.github.io/mst-content-hint/
Other
4 stars 14 forks source link

highly detailed text in video content #35

Closed aphillips closed 1 week ago

aphillips commented 4 years ago

MediaStreamTrack Content Hints, §2.2 Video Content Hints https://www.w3.org/TR/2018/WD-mst-content-hint-20180703/#video-content-hints

The track should be treated as if video details are extra important, and that significant sharp edges and areas of consistent color can occur frequently. This is generally applicable to presentations or web pages with text content. This setting would normally optimize for detail in the resulting individual frames rather than smooth playback, and may take advantage of encoder tools that optimize for text rendering. Artefacts from quantization or downscaling that make small text or line art unintelligible should be avoided.

Different written scripts might require different levels of detail (e.g., so that small features such as aspiration marks can be distinguished), and even a video content hint with a value of "text" might not be adequate for some scripts. Perhaps a "health warning" that a value of "text" provides no guarantee that textual features can be distinguished might be helpful.

This issue was originally reported by @stpeter. This is an I18N issue that we forgot to submit to you previously.

alvestrand commented 4 years ago

Given that the content-hint provides no guarantee, just "please try harder in this direction", a health warning seems appropriate.

aboba commented 10 months ago

This isn't just an internationalization issue. Encoder settings such as 4:4:4 can improve text legibility on colored backgrounds. Also, currently the libwebrtc implementation does not take the "text" hint into account in applying encoder settings, only whether the input track came from a screen share.

aphillips commented 1 month ago

This issue escaped tagging somehow. Adding the tracker label to ensure proper automation. A needs-resolution label might be added later, after the I18N WG re-discusses the issue.