Expose user preferences for text tracks to JS

[zcorpan]

See http://www.w3.org/2015/10/28-htmlcue-minutes.html

for example the browse might need to expose the subtitles preferences so that the js renderer know what to do

When using a metadata track and render the payload with JS for things that are actually subtitles or captions (just with a different cue text format), we should expose the user preferences to JS so it can enable the right metadata track.

This information is already available because the JS gets to know which subtitle or caption track gets enabled, which depends on user preference.

An alternative idea (we could do both) is to let user preference signal somehow metadata tracks as if they were subtitle or caption tracks, with a separate attribute.

<track src="foo" kind="metadata" actualkind="subtitles" label="English" srclang="en">

steps 1-3 in https://html.spec.whatwg.org/multipage/embedded-content.html#honor-user-preferences-for-automatic-text-track-selection would also apply to metadata tracks with the appropriate actualkind and set a flag on the track that the user wants it showing (e.g. set text track mode to "showing" which I think is equivalent to "hidden" for metadata tracks). [I misremembered how metadata tracks get enabled. I thought they were all set to hidden, but it's just the ones that have a default attribute that get set to hidden. So then it would make more sense to have the metadata tracks with actualkind be set to hidden per user-pref for the actualkind, and the rest are left disabled.]

[foolip]

While there is a user preference in Blink with the values "default", "captions" and "subtitles", there's also a heuristic that takes the language into account, which calculates a score for each track.

[foolip]

Another thought. If one could register a callback that is handed a Response object and a TextTrack to populate, and if this mechanism is orthogonal to kind="metadata", then the kind attribute could be used to describe the actual kind.

[zcorpan]

I don't follow. Elaborate?

[foolip]

OK, so this would be a rather big undertaking. As it is, the <track> URL is fetched and then handled by one of potentially multiple built-in parsers, although only WebVTT is in all engines. If we had a way to register a JavaScript callback to handle the fetch response instead, as well as the primitives needed for scripts to render cues in the same Shadow DOM that is between the video and its controls, then I'm guessing that would be a better fit than using WebVTT metadata cues, in which case one wouldn't have to use kind="metadata" for things that are intended to work just like other captions/subtitles, except implemented by scripts.

[zcorpan]

Ooh, OK. Yeah that seems like a big project but little benefit. It seems it's fine to overlay the video, we just need to address the user-preference thing.

[foolip]

I would say that the ability to implement WebVTT entirely using such extension points and thus also allowing any other format to be implemented with scripts on equal footing is a rather big benefit, but yes, it's still a big project.

I'm rather reluctant to have kind="metadata" be used both for hidden metadata and for supporting non-WebVTT formats that should be rendered, to have an actualkind really points to something being wrong with kind itself. So... maybe a new attribute that means that kind="captions" shouldn't be rendered by the browser itself?

[zcorpan]

WFM. Should hidden attribute have that semantic when used on track?

[foolip]

Something about that seems unusual, but I don't see why it would be a problem, and it is the perfect attribute name for this. :+1: