search

w3c / mst-content-hint

This is the specification for the content-hint attribute. This optional hint permits MediaStreamTrack consumers such as PeerConnection or MediaRecorder to encode or process track media with methods more appropriate to the type of content that is being consumed.
https://w3c.github.io/mst-content-hint/
Other
4 stars 14 forks source link

Possible values, and are multiple values useful? #21

Closed jesup closed 6 years ago

jesup commented 7 years ago

So right now we have "detail" vs "motion", and "speech" vs "music". Can we conceive of any additional values? Might there be cases where you might want multiple descriptions of a track, if we add subsidiary descriptive terms? Or if you prefer sub-categories? It can be hard to predict future uses of things like this, and it's nice not to have to obsolete interfaces, just add new values.

pbos commented 7 years ago

I don't believe it's too useful to have multiple set descriptions of a track (I have no idea how an implementation would be expected to handle combining them (motion + detail + game). This current approach supports adding new values to a future spec as follows though:

This was purposefully behaving as an enum (do nothing if the value is not defined), so that an application selecting:

1: contentHint = "motion"; 2: contentHint = "anime";

would not break (unset contentHint on line 2) on implementations that don't implement the "anime" content hint. This approach permits adding values and falling back on less-specific ones where the specific one is not supported (e.g. set ["", "motion", "animation", "3d-animation"] in order, will set the most specific supported one, will set "animation" on an implementation supporting it, but not "3d-animation" if that's not supported).

If we think it's useful we could expose a way to query for supported hints, but I'm not sure if that's useful from a programmatic standpoint and I'm not sure that they would be reasonably surfaced to users directly. If so we'd need a human-readable description of it, and I'm not sure that's a good way to do it either way.

Maybe we should make explicit in the spec why it's done like this (to support future added values)?

alvestrand commented 6 years ago

I take the current state of this issue as "describe the reasoning behind using an enum, and make it explicit that we expect more values to be added in the future". Personally, I think the bar for adding new values should be reasonably high - only add if there's clear benefit to having new values - and am fine with a registration procedure of "revise the spec".

aboba commented 6 years ago

@jesup One reason to potentially have multiple values would be situations in which "detail" was not sufficient to optimize a screen capture. For example, is "detail" sufficient to decide whether to turn on block copy, or would we need another attribute, such as "text"?

alvestrand commented 6 years ago

I see benefit in a "text" attribute - the coding tools in HEVC for "text" are unhelpful when you just want a detailed picture of a slow-moving scene. Should we open a new issue for adding "text" as a possible value?

alvestrand commented 6 years ago

Closing this issue - conclusion is "no to multiple values".

aboba commented 6 years ago

@alvestrand Agree that so far there has been no use case demonstrating a need for multiple values. For example "text" along with "motion" does not make sense and if "text" would imply all the optimizations accorded to "detail" plus additional ones, then we can have a single "text" hint.

pbos commented 6 years ago

I assume "text" would normally imply and be a specialization of "detail"?

aboba commented 6 years ago

@pbos Yes, that is my assumption.