Is there an issue with Supplemental data and capability exchange?

[johnsim]

CMAF Amd 2 adds the notion of supplemental data and supplemental data brands. This generalizes the functionality available in the first release of CMAF for 608/708 captioning (‘ccea’). It is suggested that this creates an ambiguity during capability exchange.

For example, ‘chd1’ is the WAVE video profile for HDR10. A call to isTypeSupported(mimeType) where mimeType=’”video/mp4; profiles=”cdv1,chd1”, codecs=="hev1.2.4.L153.B0”’ would be unambiguous - HDR10 with Dolby Vision dynamic metadata - but what would be the correct response if mimeType=’”video/mp4; profiles=”chd1”, codecs=="hev1.2.4.L153.B0”’?

[KilroyHughes]

In a device with a 'chd1' decoder, the appropriate response to profiles="chd1" would be yes, and to profiles="cdv1, chd1" would usually be "maybe".

The exception would be a device using its built-in display and GPU with 'cdv1' render capability and a customized user agent that was aware of that fact. The main example would be an internet TV using a streaming app. It should decode the HDR10 content and will use or ignore the 'cdv1' metadata, which could improve the accuracy of the tone mapping applied.

In the majority of HDR display systems now and for the next several years, a streaming device does the decoding, and a "TV" display connected via HDMI does the tone mapping to its particular display capabilities. Today, most TVs ignore static or SMPTE dynamic metadata present and use proprietary algorithms to analyze and tone map whatever color volume is encoded to whatever color volume they can display. The display doesn't tell the device doing the decoding how it will tone map, and the HDMI connection doesn't surface external display information to the user agent (like the display's dynamic range or how much attention it pays to what rendering metadata during tone mapping) so the user agent can't answer this capability query from a web page accurately. It will decode and display, but the method and quality of tone mapping is might look good or bad on different pieces of content.

That said, we will never improve that situation unless we profile SMPTE dynamic metadata for HDR rendering and define a brand that can be signaled in content, and eventually supported in devices. It's a chicken/egg problem.

[haudiobe]

I do not understand the value of creating a media profile, for which CTA WAVE does not understand on what the requirements in order to claim support for it on a device. I am ok if the spec says that you need to run through a Dolby certification process and we add a reference. But for improving interoperability in Internet TV, we cannot ignore these issues.

[rdoherty0]

I believe this should be straightforward.

For a client that can playback HDR10, isTypeSupported should return True for profiles="cdv1,chd1" and also return True for profiles='chd1'. Indeed the second case has nothing to do with Dolby Vision -- that's the default behavior for HDR10 profiles.

The file brand convention is simply an assistant to signal the inclusion of optional metadata in the stream, to help clients choose streams if they want the additional functionality. The metadata is optional and ignorable -- there is no case where the addition of the metadata creates an incompatibility and therefore interoperability is assured.

The value to WAVE is the documentation of this profile and approval of Supplemental Data for inclusion in players and test regimes.

[haudiobe]

I still do not understand what it means, so the media profile cdv1 has NO device capabilities assigned? I still do not see any value in documenting something. On the opposite, I find it unfortunate to document something that is breaking concepts and creates confusion without value. In the above logic the player needs to understand that cdv1 is not to be checked, but chd1 is? So we have file brands that are different from other file brands?

[rdoherty0]

These issues are important, but are more global than the cdv1 profile, and not relevant to the approval of the profile and the value of cdv1 to implementers and the ecosystem.

In particular, issues regarding the supplemental file branding needs to be understood and solved for the 608/708 subtitle formats as well. The brands are supplemental, yes, this is part of the MPEG specification. It is well documented and expected client behavior for those clients that wish to consume supplemental content. For those clients that do not wish to consume that content, it can be ignored and is still compatible and interoperable.

[johnsim]

From cstf conference call: Proposed resolution:  

MPEG made a decision, but other pieces are needed to make this  practical to use. The missing features are in HTML5 media capabilities. In fact, HTML5 media capabilities cannot be used to determine ‘chd1’ or  ‘vp9d’ support, because the profile subparameter is not supported.   We have approved media profiles despite the limitations of HTML5  media capabilities, so we should not block supplemental data profiles  solely because of HTML5 media capability limitations.

• Mr. Stockhammer noted he remains concerned with this 

• Mr.Law suggested a fourth step, to submit this problem statement to the Media  Capabilities API Incubator for use as a litmus test for future API work. 

It was agreed that, given this resolution, issue 16 will not block csv1 approval.  However, there are other issues (18, 19) that need to be resolved before  approving the ‘cdv1’ media profile. 

[johnsim]

o Proposed resolution:   It was agreed in our last meeting that we need to establish a process for  approving supplemental data profiles, likely an update to our existing  media profile approval process, including the market relevance  distinction between “existing relevance” and “new with WAVE Support”.  See issue #23.   The supplemental data profile ‘cdv1’ should be required to meet the  “new with WAVE Support” market relevance bar, meaning support  from 6 WAVE members, with a minimum of two services and two  device providers.   o Mr. Doherty agreed this is reasonable.   o Mr. Simmons noted this issue is blocking the acceptance of cdv1, since Dolby  must build consensus from 6 WAVE members to be supportive of the cdv1  profile to meet the market relevance requirement. The cdv1 supplemental data  profile will not be accepted until Dolby returns with a list of supporting WAVE  members.