Support for signaling HDR / HDR10+ in DASH with SupplementalProperty

[vish91]

System info

Operating System: all Shaka Packager Version: master

Issue and steps to reproduce the problem

Does Shaka packager support being able to pass additional information to the input while running packaging command ? For a multi codec DASH manifest , there is optional supplemental property for HDR .

<SupplementalProperty schemeIdUri="http://dashif.org/metadata/hdr" value="SMPTE2094-40"></SupplementalProperty>

DASH - IF Spec 10.3.2.4 HDR metadata.

What is the expected result? Optional parameter to pass key / value for the Supplemental Property to set. That property being set in resultant DASH manifest in the appropriate adaptation set.

[FiberOnePL]

Very useful option, I'm in favor of adding hdr info

[sr1990]

+1

Looks like HLS implementation supports specifying transfer characteristics.

https://github.com/shaka-project/shaka-packager/blob/d5ca6e84e6ccd543dc9a79ccc8d07b702010f491/packager/hls/base/master_playlist.cc#L244 and https://github.com/shaka-project/shaka-packager/blob/d5ca6e84e6ccd543dc9a79ccc8d07b702010f491/packager/hls/base/media_playlist.cc#L554 will add VIDEO-RANGE in m3u8. Example: #EXT-X-STREAM-INF:BANDWIDTH=820260,AVERAGE-BANDWIDTH=785273,CODECS="dvh1.05.01",RESOLUTION=640x360,FRAME-RATE=59.940,VIDEO-RANGE=PQ,CLOSED-CAPTIONS=NONE

Similarly, for DASH, one way would be to add a supplemental property to specify transfer characteristics as per DASH-IF IOP v4.3 6.2.5.1. the CICP for transfer characteristics can be indicated in an essential or supplementary property at the AdaptationSet level (given that representations within a set shall not differ for that property), using schemeIdUri="urn:mpeg:mpegB:cicp:TransferCharacteristics"

Example: <SupplementalProperty schemeIdUri="urn:mpeg:mpegB:cicp:TransferCharacteristics" value="16" />

This supplemental property is also supported by shaka player as per https://github.com/shaka-project/shaka-player/issues/3726. VideoConfiguration is created with the transfer function that is mentioned in the supplemental property and is used as a parameter for decodingInfo API.

Implementation could be:

1. Add transfer characteristics as AdaptationSet (adaptation_set.h) data member.
2. While creating the adaptation set https://github.com/shaka-project/shaka-packager/blob/d5ca6e84e6ccd543dc9a79ccc8d07b702010f491/packager/mpd/base/period.cc#L76 a. Add transfer characteristics in adaptation set key https://github.com/shaka-project/shaka-packager/blob/d5ca6e84e6ccd543dc9a79ccc8d07b702010f491/packager/mpd/base/mpd_utils.cc#L146 b. Initialize AdaptationSet transfer characteristics data member in https://github.com/shaka-project/shaka-packager/blob/d5ca6e84e6ccd543dc9a79ccc8d07b702010f491/packager/mpd/base/period.cc#L200
3. While creating mpd https://github.com/shaka-project/shaka-packager/blob/d5ca6e84e6ccd543dc9a79ccc8d07b702010f491/packager/mpd/base/adaptation_set.cc#L242 add the supplemental property if transfer characteristics is present.

@vish91 , @joeyparrish / @kqyang , please take a look.

[SteveR-PMP]

@vish91 @joeyparrish @kqyang @sr1990

I came up with some code near the end of RepresentationXmlNode::AddVideoInfo() that will add the SupplementalProperty for transfer characterstics:

  if (video_info.has_transfer_characteristics()) {
    std::string transfer_characteristics =
      base::UintToString(video_info.transfer_characteristics());
    RCHECK(AddDescriptor("SupplementalProperty",
                         "urn:mpeg:mpegB:cicp:TransferCharacteristics",
                         transfer_characteristics));
  }

This follows how it is done in HLS and how SupplementalProperty is done for audio in RepresentationXmlNode::AddAudioChannelInfo() https://github.com/shaka-project/shaka-packager/blob/d5ca6e84e6ccd543dc9a79ccc8d07b702010f491/packager/mpd/base/xml/xml_node.cc#L556

This should fulfill the request without needing to create a new data member in AdaptationSet. Does anyone see a problem with this solution?

[vish91]

Yeah transfer characteristics work too... infact in the latest DASH-ID IOP I cannot find this HDR metadata signaling, but the signaling using transfer characteristics is still there in the spec. https://dashif.org/guidelines/iop-v5/

@sr1990 @joeyparrish any thoughts ? what do you do on shaka-player or ExoPlayer side if you know from player perspective what is better. Also do we know the possible values for Transfer Characteristics and what they are from spec ? its in ISO-IEC spec but haven't gotten my hands into that yet.

[sr1990]

@vish91, check out the shaka player issue https://github.com/shaka-project/shaka-player/issues/3726 and PR that it mentions related to parsing the transfer characteristics. Don't see exoplayer parsing out VIDEO-RANGE in https://github.com/google/ExoPlayer/blob/release-v2/library/hls/src/main/java/com/google/android/exoplayer2/source/hls/playlist/HlsPlaylistParser.java

For values of transfer characteristics, check out https://www.itu.int/rec/recommendation.asp?lang=en&parent=T-REC-H.265-202108-I Table E.4 – Transfer characteristics interpretation using the transfer_characteristics syntax element Transfer characteristics are parsed out from SPS VUI - check H265Parser::ParseVuiParameters and H264Parser::ParseVUIParameters Also, https://github.com/shaka-project/shaka-packager/blob/d5ca6e84e6ccd543dc9a79ccc8d07b702010f491/packager/hls/base/media_playlist.cc#L554 will probably need to be updated for case 18 which should be HLG

[cosmin]

@vish91 can we consider this closed now that transfer characteristics are in? Or do we still need this specific supplemental property?

[vish91]

yes sure thing. 👍🏽 closed by https://github.com/shaka-project/shaka-packager/pull/1210