search

shaka-project / shaka-packager

A media packaging and development framework for VOD and Live DASH and HLS applications, supporting Common Encryption for Widevine and other DRM Systems.
https://shaka-project.github.io/shaka-packager/
Other
2.01k stars 511 forks source link

Packager Adaptation Set Switching doesn't work with forced command-line ordering #1402

Open SteveR-PMP opened 6 months ago

SteveR-PMP commented 6 months ago

System info

Operating System: all Shaka Packager Version: latest

Issue and steps to reproduce the problem

Issue 1393 was created to address a problem with adaptation-set-switching using representationIDs instead of adaptationsetIDs. The fix has a problem with the _--force_clindex. If _--force_clindex=false is set, the manifest is correct.

Packager Command: The problem becomes obvious with a large set of input files where the forced ordering makes a large difference. When used without _--force_clindex=false, the resulting manifest looks like this (grep "Adap|switching"):

<AdaptationSet id="0" contentType="video" maxWidth="960" maxHeight="544" frameRate="24000/1000" subsegmentAlignment="true">
  <SupplementalProperty schemeIdUri="urn:mpeg:dash:adaptation-set-switching:2016" value="22,24"/>
</AdaptationSet>
<AdaptationSet id="1" contentType="video" maxWidth="3840" maxHeight="2160" frameRate="24000/1000" subsegmentAlignment="true" par="16:9">
  <SupplementalProperty schemeIdUri="urn:mpeg:dash:adaptation-set-switching:2016" value="4,22"/>
</AdaptationSet>
<AdaptationSet id="2" contentType="video" maxWidth="1920" maxHeight="1080" frameRate="24000/1000" subsegmentAlignment="true" par="16:9">
  <SupplementalProperty schemeIdUri="urn:mpeg:dash:adaptation-set-switching:2016" value="4,24"/>
</AdaptationSet>
<AdaptationSet id="3" contentType="video" maxWidth="960" maxHeight="544" frameRate="24000/1000" subsegmentAlignment="true">
  <SupplementalProperty schemeIdUri="urn:mpeg:dash:adaptation-set-switching:2016" value="20,23"/>
</AdaptationSet>
<AdaptationSet id="4" contentType="video" maxWidth="3840" maxHeight="2160" frameRate="24000/1000" subsegmentAlignment="true" par="16:9">
  <SupplementalProperty schemeIdUri="urn:mpeg:dash:adaptation-set-switching:2016" value="19,20"/>
</AdaptationSet>
<AdaptationSet id="5" contentType="video" maxWidth="1920" maxHeight="1080" frameRate="24000/1000" subsegmentAlignment="true" par="16:9">
  <SupplementalProperty schemeIdUri="urn:mpeg:dash:adaptation-set-switching:2016" value="19,23"/>
</AdaptationSet>
<AdaptationSet id="6" contentType="video" maxWidth="1024" maxHeight="576" frameRate="12288/512" subsegmentAlignment="true" par="16:9">
  <SupplementalProperty schemeIdUri="urn:mpeg:dash:adaptation-set-switching:2016" value="21"/>
</AdaptationSet>
<AdaptationSet id="7" contentType="video" maxWidth="1920" maxHeight="1080" frameRate="12288/512" subsegmentAlignment="true" par="16:9">
  <SupplementalProperty schemeIdUri="urn:mpeg:dash:adaptation-set-switching:2016" value="0"/>
</AdaptationSet>
<AdaptationSet id="8" contentType="audio" lang="en-US" subsegmentStartsWithSAP="1" subsegmentAlignment="true">
</AdaptationSet>
<AdaptationSet id="9" contentType="audio" lang="en-US" subsegmentStartsWithSAP="1" subsegmentAlignment="true">
</AdaptationSet>
<AdaptationSet id="10" contentType="audio" lang="en-US" subsegmentStartsWithSAP="1" subsegmentAlignment="true">
</AdaptationSet>
<AdaptationSet id="11" contentType="audio" lang="en-US" subsegmentStartsWithSAP="1" subsegmentAlignment="true">
</AdaptationSet>
<AdaptationSet id="12" contentType="audio" lang="de" subsegmentStartsWithSAP="1" subsegmentAlignment="true">
</AdaptationSet>
<AdaptationSet id="13" contentType="audio" lang="es-MX" subsegmentStartsWithSAP="1" subsegmentAlignment="true">
</AdaptationSet>
<AdaptationSet id="14" contentType="audio" lang="es-MX" subsegmentStartsWithSAP="1" subsegmentAlignment="true">
</AdaptationSet>
<AdaptationSet id="15" contentType="audio" lang="es-MX" subsegmentStartsWithSAP="1" subsegmentAlignment="true">
</AdaptationSet>
<AdaptationSet id="16" contentType="audio" lang="es-MX" subsegmentStartsWithSAP="1" subsegmentAlignment="true">
</AdaptationSet>
<AdaptationSet id="17" contentType="audio" lang="fr" subsegmentStartsWithSAP="1" subsegmentAlignment="true">
</AdaptationSet>
<AdaptationSet id="18" contentType="audio" lang="fr" subsegmentStartsWithSAP="1" subsegmentAlignment="true">
</AdaptationSet>
<AdaptationSet id="19" contentType="audio" lang="fr" subsegmentStartsWithSAP="1" subsegmentAlignment="true">
</AdaptationSet>
<AdaptationSet id="20" contentType="audio" lang="fr" subsegmentStartsWithSAP="1" subsegmentAlignment="true">
</AdaptationSet>
<AdaptationSet id="21" contentType="audio" lang="pt-BR" subsegmentStartsWithSAP="1" subsegmentAlignment="true">
</AdaptationSet>
<AdaptationSet id="22" contentType="audio" lang="pt-BR" subsegmentStartsWithSAP="1" subsegmentAlignment="true">
</AdaptationSet>
<AdaptationSet id="23" contentType="audio" lang="pt-BR" subsegmentStartsWithSAP="1" subsegmentAlignment="true">
</AdaptationSet>
<AdaptationSet id="24" contentType="audio" lang="pt-BR" subsegmentStartsWithSAP="1" subsegmentAlignment="true">
</AdaptationSet>

What is the expected result? When used with _--force_clindex=false, the correct manifest is created:

<AdaptationSet id="0" contentType="audio" lang="fr" subsegmentStartsWithSAP="1" subsegmentAlignment="true">
</AdaptationSet>
<AdaptationSet id="1" contentType="audio" lang="en-US" subsegmentStartsWithSAP="1" subsegmentAlignment="true">
</AdaptationSet>
<AdaptationSet id="2" contentType="audio" lang="en-US" subsegmentStartsWithSAP="1" subsegmentAlignment="true">
</AdaptationSet>
<AdaptationSet id="3" contentType="audio" lang="es-MX" subsegmentStartsWithSAP="1" subsegmentAlignment="true">
</AdaptationSet>
<AdaptationSet id="4" contentType="audio" lang="en-US" subsegmentStartsWithSAP="1" subsegmentAlignment="true">
</AdaptationSet>
<AdaptationSet id="5" contentType="audio" lang="es-MX" subsegmentStartsWithSAP="1" subsegmentAlignment="true">
</AdaptationSet>
<AdaptationSet id="6" contentType="audio" lang="en-US" subsegmentStartsWithSAP="1" subsegmentAlignment="true">
</AdaptationSet>
<AdaptationSet id="7" contentType="audio" lang="fr" subsegmentStartsWithSAP="1" subsegmentAlignment="true">
</AdaptationSet>
<AdaptationSet id="8" contentType="audio" lang="fr" subsegmentStartsWithSAP="1" subsegmentAlignment="true">
</AdaptationSet>
<AdaptationSet id="9" contentType="audio" lang="es-MX" subsegmentStartsWithSAP="1" subsegmentAlignment="true">
</AdaptationSet>
<AdaptationSet id="10" contentType="audio" lang="pt-BR" subsegmentStartsWithSAP="1" subsegmentAlignment="true">
</AdaptationSet>
<AdaptationSet id="11" contentType="video" maxWidth="960" maxHeight="544" frameRate="24000/1000" subsegmentAlignment="true">
  <SupplementalProperty schemeIdUri="urn:mpeg:dash:adaptation-set-switching:2016" value="20,24"/>
</AdaptationSet>
<AdaptationSet id="12" contentType="audio" lang="pt-BR" subsegmentStartsWithSAP="1" subsegmentAlignment="true">
</AdaptationSet>
<AdaptationSet id="13" contentType="audio" lang="pt-BR" subsegmentStartsWithSAP="1" subsegmentAlignment="true">
</AdaptationSet>
<AdaptationSet id="14" contentType="audio" lang="pt-BR" subsegmentStartsWithSAP="1" subsegmentAlignment="true">
</AdaptationSet>
<AdaptationSet id="15" contentType="audio" lang="es-MX" subsegmentStartsWithSAP="1" subsegmentAlignment="true">
</AdaptationSet>
<AdaptationSet id="16" contentType="audio" lang="fr" subsegmentStartsWithSAP="1" subsegmentAlignment="true">
</AdaptationSet>
<AdaptationSet id="17" contentType="video" maxWidth="960" maxHeight="544" frameRate="24000/1000" subsegmentAlignment="true">
  <SupplementalProperty schemeIdUri="urn:mpeg:dash:adaptation-set-switching:2016" value="21,23"/>
</AdaptationSet>
<AdaptationSet id="18" contentType="video" maxWidth="1024" maxHeight="576" frameRate="12288/512" subsegmentAlignment="true" par="16:9">
  <SupplementalProperty schemeIdUri="urn:mpeg:dash:adaptation-set-switching:2016" value="22"/>
</AdaptationSet>
<AdaptationSet id="19" contentType="audio" lang="de" subsegmentStartsWithSAP="1" subsegmentAlignment="true">
</AdaptationSet>
<AdaptationSet id="20" contentType="video" maxWidth="1920" maxHeight="1080" frameRate="24000/1000" subsegmentAlignment="true" par="16:9">
  <SupplementalProperty schemeIdUri="urn:mpeg:dash:adaptation-set-switching:2016" value="11,24"/>
</AdaptationSet>
<AdaptationSet id="21" contentType="video" maxWidth="1920" maxHeight="1080" frameRate="24000/1000" subsegmentAlignment="true" par="16:9">
  <SupplementalProperty schemeIdUri="urn:mpeg:dash:adaptation-set-switching:2016" value="17,23"/>
</AdaptationSet>
<AdaptationSet id="22" contentType="video" maxWidth="1920" maxHeight="1080" frameRate="12288/512" subsegmentAlignment="true" par="16:9">
  <SupplementalProperty schemeIdUri="urn:mpeg:dash:adaptation-set-switching:2016" value="18"/>
</AdaptationSet>
<AdaptationSet id="23" contentType="video" maxWidth="3840" maxHeight="2160" frameRate="24000/1000" subsegmentAlignment="true" par="16:9">
  <SupplementalProperty schemeIdUri="urn:mpeg:dash:adaptation-set-switching:2016" value="17,21"/>
</AdaptationSet>
<AdaptationSet id="24" contentType="video" maxWidth="3840" maxHeight="2160" frameRate="24000/1000" subsegmentAlignment="true" par="16:9">
  <SupplementalProperty schemeIdUri="urn:mpeg:dash:adaptation-set-switching:2016" value="11,20"/>
</AdaptationSet>
vish91 commented 6 months ago

@cosmin @joeyparrish this is from some testing after the PR for https://github.com/shaka-project/shaka-packager/issues/1393 was merged. looks like its still failing from what @SteveR-PMP is testing.

unext-wendong commented 2 months ago

I think the problem comes from here. https://github.com/shaka-project/shaka-packager/blob/v3.2.0/packager/mpd/base/period.cc#L147-L155

  // Iterate thru AdaptationSets and add them to one big Period element.
  // Also force AdaptationSets Id to incremental order, which might not
  // be the case if force_cl_index is used.
  int idx = 0;
  for (const auto& adaptation_set : adaptation_sets_) {
    auto child = adaptation_set->GetXml();
    if (!child || !child->SetId(idx++) || !period.AddChild(std::move(*child)))
      return std::nullopt;
  }

It's reseting the AdaptationSet id without updating the adaptation-set-switching SupplementalProperty during the MPD generation.

It feels unnecessary to me considering that the AdaptationSets have already been sorted in the beginning of the function (i.e. Period::GetXml).