Support segmentTemplate on Representation level

[tysonite]

I understand that livesim2 does not support it now. Even though, I would like to file such enhancement request as some our usage scenarios of livesim2 expect more than one representation.

We use MP4Box to generate MPD. E.g.:

MP4Box  -dash 2000 -rap -frag-rap -dash-profile simple  -out dash/mpd -segment-name \$RepresentationID\$\/segment_\$Time\$ -segment-timeline -url-template paris-720p.mp4#video paris-720p.mp4#audio paris-1080p.mp4#video

The MPD looks like that:

<?xml version="1.0"?>
<!-- MPD file Generated with GPAC version 2.5-DEV-rev298-g2e71a1ec-master at 2024-06-29T12:41:07.850Z -->
<MPD xmlns="urn:mpeg:dash:schema:mpd:2011" minBufferTime="PT1.500S" type="static" mediaPresentationDuration="PT0H17M14.936S" maxSegmentDuration="PT0H0M6.240S" profiles="urn:mpeg:dash:profile:full:2011">
 <ProgramInformation moreInformationURL="https://gpac.io">
  <Title>mpd.mpd generated by GPAC</Title>
 </ProgramInformation>

 <Period duration="PT0H17M14.936S">
  <AdaptationSet maxWidth="1920" maxHeight="1080" maxFrameRate="30000/1001" par="16:9" mimeType="video/mp4" startWithSAP="1">
   <SegmentTemplate media="$RepresentationID$/segment_$Time$.m4s" initialization="$RepresentationID$/segment_.mp4" timescale="30000" startNumber="1"/>
   <Representation id="1" codecs="avc1.64001F" width="1280" height="720" frameRate="30000/1001" sar="1:1" bandwidth="2041340">
    <SegmentTemplate startNumber="0">
     <SegmentTimeline>
      <S t="0" d="160160" r="3"/>
  ..............
     </SegmentTimeline>
    </SegmentTemplate>
   </Representation>
   <Representation id="3" codecs="avc1.640028" width="1920" height="1080" frameRate="30000/1001" sar="1:1" bandwidth="4306226">
    <SegmentTemplate startNumber="0">
     <SegmentTimeline>
      <S t="0" d="160160"/>
..................
     </SegmentTimeline>
    </SegmentTemplate>
   </Representation>
  </AdaptationSet>
  <AdaptationSet segmentAlignment="true" lang="en" mimeType="audio/mp4" startWithSAP="1">
   <SegmentTemplate media="$RepresentationID$/segment_$Time$.m4s" initialization="$RepresentationID$/segment_.mp4" timescale="44100" startNumber="1">
    <SegmentTimeline>
     <S t="0" d="88064" r="6"/>
   .................
    </SegmentTimeline>
   </SegmentTemplate>
   <Representation id="2" codecs="mp4a.40.2" audioSamplingRate="44100" bandwidth="127999">
    <AudioChannelConfiguration schemeIdUri="urn:mpeg:dash:23003:3:audio_channel_configuration:2011" value="2"/>
   </Representation>
  </AdaptationSet>
 </Period>

livesim2 does not parse it and log the following error:

Asset loading problem. Skipping","asset":"mpd.mpd","err":"segmentTemplate on Representation level. Only supported on AdaptationSet level

[tobbee]

@tysonite A reason for not supporting Representation-level SegmentTemplate is that one can do a lot of complex things like setting different parameters for different representations. It then becomes very complex to support things.

livesim2 wants all representations to be as similar is possible to be able to generate common numbers, timestamps etc. In your example, the startNumber is first set to 1 on the AdapationSet level and then changed to 0 on the representation level. I suppose it is set to 0 for all Representations, but in principle it could be different for different Representations, which would be sort of a mess. Allowing SegmentTemplate on Representation level opens up for such possibilities.

In general, VoD assets are not directly possible to use by livesim2, but needs some adaptation. For example, a last starter trailing segment should normally be dropped since it does not have the same duration as the others. That is a slight change of the MPD but you should also drop a file. Changing the SegmentTemplate to be on the AdaptationSetLevel is also a simple manual change of the MPD.

In case you want to make a PR to livesim2 to support this, please try to make the dynamic MPD generated by livesim2 only having SegmentTemplate on the AdaptationSet level by processing and removing any SegmentTemplate data on the Representation level. In that way, the logic of livesim2 segment pattern matching can be kept a bit less complex.

From my viewpoint, a more urgent feature than supporting all kinds of assets automatically is to give clear indications on what is not accepted. I think that the SegmentTemplate error message is rather clear in that case. However, what I think is lacking is a good guide of the tweaks you may need to do to make an MPD and its media streams working well with livesim2. I made a new issue for that: #205

[tysonite]

Fair enough, @tobbee , thank you. I will see where we end up.