Open haudiobe opened 1 year ago
AHG 2023/03/03
Different options discussed:
1) Add a SegmentTimeline extensions that changes entirely the addressing scheme. Can be used with $Time$. Needs MPEG involvement. Has player impact
2) Use $Number$ for audio and provide auxiliary information on the exact segment duration using a new element. This new element may be combined with @duration
or with SegmentTimeline
. This can be done quickly in DASH-IF, would not even have a player impact.
3) Do nothing but document the best practices
Other options? Please comment
From my own notes of the discussion, with listing of some pros/cons
I think at a high level, if a content producer is using explicit addressing mode, there are performance advantages to providing a mechanism for reducing the manifest size (primarily on the initial request, as patch syntax allows for reduced update sizes).
AHG Call 2023/03/24
Proposal
@koceskik @haudiobe check on this.
I'll write up a revised proposal for a more backwards-compatible PatternTimeline
syntax (mainly the same as indicated, but simply not embedding in the existing SegmentTimeline
).
Of note, I was reviewing documentation and implementations. In particular, one case for having an explicit addressing mode (ie an explicit timeline) is when an Ad Insertion MPD Manipulator proxies directly from an IF-2 or IF-3 input. (see https://dashif.org/docs/CR-Ad-Insertion-r7.pdf section 8.1.3 Architectures for guidance on this scenario)
In particular, there exist cases where ad insertion cannot fill an ad break, and must return to main content, or otherwise adjust the Period@start and prescribed segments in a Period. With simple addressing mode, adjusting Period@start is limited by the DASH-IF timing constraints: https://dashif-documents.azurewebsites.net/Guidelines-TimingModel/master/Guidelines-TimingModel.html#addressing-simple-startpoint
In particular:
The rest of this chapter assumes that the nominal timing of media segments matches the real timing. If you cannot satisfy this constraint but still wish to move the period start point, convert to explicit addressing. See § 18.4.3 Converting simple addressing to explicit addressing.
As such, in an ad-insertion architecture, simple-addressing mode either:
With a PatternTimeline
, even if an end DASH Client does not consume the PatternTimeline (and instead consumes the simple-addressing based on SegmentTemplate@duration), an ad insertion MPD manipulator proxy could rely on PatternTimeline
to adjust Periods accurately.
I was also thinking about dynamic ad-insertion scenario for which I thought it will be very tricky to manage the matching between the segment numbers and the PatternTimeline.
Also, consider the case you currently deliver manifests with SegmentTimeline, primarily to get around issues introduced by simple-addressing mode, especially the ones Kyle listed in previous comment. If you want to reduce the manifest size using this PatternTimeline, you will have to rely on numbers for legacy players, and then introduce issues (seeking, gaps, dynamic ad-insertion, splicing) for these players you would not encounter when using SegmentTimeline.
In short, you would resolve the manifest size issue for up to date players compatible with the PatternTimeline, while introducing some other issues for legacy players who will have to use the numbers.
Accordingly, if one wants to reduce size of manifests using SegmentTimeline, there may be no alternative way than introducing this new Pattern syntax under the existing SegmentTimeline and breaking the backward compatibility with legacy players.
Notes from meeting today:
Potential options:
<PatternTimeline>
<Pattern t="0" r="1" />
<P d="95232"/>
<P d="96256" r="2"/>
</Pattern>
<S t="768000" d=“44307"/>
</PatternTimeline>
In conjunction:
SegmentTimeline
MAY be absent completely, in which case a legacy DASH client will fall back to simple-addressing mode via SegmentTemplate@duration
using average segment timelines
PatternTimeline
as a means of enforcing accurate Period@start manipulationSegmentTimeline
repetition (with signaling of accuracy), and require $Number$
-based referencing if using an inaccurate timeline:
<SegmentTimeline accurate="false">
<S t="0" d="96000" r="7"/> <!-- This is an inaccurate, averaged expansion of the pattern -->
<S t="768000" d=“44307"/> <!-- Using a SegmentTimeline, the explicit cut-short segment may be explicitly signaled -->
</SegmentTimeline>
, whereby "loosen" would be restricted to being off by a particular threshold (I'm not sure by how much, however).
For a DASH client which relies on time-accuracy, the PatternTimeline
should be read and interpreted. For a DASH client which relies on SegmentTimeline
today, but is not restricted by inaccuracies in the SegmentTimeline
, the SegmentTimeline
with $Number$
should be backwards-compatible.
@bbert Is your assertion that using a PatternTimeline
would necessitate $Number$
addressing? And in doing so, it may result in incompatibility with non-updating DASH clients?
@koceskik yes my understanding of @haudiobe's initial proposal is that if you want to signal a PatternTimeline
to reduce the manifest you would necessitate the $Number$
addressing to be compatible with legacy DASH clients. Thus you loose time accuracy for these legacy clients, and indeed it may result in incompatibility issues especially with ad-insertion manifest manipulators.
Now thinking back on last proposal, with this example:
<SegmentTimeline accurate="false">
<S t="178577070976" d="96000" r="7"/>
</SegmentTimeline>
<SegmentTimelineNew>
<Pattern t="178577070976" r="1">
<S d="95232"/>
<S d="96256" r="2"/>
</Pattern>
<S t="178577838976" d="95232"/>
</SegmentTimelineNew>
and for which the segment starting for example at timestamp 178577550208
would be addressed at time 178577550976
.
Is that right?
Finally the result is the same except that, compared to previous solution, you can signal for example gaps in both timelines since you are not forced to address segments using $Number$
.
However that solution still requires some tricky processing for clients that read and interpret that new pattern timeline since you need to keep matching between the segments from both timelines, especially for content replacement. That will be source of errors and interoperability issues.
Questions:
SegmentTimeline
?SegmentTimelineNew
of a manifest that has been meantime manipulated by a legacy manipulator but that is not aware of this new element? AHG 2023/04/14
@agiladi suggests to also indicate the tolerance - a concrete proposal would be welcome.
@agiladi mentioned milliseconds, but the most natural tolerance would be to use the same units as the rest of the timestamps. Maybe his suggestion would be different, but I think the following example would make sense:
For 48kHz audio with timescale 48000, an AAC frame's duration is 1024 ticks. It is therefore possible to always achieve a duration of an audio segment that is within 1024 ticks from the average. A bigger variation would require a new <S>
element. A possible syntax should look like:
<SegmentTimeline tolerance="1024">
rather than
<SegmentTimeline accurate="false">
[edited for @tobbee 's comment above] Instead of
<SegmentTimeline accurate="false">
<S t="178577070976" d="96000" r="7"/>
</SegmentTimeline>
we can explicitly specify tolerance
<SegmentTimeline tolerance="1024">
<S t="178577070976" d="96000" r="7"/>
</SegmentTimeline>
The semantics will be that for any segment the actual value in tfdt
will be within 1024 clock ticks (in units of timescale) from the value calculated using S@t
and S@d
. A possibly better syntax may be introducing SegmentTemplate.tolerance
in which case this applies to cases where we do not use SegmentTimeline at all (i.e., we rely on the @duration
value)
Assuming audio frame duration of 1024, this lets us do a pattern where we add an extra audio frame to every Nth segment to maintain a/v segment alignment.
@agiladi mentioned milliseconds, but the most natural tolerance would be to use the same units as the rest of the timestamps. Maybe his suggestion would be different, but I think the following example would make sense:
For 48kHz audio with timescale 48000, an AAC frame's duration is 1024 ticks. It is therefore possible to always achieve a duration of an audio segment that is within 1024 ticks from the average. A bigger variation would require a new
<S>
element. A possible syntax should look like:<SegmentTimeline tolerance="1024">
rather than
<SegmentTimeline accurate="false">
Completely agree with the above, it should be in units of the same timescale as used in SegmentTemplate
. Updated previous comment to account for this
It should not be tfdt, but presentation time.
I'm of the opinion that adding @accurate
or @tolerance
to the SegmentTimeline
and then asserting that the segments within aren't accurately reflections of the segment timings is actually not backwards-compatible. If a DASH client needs to understand these new attributes to behave correctly, they'd need to update, and if the DASH client already handles simple addressing mode, why add a SegmentTimeline at all? If it's purely to add PatternSegmentTimeline
for accurate timings (such as for a manifest manipulator), I still don't see the value in adding complexity of a new syntax to the existing SegmentTimeline
.
At which point, why not go with the original proposal of adding a Pattern
element within the SegmentTimeline
and explicitly not being backwards-compatible.
In general, I think it could make sense to include PatternSegmentTimeline
as a new child element of SegmentTemplate
, which would produce backwards-compatible manifests where simple addressing mode is used and supported by the DASH client. A service offering could, theoretically, support accurate manifest manipulations by updating a manifest manipulation service to support explicit pattern addressing, even if an on-device DASH client couldn't be updated (by falling back to simple-addressing mode since it would ignore the new PatternSegmentTimeline
tag completely).
However, if explicit addressing mode is necessary, either because the DASH client can't handle the variability in fragment timings correctly, or the manifest manipulator (ex: for ads insertion) needs accurate timings for handling period bounds manipulation, you're going to need to update your DASH client/manifest manipulator anyways.
So, I'm unconvinced that the backwards-compatibility argument is a major point in making a decision, because service offerings that will rely on explicit addressing mode will need to update anyways.
At that point, does it make sense to add complexity of a new tag (PatternSegmentTimeline
) and the ruleset for deciding 1) that if PatternSegmentTimeline
exists, it is an accurate patternized timeline, but SegmentTemplate@duration
is still required to backwards-ly support cases where PatternSegmentTimeline
isn't understood, 2) if PatternSegmentTimeline
doesn't exist, but SegmentTimeline
does, then SegmentTimeline
is an accurate non-patternized timeline, 3) and that if they both don't exist, it's simple-addressing mode.
(it doesn't sound too complex in that simplified paragraph, but I think there's a few other cases to author explicit clarity on)
Live TF 2023/05/05
@koceskik @bbert @ZmGorynych @technogeek00 any updates on the google docs that we can check tomorrow?
I've created a doc here:
https://docs.google.com/document/d/1O4diz48Lr3LJozloy2MdLO79_Ylj6yjg5f-Jhi0UjPQ
f2f June
Decision:
Did a first pass read of the document, very nicely done @koceskik clearly articulated. I'll think on some descriptive aspects, should we provide comments directly back in the doc?
Did a first pass read of the document, very nicely done @koceskik clearly articulated. I'll think on some descriptive aspects, should we provide comments directly back in the doc?
Yes, please comment directly in the doc. Until next week, in order to complete submission to MPEG.
Kyle from AWS presented this now in the IOP WG call. New topics • Kyle presents: IOP23011.PatternTemplateManifest_DASH-IF_Reintroduction_2023.pptx o We will add this to the DASH-IF Live TF on Friday, March 3, 2023 We will discuss this in more details in the Live TF on Friday.