Loosen restrictions on SIDX usage

[mstattma]

Currently SIDX capabilities in CMAF are constrained to a single SIDX box which can only index fragments (and not CMAF chunks).

For some use cases it would be desirable to use daisy chaining of SIDX boxes, as well as allow referencing chunks.

For maximum player compatibility it's likely advisable to require daisy chaining when chunks which don't start with a RAP should be referenced, rather than loosening the requirement that a single SIDX can only reference fragments.

[cconcolato]

Can you clarify the use cases you're referring to? Is it for static/VoD files (e.g. to allow seeking to a chunk) or is it for live streaming?

[mstattma]

The use case which made me discover the limitation is around VoD, to enable direct byte range addressing of iframes to improve trick play performance without a dedicated trick play rendition.

I think for live SIDX comes with too many issues.

[krasimirkolarov]

Can you please suggest a concrete proposal on how to address this in CMAF? We discussed that at MPEG 142 but are not sure how to proceed.

[mstattma]

The goal is to emulate an iframe-only file.

The basic idea and file structure would be to package all iframes, and all frames between iframes (or the remainder of the GOP), into their own chunks, and use the SAP flags on SIDX to signal to the player which chunks have iframes. In a typical file this would mean that an average segment is split into two chunks, the iframe chunk (carrying one frame only) and then the rest of the segment (leaving aside that there may be multiple iframes in a segment).

As current players assume that a referenced segment (or here chunk) with a SAP isn't just one frame long, the proposal is to add a second "top level" SIDX which uses daisy chaining of at least two additional SIDX (one for all the SAP chunks, and one for all the other chunks) to provide the new byte ranges.

[haudiobe]

Waiting for input, we need to see if we could easily address it.

[krasimirkolarov]

It is not clear if this has a wide support. We want to encourage more feedback before resolution.

[cconcolato]

We will close this issue unless we receive feedback before the next MPEG meeting (January 2024)

[ZmGorynych]

I think I understand the point, but I'm unsure whether two separate sidx boxes are needed. If we need an I-frame like "alternative" sidx, we can have an external file, this already exists. An alternative would be to use ssix along with sidx

[mstattma]

I like the idea using ssix, it seems to do the trick, at least based on quickly checking the spec. However, it doesn't look like ssix is allowed in CMAF at all. Or do I over interpret the track file structure constraints?

Regarding the other proposal and also the question of @krasimirkolarov: the use case came up in the context of standardization in the APEX Association where we specified the next gen media format for inflight entertainment. A version of the proposed solution is actually in the latest spec draft of APEX 0415. The main selling point is to have the most basic enablement and common denominator for trick play in a self contained track file, without requiring higher level specs like DASH or HLS. In that market there is a great desire for reuse of encoded content across systems and airlines, so we were looking for something which could be "built-in" on content level - while remaining CMAF compliant.

Currently a lot of TS based content is used which is encoded with short GOPs to enable the same in a very legacy way. We tried to improve on that.

[podborski]

Do we want to close this issue or are we expecting some concrete proposal to the CMAF group?

[mstattma]

The concrete proposal is to either loosen the restriction mentioned above, or allow 'ssix' as proposed above.

Is anything more concrete needed than this? Like a redline or similar?

[jpiesing]

How does this fit with the CMAF profiles / brands? Loosening restrictions in an existing profile / brand is not backwards compatible. Is there going to be a looser profile that it could be added in?

[mstattma]

Maybe my original proposal of daisy chaining 'sidx' would pose issues when allowed in existing profiles. But allowing an optional 'ssix' shouldn't break anything I think.

[jpiesing]

Maybe my original proposal of daisy chaining 'sidx' would pose issues when allowed in existing profiles. But allowing an optional 'ssix' shouldn't break anything I think.

If it was made optional for content in one of the existing profiles then a file reader / decoder may find a box which it's not been tested against. That's not backwards compatible.

[mstattma]

a file reader / decoder may find a box which it's not been tested against. That's not backwards compatible.

Well, 14496-12 is pretty clear about this: Boxes with an unrecognized type shall be ignored and skipped.

I didn't check if CMAF defines a different behavior, but 14496-12 also says in the informative Annex B (i.e. Guidance on deriving from it): Be as permissive as possible with respect to the presence of other information in the file; indicate that unrecognized boxes and media may be ignored (not “should be ignored”).

[mstattma]

Btw, unless CMAF doesn't contradict that guidance elsewhere (does someone know for sure?), then when looking at 6.6.3, and specifically 7.3.3. bullet b) Additional boxes, such as SegmentIndexBox(es), may be present between the CMAF header and the first CMAF fragment., the language IMHO suggests that it's already allowed to have additional information (not specified in CMAF) like a 'ssix' box following the 'sidx'.

WDYT? It would solve this without need for any change.

However, for cosmetics, what I noticed when checking now, is that the spec isn't 100% clear about whether it's really limited to a single 'sidx' box as the text uses plural form at various places like the one copied above. Only Table 10 says clearly 0/1 for cardinality. Furthermore is the language a bit unclear when it comes to what can be referenced in the 'sidx' box. 7.3.3.3 first talks about CMAF media objects (which includes Chunks) but then in bullet c) it says each subsegment referenced in the SegmentIndexBox shall be a single CMAF fragment contained in the CMAF track file.

So maybe this is an opportunity to clarify those things while also making clear that additional boxes not mentioned in CMAF may be used?

[krasimirkolarov]

From CMAF BoG at MPEG146: We will add a text to the WD of Amd. 2 of Ed. 3 to include .ssix box - Cyril will provide text. For all other requests for clarifications we welcome contributions with concrete text to be discussed.