Potential fragmented-only decoder contraint

[jaikrishnan]

Based on the AOM call and discussion on the Chrome bug for AVIF, there may be interest in allowing a constrained decoder to only support AVIF in fragmented MP4, as opposed to requiring unfragmented MP4 support.

This could be similar to the way the decoder constraint for reduced / intra-only tooling is handled. Not sure if a flag is required or if the file can simply be inspected.

[aklemets]

MIAF already defines brands for this. Since AVIF is defined on top of MIAF, the same brands could be used in an AVIF file. The “brands” are codec-independent (unlike MIAF “profiles”, which are tied to HEVC.) So, AVIF can use MIAF “brands” but not the “profiles”.

There are three relevant brands:

'MiCm' – This indicates that the image sequences use only fragmented tracks and that those tracks are compatible with CMAF (Common Media Application Format for Segmented Media, ISO/IEC 23000-19)

‘MiAn’ – This brand is for animations. There is exactly one image track or video track, and 0 or 1 alpha tracks, and 0 or 1 audio tracks.

'MiAC' – This brand indicates that the animation has an associated alpha plane track. The ‘MiAn’ and 'MiCm' brands are required to be specified. So, it means that that all the tracks are fragmented. As a further restriction, the animation track must be a video track (not a picture track).

Anders

---

From: Jai Krishnan notifications@github.com Sent: Wednesday, July 31, 2019 10:09:04 AM To: AOMediaCodec/av1-avif av1-avif@noreply.github.com Cc: Subscribed subscribed@noreply.github.com Subject: [AOMediaCodec/av1-avif] Potential fragmented-only decoder contraint (#64)

Based on the AOM call and discussion on the Chrome bug for AVIFhttps://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugs.chromium.org%2Fp%2Fchromium%2Fissues%2Fdetail%3Fid%3D960620%23c33&data=02%7C01%7C%7Cdb70d97395b3466e7dd008d715d9cac5%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637001897459578241&sdata=UdfOwNIbzHWN%2FysTngF2hk%2FQQudikii0VvaGAdDtd88%3D&reserved=0, there may be interest in allowing a constrained decoder to only support AVIF in fragmented MP4, as opposed to requiring unfragmented MP4 support.

This could be similar to the way the decoder constraint for reduced / intra-only tooling is handled. Not sure if a flag is required or if the file can simply be inspected.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FAOMediaCodec%2Fav1-avif%2Fissues%2F64%3Femail_source%3Dnotifications%26email_token%3DAGTLPAUXOYD3USSOAGAT3ETQCHBLBA5CNFSM4IIIY5PKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4HCT4PIA&data=02%7C01%7C%7Cdb70d97395b3466e7dd008d715d9cac5%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637001897459588240&sdata=VCzkZd2vXXPn84CWd10wRSTUfAXzIk7xfay%2BE6VJ7xo%3D&reserved=0, or mute the threadhttps://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAGTLPAWSHCSH3BNPAX47FGDQCHBLBANCNFSM4IIIY5PA&data=02%7C01%7C%7Cdb70d97395b3466e7dd008d715d9cac5%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637001897459608262&sdata=zyulXDix99SBXeLI9ScwYIOKv2Mk1LESOXRDNYwKZjw%3D&reserved=0.

[jaikrishnan]

So would a potential resolution would be to create one or more AVIF profiles which support a subset of these brands?

[aklemets]

I think these brands can already be used today in AVIF files. For example, an AVIF file can be tagged 'MiCm' to indicate that all the tracks are Fragmented.

However, I am not sure exactly what that solves – by the time the AVIF parser has inspected the brands, it has already started downloading the AVIF file, at which time it can easily determine if tracks are fragmented, or not, by parsing the rest of the file.

I think that to be really useful, the fact that all tracks are fragmented has to be stated up-front, before the client downloads the file. This could be done by specifying the brand as parameters to the MIME type (as is done in MPEG-DASH streaming scenarios), or by using a completely different MIME type (e.g., image/avif-fragmented-sequence).

Anders

From: Jai Krishnanmailto:notifications@github.com Sent: Wednesday, July 31, 2019 10:39 AM Subject: Re: [AOMediaCodec/av1-avif] Potential fragmented-only decoder contraint (#64)

So would a potential resolution would be to create one or more AVIF profiles which support a subset of these brands?

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FAOMediaCodec%2Fav1-avif%2Fissues%2F64%3Femail_source%3Dnotifications%26email_token%3DAGTLPAW43PE6SVKLHCQU4MLQCHE4PA5CNFSM4IIIY5PKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD3IAGGQ%23issuecomment-516948762&data=02%7C01%7C%7Ccdbac112ae764468b96308d715de0524%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637001915616365378&sdata=Hb4o5LIneXQ0BL6BCuJbhIFnDagAAKXCkv5XTYLQsNQ%3D&reserved=0, or mute the threadhttps://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAGTLPAS3QFV2DZWXKBUNKWTQCHE4PANCNFSM4IIIY5PA&data=02%7C01%7C%7Ccdbac112ae764468b96308d715de0524%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637001915616385406&sdata=Vz379iVQZiCsEeeC62cz4SYcXr6Kn0JWb6Mjh9lWxSs%3D&reserved=0.

[dalecurtis]

Yeah, not sure this is all that helpful. It would only be useful if we required all AVIF to be fragmented or not fragmented, instead of letting AVIF encompass the full scope of possibilities for an MP4 container :|

[jaikrishnan]

I'm ok closing this if no one else is interested in this distinction.

[dalecurtis]

Yeah, I think the ship has sailed on this unfortunately. Unless you want to change the AVIF spec to require fragmentation then I don't think there's any utility here.

@joedrago in case he has a different opinion.

[jaikrishnan]

closing based on discussion consensus that this feature would not be used