AVIF should have preamble explaining the relationship with referenced specs

[leo-barnes]

Per meeting, it would probably be beneficial if the AVIF spec had a short preamble that explained roughly what is contained in the various specs that are referenced.

At the top of my head:

• ISO-BMFF: Contains the basic box structure used for both HEIF and MP4 as well as some common boxes used by both
• AV1: The video/image codec used by AVIF
• AV1-ISO-BMFF: Outlines how AV1 is embedded in MP4/HEIF
• HEIF: Contains most of the boxes used for still images
• MIAF: Adds restrictions on HEIF to make interoperability of still images easier
• CMAF: Adds restrictions on ISO-BMFF to make interoperability of video easier

[baumanj]

I can only think of two more off the top of my head:

1. ICC: Specifies the rICC and prof variants of the colr box
2. ITU-T H.273: Specifies the nclx variant of the colr box

Should this also be the issue for describing all the places that the reference implementation deviates from the published versions of the specifications and how? Or should I create a separate issue for that?

[wantehchang]

While working on libavif and Chrome's AVIFImageDecoder class, I have never needed to consult CMAF. So I suggest we take CMAF off the list.

Re: ICC and ITU-T H.273: AVIF treats ICC profiles as opaque data, so we don't need to consider ICC as a referenced spec. H.273 is only indirectly referenced (by MIAF at least). Probably best to omit H.273 to keep the preamble short.

Note: I do consult H.273 regularly, but it is mainly for the YUV-to-RGB conversion.

[cconcolato]

https://github.com/AOMediaCodec/av1-avif/pull/170 (still a draft) may help.

@wantehchang CMAF is in the picture when we talk about image sequences. There are essentially 2 types of storage of tracks (video, audio, image sequence, ...): non-fragmented and fragmented. CMAF puts restrictions on how to store fragmented tracks. We could mandate that image sequences never fragmented. That would remove CMAF from the picture. I could imagine edge use cases where that could be a problem.

[cconcolato]

I think we could be more precise and highlight different requirements depending on the 2 main use cases: image items and image sequences.

• ISOBMFF: This is a large specification and it is not necessary to implement it all (by far) to implement AVIF. You need at least the top-level boxes ftyp and mdat and then additional boxes that depend on the use case:
  • for image items, only the meta box and its hierarchy is needed, up to a certain edition of ISOBMFF
  • for image sequences, the moov box (and its hierarchy) is also needed for non-fragmented files, while additional boxes (moof and its hierarchy is needed) are also needed for fragmented files; again up to a certain edition of ISOBMFF.
• HEIF: we should highlight that not all the tools in HEIF need to be implemented, and again only in a specific edition of HEIF (e.g. no need to support predictively coded image items)