search

AOMediaCodec / av1-avif

AV1 Image File Format Specification - ISO-BMFF/HEIF derivative
https://aomediacodec.github.io/av1-avif/
BSD 2-Clause "Simplified" License
451 stars 40 forks source link

Support for derivation chains #124

Closed cconcolato closed 3 years ago

cconcolato commented 3 years ago

AVIF inherits the notion of image derivation (transformation, grids, overlays) from HEIF and the constraints on so-called "derivation chain" from MIAF (see section 7.3.11.1) which says:

A MIAF file may include any derived image defined in ISO/IEC 23008-12 with the constraints defined in the
following clauses.
A MIAF player shall process all derived images of the types identified in 7.3.7.
The maximum derivation chain is shown in the following list; the Identity, Grid and Overlay derivations are
optional. All derivation chains shall originate from one or more coded images. An identity derivation shall
not be derived immediately from another identity derivation. If derivations occur, they shall be in this
order:
• Mandatory Coded Image(s)
• Optional Identity derivation (7.3.11.2)
• Optional Grid (7.3.11.4)
• Optional Identity derivation (7.3.11.2)
• Optional Overlay (7.3.11.3)
• Optional Identity derivation (7.3.11.2)

This means you cannot create a grid of grids but you can create a grid of rotated images or a grid of cropped images. I created such image and it is currently not supported in libavif. Should we support it or should we restrict AVIF to only support transformation (rotation, mirroring, cropping) on the final image?

Attached image (zipped because GitHub does not support AVIF yet. Do we know how to request support for it?) clap_grid_2x2_2048x858.avif.zip

FrankGalligan commented 3 years ago

This is a good question. Does anyone have any use cases for applying transformations on anything other than the final image?

leo-barnes commented 3 years ago

There are mainly two use cases that we identified back when we added the MIAF restrictions:

  1. HW restrictions may mean that your encoder will add padding to images. If you want to create a grid of padded images, you need to support crops on the individual tiles. I'm not sure how realistic this use case is, but it's at least something that could happen.
  2. It's slightly more plausible to happen with overlays since they are a bit more artistic in nature. If you want create a collage in the form of an overlay it's very likely that you will want to crop the inputs to it. If you also want to display the inputs as full images you will need to make the inputs to the overlay cropped 'iden' items.

If these are not things we see as very likely to be useful we can of course restrict it a bit more.

leo-barnes commented 3 years ago

The conclusion of the last meeting was to suggest constraining AVIF to only allow transformations on the final image, but leave this issue open for a while and seek feedback from external users.