Potentially make 'pixi' optional iff it's [8,8,8]/[8]

[leo-barnes]

There was a brief period during which libheif was creating AVIF (which is derived from MIAF) files that were missing the 'pixi' property. This has now been fixed, but there are now some files in circulation that are technically invalid files due to MIAF requiring 'pixi' to be present.

While discussing this and other issues, we have heard feedback that HEIF/MIAF has a bit too much header size overhead, so one change that could mitigate both issues is to change MIAF from always strictly requiring 'pixi' to only requiring it if the content differs from a defined default.

If we make [8,8,8] the default for non-auxiliary images (main and thumbnail image items) and [8] the default for auxiliary image items, we can most likely allow content creators to reduce header overhead and make most existing 'pixi'-less images valid.

[dwsinger]

for properties that are optional in HEIF, like pixi, we should be clearer what the default is anyway (in this case, 'ask the codec')

[njdoyle]

At Akamai, we have encoded many AVIF images with the affected version of libheif. It isn't necessarily easy, practical, or possible to replace or reencode all of these long lived images. Making pixi optional would definitely help our situation in the sense that browsers want to follow the AVIF specification and also don't want to introduce breakage where the specification as it exists today hasn't been followed. Aside from avoiding this breakage, I agree that the overhead of HEIF/MIAF is substantial and welcome any changes to reduce this overhead with practical defaults.

[baumanj]

Considering the statement in ISOBMFF (14496-12:2020) § 12.1.5.1:

If colour information is supplied in both this box, and also in the video bitstream, this box takes precedence, and over-rides the information in the bitstream.

Should there be something similar here?

1. What should happen if the pixi box(es) is/are omitted and the data in the bitstream doesn't match the default?
2. What should happen if [an] explicit pixi box(es) is/are present and the data in the pixi box(es) contradicts the bitstream?

The options that occur to me are:

1. Reject the input as invalid
2. Use the value in the bitstream

Unlike with colr, I don't think it's feasible to "use" the pixi values if that doesn't match the data in the bitstream.

[leo-barnes]

Unlike with colr, I don't think it's feasible to "use" the pixi values if that doesn't match the data in the bitstream.

I agree. The whole point of mandating pixi was so that you wouldn't need to parse the bitstream to figure out the bitdepth. So I would vote for option 1.

[dwsinger]

Yes. One could argue that this enables one to fix a stream that's mislabeled internally, but I think the right way to fix it is to fix the internals. Mislabeled color does sometimes happen, and it can help to be able to fix it externally. Mislabeled pixel info seems unlikely.

[leo-barnes]

Accepted into MIAF 2nd edition.

We will add a note warning implementers that there may be files in existence lacking the pixi that don't actually conform to the default values.