HEIF - Validator should reject files that have essential bit set on descriptive properties other than those permitted

[cconcolato]

There are essentially 3 classes of properties regarding the essential bit:

• Descriptive properties that are essential according to the spec (e.g. codec config like av1C, ...). They must have the bit set to 1.
• Other descriptive properties. They must have have the essential bit set to 0.
• Transformative properties. These may or may not have the essential bit set.

[rbouqueau]

No problem to add it. Where is this specified?

[cconcolato]

The following indicates that transformative properties can be essential or not:

NOTE When an image item has non-essential transformative item properties, the pixel values of the output image might depend on the reader's capability and/or choice of applying the non-essential transformative item properties.

The following indicates covers most descriptive properties:

Descriptive properties are non-essential, unless stated otherwise in their specification.

and this covers the rest: decoder configurations:

essential shall be equal to 1 for an 'hvcC' item property associated with an image item of type 'hvc1'. essential shall be equal to 1 for an 'lhvC' item property associated with an image item of type 'lhv1'. essential shall be equal to 1 for an 'avcC' item property associated with an image item of type 'avc1'. essential shall be equal to 1 for an 'jpgC' item property associated with an image item of type 'jpeg'.

and other descriptive properties:

essential shall be equal to 1 for an 'lsel' item property. essential shall be equal to 1 for an 'tols' item property.

[rbouqueau]

So I committed a check about the essential bit with the listed descriptive properties. However I'm unsure about a normative expectations for others to be at zero when I read this:

Descriptive properties are non-essential, unless stated otherwise in their specification.

[cconcolato]

Yes, you have to go through the spect text one by one to see if they are allowed to be essential. That's what I did above. So far, only the 5 above are essential.

[rbouqueau]

Yes that's what https://github.com/gpac/ComplianceWarden/commit/c253623e72bc89e85a5e5177ec3c72d6b367a0e4 does. So my perception is that we can close this issue.

[cconcolato]

I'm not sure.

You seem to be checking that clap, irot and imir are essential here: https://github.com/gpac/ComplianceWarden/commit/c253623e72bc89e85a5e5177ec3c72d6b367a0e4#diff-c13f959cf856284d748adbeba70b1365R1195 They may be essential, that's an author's choice. So you should not fail if they are not.

Also, can you confirm that the following file now fails (because of pixi wrongly marked essential): https://github.com/AOMediaCodec/av1-avif/blob/master/testFiles/Link-U/kimono.avif

[rbouqueau]

For clap, irot and imir, this is for MIAF (from the code):

"Section 7.3.9\n" "All transformative properties associated with coded and derived images required\n" "or conditionally required by this document shall be marked as essential, and\n" "shall be from the set that are permitted by this document or the applicable\n" "profile. No other essential transformative property shall be associated with\n" "such images.",

Also, can you confirm that the following file now fails (because of pixi wrongly marked essential): https://github.com/AOMediaCodec/av1-avif/blob/master/testFiles/Link-U/kimono.avif

No, cf ComplianceWarden output below.

HEIF:

rbouqueau@rose:~/works/ComplianceWarden$ bin/cw.exe heif kimono.avif 
No errors.

For MIAF:

rbouqueau@rose:~/works/ComplianceWarden$ bin/cw.exe miaf kimono.avif 
[Rule #38] File extension for "kimono.avif" doesn't match: expecting one of 'heif hif ', got 'avif'
1 error(s).

Specification description: MIAF (Multi-Image Application Format)
MPEG-A part 22 - ISO/IEC 23000-22 - w18260 FDIS - Jan 2019

Error rules description:

[Rule #38] Section 10.1
A MIAF file shall use the filename extensions specified by HEIF to identify the
presence of specific image coding formats

"Descriptive properties are non-essential, unless stated otherwise in their specification."

This part is problematic for two reasons:

• It is not a normative statement, it contains no 'shall'. I don't know what the editor meant but it could be a remark for a Reader implementor. Clarification is welcome.
• "unless stated otherwise in their specification" may create false positives, unless we explicitly list the descriptive properties expected to be at zero, which imho should be specified in the spec itself. Allowing MIAF and non-MIAF content within the same ISOBMFF content creates quite some complications on the conformance side.

Opinion?

[rbouqueau]

I'm trying to close open issues. I understand the current status is not prefect but I don't understand what could be improved.

First the following statement doesn't contain a "shall" statement: Descriptive properties are non-essential, unless stated otherwise in their specification. Secondly, if we considered it as a "shall" statement, it would break the separation of concerns as MIAF and now AVIF requires the essential bit on other boxes: HEIF could return that "av1C" shall be non essential while AVIF states the contrary.

As a consequence I think that the current status is a good compromise.

[cconcolato]

Can you recap the current status?

[rbouqueau]

Check for bit essential at 1:

• HEIF: lsel, hvcC, lhvC, tols, avcC, jpgC
• MIAF: clap, irot, imir
• AVIF: av1C

[cconcolato]

Note regarding AVIF: https://github.com/AOMediaCodec/av1-avif/pull/144

[rbouqueau]

This has not been updated to avoid mixing versions of the spec. Is there a will from AOM to keep aligned on the latest version of the spec?

[rbouqueau]

Closing. Please re-open if there is news to share here.