search

gpac / ComplianceWarden

A pluggable compliance checker (ISOBMFF, HEIF/MIAF/AVIF, AV1 HDR10+)
https://gpac.github.io/ComplianceWarden-wasm/
Other
16 stars 7 forks source link

Warning for AVIF file with no `colr` box and non-default CICP in Sequence Header OBU #36

Open leo-barnes opened 1 year ago

leo-barnes commented 1 year ago

We recently came across some AVIF files that rendered incorrectly on macOS/iOS. The reason turned out to be that the files were encoded as video-range, but had no colr box.

Per MIAF, an image item without colr box shall default to NCLX 1,13,5/6 full-range, no matter what is in the bitstream. So files like these should be rendered as if encoded as full range.

It would be great if the compliance warden could detect this and warn about it since the file creator does most likely not intend for that to happen. Both CICP and full/video-range mismatches would be good to detect.

rbouqueau commented 1 year ago

The container indeed takes precedence over the bitstream. However in this case you'd like a warning when parameters are inconsistent across layers.

I think the whole color management is actually error-prone and not well understood (as also show #37 and #38). Do you know if there exist a dump tool for such parameters in AVIF/MIAF?

We could add a fake rule in CW but we need to clearly identify what we want to check.

cconcolato commented 1 year ago

A warning would be good, associated to the following sentence in ISOBMFF:

If colour information is supplied in both this box, and also in the video bitstream, this box takes precedence, and over-rides the information in the bitstream.

It should check all the values from nclx: CP/TC/MC/Range.

leo-barnes commented 1 year ago

Yeah, I agree that the color management part is a bit messy. But there is now at least a way to fully specify it in such a way as to leave out any possibility of undefined behaviour: By having both an ICC profile and an NCLX box.

So if we detect that there is a mismatch between whatever ends up happening at the container level and the video bitstream, we should ideally warn about it and mention that two colr boxes could be used to remove the ambiguity.

rbouqueau commented 1 year ago

What do we do from here? Should we compare the color management information given at different layers and report inconsistencies?

leo-barnes commented 1 year ago

What do we do from here? Should we compare the color management information given at different layers and report inconsistencies?

I think that would make sense.

cconcolato commented 1 year ago

I think ComplianceWarden should not implement rules that do not correspond to "shall"s/"should"s in the spec. If there is a need for ComplianceWarden to emit a warning (I think the need is valid here), then we should modify the spec to add a "should". In this case, I would suggest modifying the sentence in ISOBMFF that says:

If colour information is supplied in both this box, and also in the video bitstream, this box takes precedence, and over-rides the information in the bitstream.

to:

The colour information supplied in both this box and in the video bitstream SHOULD match. If it is not the case, this box takes precedence, and over-rides the information in the bitstream.

But even then the above change in ISOBMFF uses "this", assuming that there is only one such box but in HEIF we may have more than one. So we may need to update HEIF as well, again ...

rbouqueau commented 1 year ago

Couldn't a side tool that analyzes color management inconsistencies help? It could not be cw.exe but a side executable - or may more pragmatically a mediainfo dump of the container versus the extracted bitstream.

Opinion?

leo-barnes commented 1 year ago

@rbouqueau

Couldn't a side tool that analyzes color management inconsistencies help?

It could, I just doubt people would actually run it.

Could we add a "color" section to the CW output that warns about these issues? Not necessarily errors, but warnings that the file is underspecified (or in the case of completely missing colr box and video range content) most likely will render differently in various implementations.

rbouqueau commented 1 year ago

CW could implement it if you write a normative document that list such cases. Would that make sense?

podborski commented 1 year ago

CW could implement it if you write a normative document that list such cases. Would that make sense?

I believe we could do it in AVIF spec using bikeshed asserts. Similar to how we are trying to do it in HDR10+ spec at the moment. https://github.com/AOMediaCodec/av1-hdr10plus/issues/26

rbouqueau commented 1 year ago

Is there any news here?

rbouqueau commented 1 year ago

Any news here? This also blocks #37.