search

gpac / ComplianceWarden

A pluggable compliance checker (ISOBMFF, HEIF/MIAF/AVIF, AV1 HDR10+)
https://gpac.github.io/ComplianceWarden-wasm/
Other
17 stars 7 forks source link

Check that hvcC matches `ispe` and `pixi` #58

Open leo-barnes opened 1 year ago

leo-barnes commented 1 year ago

I investigated the file mentioned in this comment and concluded that the issue is that the ispe does not match the actual dimensions of the coded item.

In this case the hvcC says that the dimensions of the coded item is 1600x1096, while the ispe says 1599x1096. This in turn leads to the clap property specifying a crop rect that starts at a negative offset.

It would be great if the compliance warden could do some minimal validation of the HEVC codec config with regards to the ispe. Other properties that could be checked would potentially be pixi, colr and potentially the MIAF brands that set limits on profiles.

leo-barnes commented 1 year ago

I realize this might be a large ask, but given that the compliance warden can handle the av1C, it would be a nice addition.

rbouqueau commented 1 year ago

We need two things:

  1. Parse de NALUs here
  2. Define a rule. Is it something you aim at doing normative (i.e. add to HEIF or MIAF, either as a SHALL or a SHOULD?)
leo-barnes commented 1 year ago

Define a rule. Is it something you aim at doing normative (i.e. add to HEIF or MIAF, either as a SHALL or a SHOULD?)

I would say that this is already a requirement. But potentially it's not spelled out as such clearly enough. @podborski and @cconcolato may know more details.

From HEIF:

6.3 Derivation of an output image of an image item

The reconstructed image of an image item is derived as follows:

  • if the image item contains a coded image, the coded image is decoded and the reconstructed image is the output of the > decoding process specified for the item type of the image item; ...

    6.5.3 Image spatial extents

    ... The ImageSpatialExtentsProperty documents the width and height of the associated image item. Every image item shall be associated with one property of this type, prior to the association of all transformative properties. ... image_width specifies the width of the reconstructed image in pixels, as specified in 6.3. image_height specifies the height of the reconstructed image in pixels, as specified in 6.3.

So we have:

  1. ispe shall be present
  2. ispe documents the dimensions of the reconstructed image
  3. If the image item is a coded item, the reconstructed image is the output of the decoding process specified for that type of item

I don't know my way around the HEVC spec well enough to immediately say which section describes the output dimensions, but it's most likely a combination of pic_width_in_luma_samples, pic_height_in_luma_samples and the conformance cropping window (if specified) in the VPS.

rbouqueau commented 1 year ago

Is anyone able to comment here?

cconcolato commented 1 year ago

I agree with @leo-barnes's analysis