Predictive images and unknown essential properties

[cconcolato]

HEIF 2nd Edition AMD 1 specifies the ability to store predicted images (inter frames) as items. The item type of such image is the same as for intra images. The way to detect that an image is predicted is by search for an essential property of type pred.

We should determine if we want to have support for AV1 inter frames or not. If so, should we wait for a possible MIAF update?

We should raise awareness of implementers that they should not process image items containing essential properties that they don't understand.

[leo-barnes]

For context, the goal of the new property was to support the following use-case:

1. You have a HEIF file that contains an image sequence (think Live Photo/Living Image)
2. The primary item of the HEIF file points to one of the I-frames in the sequence
3. You want to change the primary item to point to one of the predicted frames in the sequence

Without this property you would have to decode the predicted frame, re-encode it as a non-predicted image item and make that the primary item. With this property you're now able to create an image item out of the predicted frame in the sequence without having to do any re-encoding.

[cconcolato]

Note that the actual property which is required and marked as essential for predictively coded images is actually called rref, with the following syntax:

aligned(8) class RequiredReferenceTypesProperty 
    extends ItemFullProperty('rrtp''rref', version = 0, flags = 0){
    unsigned int(8) reference_type_count;
    for (i=0; i< reference_type_count; i++) {
        unsigned int(32) reference_type[i];
    }
}

pred is both a brand and a reference type to be set between the predictive coded image and the images that need to be decoded before.

[cconcolato]

Predictive images would not be permitted as primary item until MIAF allows it. Could be allowed as additional item, but would be ignored by old decoders that don't interpret the property (marked as essential).

[cconcolato]

Closing the issue for now. We can reopen when the need for predictive image items (as primary item) arises or if MIAF is updated.