search

AOMediaCodec / av1-avif

AV1 Image File Format Specification - ISO-BMFF/HEIF derivative
https://aomediacodec.github.io/av1-avif/
BSD 2-Clause "Simplified" License
450 stars 40 forks source link

Does AVIF support multi-layer images to be stored in image sequences? #155

Open wantehchang opened 3 years ago

wantehchang commented 3 years ago

The third paragraph in Section 1 Scope of the AVIF spec says:

AV1 Image File Format also supports multi-layer images as specified in [AV1] to be stored both in image items and image sequences.

Does AVIF support multi-layer images to be stored in image sequences?

Section 2.3 describes AV1 Layered Image Items, but there is no corresponding section for AV1 Layered Image Sequences in the spec. Section 3 Image Sequences does not mention any relevant terms such as "multi-layer", "layered", or "operating point".

In HEIF (ISO/IEC 23008-12:2017), the term "layer" seems to be used with image items only. I am wondering if the third paragraph in Section 1 should be changed to:

AV1 Image File Format also supports multi-layer images as specified in [AV1] to be stored in image items.

wantehchang commented 3 years ago

This sentence is in v1.0.0 and is not new. However, since we dealt with layered images in the new version of the spec, it would be good to resolve this issue -- either "Won't Fix", or remove "image sequences" from this sentence.

cconcolato commented 3 years ago

@wantehchang As you noted, the sentence was in v1.0.0 as an indication that layered images could be stored (in items and sequences) omitting the fact that no specific tool was specified to enable efficient usage.

In the new version, we only defined tools for optimizing the use of layered images in items. In my view, this means that layered images can be used in sequences but no specific tools are defined. The intent was that if one had encoded a video with layered images and wanted to change that into an image sequence, it should be possible.

Of course, if needed, we could add specific tools (e.g. sample groups) to help processing of layered image sequences, but we should have clear use cases.

Hukuma1 commented 2 years ago

Apologies if this is in the wrong place, but was trying to Google for an answer.

AVIF is mentioned to support overlays (layers?) everywhere, but I have no idea how to use it. I would like to separate an image and text on top of it as two separate layers. This would help keep the text cleaner, am I understanding that right?

I would then love to take it a step further and convert a video to an animated sequence, while also having this text be separated. Is this ability part of the newer spec coming out? Right now AVIF blows WEBP out of the water for efficiency for the same animated sequence (to replace GIF usage). If I could further optimize the text I have by putting it onto a separate layer, then I imagine it would be even better.

leo-barnes commented 2 years ago

AVIF is mentioned to support overlays (layers?) everywhere, but I have no idea how to use it. I would like to separate an image and text on top of it as two separate layers. This would help keep the text cleaner, am I understanding that right?

I think you may be conflating two things here. (Which is understandable given the similar naming.)

Overlays Overlays are a container level feature. It's defined in HEIF/MIAF (which is what AVIF is based on). It basically allows you to specify a canvas of a specific size and then allows compositing images into this canvas with blending. This can definitely be used to support the use-case you had in mind for still images. I know overlay tracks have been proposed in MPEG, but I'm not sure if it's made it's way into a published spec yet.

I'm not sure if libavif has added support for overlays. @wantehchang or @baumanj do you know if this is the case?

Multi-layer streams The AV1 codec supports multi-layer streams. This is typically used to have a lower quality/resolution base layer and a higher quality/resolution refinement layer. This can be used to do progressive decoding for example. The base layer is typically used for prediction when encoding the higher layers, so this doesn't really work for the use-case you have in mind since you can't really do blending AFAIK.

Hukuma1 commented 2 years ago

I see, thanks so much for clarifying! Much appreciated.

rubyFeedback commented 7 months ago

Having support for that would be great. Right now I kind of convert .avif into .jpg and then into a .pdf file, just to have a single file, which isn't that awesome. Having a single .avif that holds multiple "sub"-images would be much better. Evidently I may lack some knowledge how to do this right now though.