AVIF Decode

[jaikrishnan]

Hello TAG!

I'm requesting a TAG review of AVIF Decode.

• Specification URL: https://aomediacodec.github.io/av1-avif/
• Primary contacts:
  • AOMedia for the specification
  • @wantehchang (Google) @jaikrishnan (Google) for the implementation
• Organization(s)/project(s) driving the specification: AOMedia
• Key pieces of existing multi-stakeholder review or discussion of this specification: https://github.com/AOMediaCodec/av1-avif/issues
• External status/issue trackers for this specification (publicly visible, e.g. Chrome Status):
  • crbug
  • bugzilla

Further details:

• [x] I have reviewed the TAG's API Design Principles
• Relevant time constraints or deadlines: N/A
• The group where the work on this specification is currently being done:
• The group where standardization of this work is intended to be done (if current group is a community group or other incubation venue): Alliance for Open Media
• Major unresolved issues with or opposition to this specification: N/A
• This work is being funded by: N/A

You should also know that...

N/A We'd prefer the TAG provide feedback as (please delete all but the desired option):

💬 leave review feedback as a comment in this issue and @-notify [github usernames]

[cynthia]

Hi @jaikrishnan!

It's pretty unclear what we are expected to do here; there is no explainer, nor from our side an API to review. From what we see it's the browser side implementation to allow decoding a new image format - we don't quite have a review process for these cases.

This does introduce a new image format onto the web platform, which means that any form of I/O that allows a still image (e.g. canvas, clipboard) would probably require some level plumbing to support it (e.g. registries, whitelists) once it becomes something beyond a blackbox codec that is only expected to be used as decoded internal state of an HTMLImageElement. (e.g. SkImage?)

[kenchris]

This seems somewhat related https://www.w3.org/TR/mse-byte-stream-format-isobmff/

Also maybe it would be nice explaining where these other standards used are used today. It seems to be used for MP4 files today. If such info and other background could be part of the intro in the spec that would be nice, if not, it definitely belongs as part of the explainer document.

In cases like "in the generic image file format [HEIF]" it would be nice to spell out what HEIF actually stands for like High Efficient Image Format - that might not be clear to the reader.

Does containers like ISO-BMFF and HEIF follow same patent policies etc as AV1 video format?

From a web perspective, I think it would be nice to know if there are any parts of the platform that needs to be patches to support AVIF property. Like you can convert an offscreen canvas to say a PNG and set certain settings while doing so: https://developer.mozilla.org/en-US/docs/Web/API/OffscreenCanvas/OffscreenCanvas.convertToBlob()

Those settings might be different for AVIF and I think that should all be considered while proposing adding AVIF to the web platform.

[kenchris]

Does AVIF support alpha channel?

[wantehchang]

On Mon, Apr 20, 2020 at 8:33 AM Kenneth Rohde Christiansen notifications@github.com wrote:

Does AVIF support alpha channel?

Yes, it does.

Wan-Teh

[kenchris]

Could you explain why the HEIF container was used in comparison with other containers (WebP uses RIFF). How does this choice affect the use on the web?

[cynthia]

So a bit of a blunt question based on what we discussed in today's call - this is based off of HEIF, which is not a royalty-free standard as we understand it. What was the technical background of the decision, and is there a path forward given the provenance of the core container format this is based off of?

[cynthia]

(Ugh, for some reason I was seeing a cached version without @kenchris's comment.)

[kenchris]

@wantehchang @jaikrishnan any update to our comments/questions?

[jaikrishnan]

Could you explain why the HEIF container was used in comparison with other containers (WebP uses RIFF). How does this choice affect the use on the web?

Benefits of the container design are primarily familiarity for developers, particularly those whose pipelines include uploads from mobile cameras.

So a bit of a blunt question based on what we discussed in today's call - this is based off of AVIF, which is not a royalty-free standard as we understand it. What was the technical background of the decision, and is there a path forward given the provenance of the core container format this is based off of?

Re: provenance, the AVIF image file format describes the use of an AV1 bitstream with a subset of the ISO-BMFF container, i.e. the MiAF constraints. Our view is the AVIF format would be available for use by open platforms like Chromium, in line with AOM's position on AV1 royalties.

[jaikrishnan]

It's pretty unclear what we are expected to do here; there is no explainer, nor from our side an API to review. From what we see it's the browser side implementation to allow decoding a new image format - we don't quite have a review process for these cases.

This is a good point, let me confirm with blink-owners if AVIF should go through a TAG Review, and then add an explainer and other design context that would be helpful

[cynthia]

@jaikrishnan Did you hear anything back? We'd be happy to review as long as it's clear what aspect we should be reviewing.

[jaikrishnan]

Thanks @cynthia, we looped back with blink-api-owners and confirmed that a Tag Review wouldn't be applicable for an image decoder implementation. I'll close this issue and note this in the intent to ship.