`main-still` profile?

[indolering]

Forking discussion of my off-topic suggestions from #128.

WebP and friends are apparently not decoded in hardware in most browsers. However, Apple's use of HEIC is constrained in ways to make it more friendly to hardware decoding, such as fixing tile sizes 256px so that hardware doesn't need to be re-initialized.

This would seem to make the case for a similarly constrained profile (main-still?) to remove the initialization overhead (or at least amortize it across a large number of images) when decoding images via a GPGPU or fixed-function hardware. The primary target of this profile would be mobile: where bandwidth and battery life are at a premium and images are small enough that 8 or 10-bit color banding, 4:2:0 pixel offsets, and small tile sizes won't be noticeable.

Note that I have ZERO implementation experience and I am basing this off of other people's observations about Apple's use of HEIC. If this is anti-helpful, feel free to close it. I just thought it was worth mentioning.

[indolering]

If we add a 12-bit profile with level constraints, does that give clients "false" hope that they will get HW acceleration if they stay within the profile? ... Why do we need a profile for this? It's already supported by the file format itself.

So my understanding is that allowing variable tile sizes means that the hardware decoders have to be re-initialized for each image. This ultra-constrained profile essentially fixes the tile size and whatever else is necessary to prevent re-initialization so that implementers can just warm up the hardware once and reuse it multiple times.

Having a dedicated profile is essentially cat herding, CDNs are more likely to use it for mobile clients if there is a higher likelihood of support for hardware decoding in browsers (for example). Whether that hope if true or false depends on whether the profile makes hardware decoding practical in places where it currently isn't used. I'm assuming that whatever Apple did with HEIC could be transposed here.

[tdaede]

Specific to AV1's case, we also have tiling as as bitstream feature. This has the advantage of variable sized tiles, yet also does not require a hardware round-trip (AFAIK most hardware implementations support tiles natively), and also filters across tiles, making a seam on the tile boundary less likely.

[leo-barnes]

So my understanding is that allowing variable tile sizes means that the hardware decoders have to be re-initialized for each image. This ultra-constrained profile essentially fixes the tile size and whatever else is necessary to prevent re-initialization so that implementers can just warm up the hardware once and reuse it multiple times.

Having a dedicated profile is essentially cat herding, CDNs are more likely to use it for mobile clients if there is a higher likelihood of support for hardware decoding in browsers (for example). Whether that hope if true or false depends on whether the profile makes hardware decoding practical in places where it currently isn't used. I'm assuming that whatever Apple did with HEIC could be transposed here.

The things we (Apple) did with HEIC to make it HW friendly is already pretty much already built into AVIF by virtue of it being an extension of the MIAF spec (which started out as a cross-company discussion on best practices with HEIF to make it more interoperable).

Grids require all tiles to have the same codec configuration, which should guarantee that the same HW setup can be used for all tiles, so that part is already covered.

[indolering]

Sounds like main is good enough. Feel free to reopen if that's incorrect.