AOMediaCodec / av1-avif

AV1 Image File Format Specification - ISO-BMFF/HEIF derivative
https://aomediacodec.github.io/av1-avif/
BSD 2-Clause "Simplified" License
451 stars 40 forks source link

`professional-still` profile or `.avifpro` format? #142

Closed indolering closed 3 years ago

indolering commented 3 years ago

Forking discussion of my off-topic suggestions from #128.

Comparisons between AVIF and JPEG-XL had some people on the AV1 subreddit arguing that AVIF should support things like massive tile sizes, 14/16/32 bit color, floating point colors, a larger number of channels, better lossless compression, and other niceties that are only useful in professional image manipulation settings.

I'm not sure if this would be better off as a profile, a related image format, or both. There is demand out there for an open-source video codec for video editing. A distinct "pro" variants of AVx and AVxF supporting these features could be of use to others without increasing the implementation burden. But others argued that having distinct formats would lead to JPEG-XL becoming the One True Image Codec.

Either way, I thought it was an interesting idea worth floating here. My apologies if this is anti-helpful/stupid!

indolering commented 3 years ago

If we add an unconstrained 12-bit profile, how do we make it clear that it's not there for interoperability? What does a client gain from marking an image as belonging to this profile? ... What does a user gain by marking an image as using a non-constrained profile?

I don't think a lack of constraints is necessary, just whatever is needed for dumping raw image sensor values. 8/10/12/14/16 bit integer and 16/32 bit floating point seem like reasonable limits to me. I don't understand JPEG-XL choice of 24 bits as their integer limit, but apparently OpenCL supports 24 bit integer multiplication? Maybe Cloudinary is confusing precision with BPP?

JPEG-XL's 2^30 x 2^30 tile size limit (1,073,741,823 x 1,073,741,823) seems ... excessive. Even 8K is silly, but marketers know no bounds: DisplayPort's maximum resolution is 16K and there are some cameras on the market pushing 100+ megapixels. 32K sounds like an unreasonably large limit to me, a gigapixel would handle most radio telescopes and would probably be unlikely to fit into a consumer device ... ever.

JPEG-XL's choice of 4,100 color channels (which I'm pretty sure is actually 4096 but I'm too lazy to check) doesn't make any sense to me. Designers might use more than 5 colors if they are using spot colors, but printing presses rarely support more than a single spot color. Anyone who wants more than the 5 channels offered by AVIF would be better off using a PDF. But if we are in this to make some marketing department happy, 8-16 bits are beyond any practical use case.

But I personally agree that a professional-image profile would be of marginal benefit: it's like arguing Opus should support 96kHz audio when those people would be better off using FLAC. The only part that I think might have some merit would be a distinct codec for raw images and/or video editing. There would be synergy for professional applications, as supporting vanilla AV1/AVIF would make AV1-Pro/AVIF-Raw (or whatever) the obvious choice as an intermediate codec and vice-versa.

Breaking them off into a distinct format (even if it's just a specific profile under-the-hood) would mean browsers (etc) don't have to support a bunch of random features that aren't helpful for saving bandwidth. I personally think JPEG-XL is great and the lossless JPEG re-compression feature is really cool. But trying to compete with JPEG-XL on lossless compression or speed will require waiting for AV2 anyway, so maybe just punt this to AV2?

leo-barnes commented 3 years ago

I'm still not completely clear on what the ask is here. AVIF is a container format for storing AV1 in HEIF/MIAF. It only supports whatever AV1 supports. AV1 does not support 14/16 bit integer content or 16/32 bit floating point content, so we can't create a profile for that.

That type of content might end up being supported by AV2, but that's kind of beside the point given that AVIF currently only supports AV1.

indolering commented 3 years ago

It only supports whatever AV1 supports. AV1 does not support 14/16 bit integer content or 16/32 bit floating point content, so we can't create a profile for that.

I guess I was confused by debate in one of the other tickets concerning whether AVIF needed to explicitly support 12 bits. So by "unconstrained" you mean levels? I guess a maximum level somewhere in the 16K/32K/~1 gigapixel range?

That type of content might end up being supported by AV2, but that's kind of beside the point given that AVIF currently only supports AV1.

Well, then let's just close this ticket and throw it on the pile of ideas for AV2. I'm not really a fan of these enhancements anyway 😝.