search

w3c / png

Maintenance of the PNG specification
https://w3c.github.io/png/
Other
46 stars 11 forks source link

Web-compat restriction on location of `eXIf` chunk #146

Closed svgeesus closed 2 years ago

svgeesus commented 2 years ago

Raised by Phil Harvey phil@owl.phy.queensu.ca

Although the original eXIf specification didn't mandate a location for the eXIf chunk, web developers are advising to ignore the eXIf if it comes after IDAT:

https://github.com/w3c/csswg-drafts/issues/4929

As such, you may want to consider amending the specification to reflect this.
Minimally, it should be recommend to place eXIf before IDAT for web compatibility.
Or possibly requiring it to come before IDAT.

Originally noted here and should have been raised as a separate issue

svgeesus commented 2 years ago

@tabatkins wrote, in 2020

Chrome, Safari, and Firefox all now respect image-orientation: from-image on JPGs (in their latest versions, make sure you're updated), as shown in https://ericportis.com/etc/PNG-EXIF-orientation/.

Chrome and Firefox don't currently support it on PNGs at all (but plan to fix that). Safari does support EXIF on PNGs, but only if it comes before the image data (the PNG spec technically allows it to show up before or after). (The test linked above has the EXIF after the image data.)

svgeesus commented 2 years ago

At minimum we should say that Web browsers displaying images in HTML will ignore the EXIF data in PNG if it comes after IDAT.

Preferably, we should say that eXIf must come before IDAT

I'm currently adding the missing eXIF chunk to the chunk-ordering diagrams, so blocked on what we decide here. https://github.com/w3c/PNG-spec/pull/142

svgeesus commented 2 years ago

Also this useful comment by @eeeps

It might be helpful to differentiate between render-impacting and non-render-impacting metadata, and not be specific about the EXIF metadata format.

Encoders often want to put non-render-impacting metadata after the image data, so that image data is received (and in the case of progressive formats, can be painted) ASAP. See for instance, the suggested order of "extended" WebP metadata: https://developers.google.com/speed/webp/docs/riff_container#extended_file_format

Some formats may go further and mandate, rather than suggest, that EXIF come after. I think JPEG-XL does this? (though, IIRC, it has special-purpose format-specific metadata fields for render-impacting things like orientation and resolution that must come before image data). cc: @jonsneyers

svgeesus commented 2 years ago

Actually, looking at the Chunk ordering requirements almost all chunks are required before IDAT. The only ones that are not are:

so it would be entirely consistent to require eXIf before IDAT.

ProgramMax commented 2 years ago

I agree to require it before IDAT. Anything which could impact how the image is decoded / rendered should have that requirement.

EXIF is a bit odd because some data might impact decode & rendering while other data doesn't. As mentioned by @eeeps ideally we could put non-impact data at the end of the image to not delay decode / rendering. But it is what it is.

I worried that updates an old requirement to become more strict would break things. But if no browsers respected it anyway, I'm not as worried about breaking things. :)

randy408 commented 2 years ago

If it was made a requirement wouldn't that technically break existing images? Usually when a known, non-critical chunk is found in the wrong location it's ignored by decoders. If this requirement was ignored (to keep compatibility with existing images) would that make the decoder non-conformant?

ProgramMax commented 2 years ago

With web browser decoders already ignoring it, nothing in their decode->render pipeline changes. Other decoders might behave differently but at that point it is lesser-of-two-evils.

svgeesus commented 2 years ago

If it was made a requirement wouldn't that technically break existing images?

It could; that is because eXIf started out in an extensions spec and was rather under-specified.

But in practice, that turns "this image is technically allowed but browsers don't display it properly" into "this image does not display properly because it is wrong (and fixable)"

svgeesus commented 2 years ago

Having recently edited all the lattice diagrams which show chunk ordering constraints, it is clear that the only chunks allowed after IDAT are:

  1. Chunks with no effect at all on display of the static image (tEXt, zTXt, iTXt, tIME)
  2. Chunks which provide animated frames, which will display after IDAT (fdAT, fcTL)

Since eXIf can affect display of the static image, it should be specified to come before IDAT, which is currently the case.

I think this issue can thus be closed, with no further edits needed.