[css-images] Should CSS decorative images respect EXIF-orientation by default

[smfr]

https://drafts.csswg.org/css-images/#the-image-orientation says that image-orientation only affects content images, not decorative images.

But what is the default orientation for a decorative image (e.g. CSS background) which has EXIF data that describes a rotation?

[tabatkins]

Undefined by this specification!

But it obviously makes sense to align with image-orientation, and say that you respect EXIF.

[frivoal]

That makes sense indeed, but is it web compatible?

[smfr]

It's also odd that image-resolution is specified to affect "all raster images used in or on the element. It affects both content images (e.g. replaced elements and generated content) and decorative images" but image-orientation is not.

Another option would be to have an argument to the image() function that allows image-orientation-like control.

[css-meeting-bot]

The CSS Working Group just discussed Should CSS decorative images respect EXIF-orientation by default, and agreed to the following:

• RESOLVED: default behavior for all images to respect exif orientation

The full IRC log of that discussion

<dael> Topic: Should CSS decorative images respect EXIF-orientation by default
<dael> github: https://github.com/w3c/csswg-drafts/issues/4165
<dael> astearns: Added by smfr
<dael> smfr: Cleaning up issues for if images respect EXIF. Question came up if decorative ones should respecte exif. Should we allow authors to control orienation in decorative images?
<dael> smfr: Simple question is do decorative images respect exif by default
<dael> AmeliaBR: I think default behavior should be to respect it, esp if that's the default for content images. We should be consistent unless there is webcompat reasons
<dael> AmeliaBR: Like having extra param that can change things. Adding the extra shouldn't delay spec default
<dael> fantasai: Default behavior will have to go in L3, image function is in L4 so it would go there
<dael> smfr: One data point I don't think w have web compat data b/c no platform has respected exif for decorative images by default
<dael> astearns: Disctinction between content and docorative is backgroud?
<dael> smfr: Yeah. Content images are images in replaced elements
<dael> AmeliaBR: Instead of decorative we should say css images. Images spec through css prop. Exception is content property. New image orientation would effect images embedded using content
<dael> fantasai: Yeah. Early defined replaced to work using content property. Def interchangable. List markets and background images and stuff are not replace in box tree. Content images are
<dael> smfr: Images in SVG too which we haven't talked about
<AmeliaBR> s/New image orientation/New image-orientation property/
<dael> astearns: Concern was on compat for spec default to honor exif. Do you have any idea of what % images used on web have exif data?
<dael> smfr: TabAtkins had data. It's less common to use thing with exif data as decorative. I'm not as concerned as I was with content images. We can try and see what happens
<dael> astearns: I know people use background image for content data. I think it would be surprising to take a URL that's an image, but it in background and it flips
<TabAtkins> Yeah agree.
<dael> fantasai: Agree with AmeliaBR default should be consistent between all the images
<TabAtkins> Should agree unless there's compelling compat data.
<dael> smfr: Happy to resolve now and come back if compat
<dael> astearns: Prop: default behavior for all images to respect exif orientation
<dael> astearns: Additional concerns?
<dael> astearns: Obj?
<dael> RESOLVED: default behavior for all images to respect exif orientation
<dael> astearns: My understanding is we do not yet have param in image function to override in next module
<dael> fantasai: No. Inclination is not to add unless authors say they want this. We've historically had problems getting image-orientation impl so I'd rather not add without demand
<fantasai> s/image-orientation/image()/
<dael> AmeliaBR: Have vauge resolutions to add params to control image for things like lazy-load. Once we get all that ecosystem of params maybe this gets added in if there is demand
<dael> astearns: Seems fine place to leave it. Anyone want to fight to put in a param now?
<dael> smfr: I think it's fine. Agree with fantasai to wait
<dael> smfr: It is odd we have image-resolution apply to all aster but image-orientation is content images. Should think in the future
<dael> AmeliaBR: Re-access image resolution?
<dael> smfr: I would go in other where image-orientation should effect all images.
<dael> astearns: Since we're consistent in orientation data, consisten in orientation prop makes sense
<dael> fantasai: Reason why not is that images typically come from different sources. Might have a bunch of SVG used for background or border. NOt going to want to have that roate but might correct rotatio os a photograph. Discussed this in the past and decided does not effect anything other then content and images
<dael> fantasai: Makes sense to be consistent on exif. I don't think explicit values need to be everywhere. I don't think orientation wants to effect decorative and content
<dael> smfr: from-image is the only one you'd care abuout for css images
<dael> AmeliaBR: ONly you can assume applies to many images. If you had content and bg images wouldn't nec. align. Prob try for image-resolution prop. might be worth considering finger grain control there. I don't know if there are impl of that to get author feedback
<dael> fantasai: Idea was for css images that we would address use case to override explicit orientation through image()
<dael> astearns: Sounds like we're toward no change for rest of questions in issue. Correct?
<dael> fantasai: Open to adding image oritentation overrides in image() if there's demand. Defeault is respect exif
<dael> astearns: Then let's close this issue.

[zcorpan]

Can it be clarified what the above resolution means, in more detail? (To the extent that I or someone else could propose a PR and write tests.)

[heycam]

It's unclear from the meeting minutes whether it's intended that image-orientation affects CSS images used in an element, or if the orientation metadata in the image should unconditionally be honored.

Chrome Canary does look at image-orientation on the element, and a recent WebKit build with support for image-orientation does not (and always follows the orientation metadata).

[svgeesus]

@zcorpan wrote:

Can it be clarified what the above resolution means, in more detail? (To the extent that I or someone else could propose a PR and write tests.)

It seems to mean that EXIF orientation is always respected, if it occurs before the image data in the file; and always ignored, if it is afterwards.

[svgeesus]

@tabatkins @fantasai @LeaVerou

[svgeesus]

There are now 3 tests in WPT although with few passes

[schenney-chromium]

I did the Chromium implementation and the intention was to make use of EXIF orientation everywhere while respecting the "image-orientation: none" property if possible (i.e. if there is a reasonable element to pull it from).

There are open issues on WHATWG to add orientation control for canvas image methods. https://github.com/whatwg/html/issues/7210 https://github.com/whatwg/html/issues/8085

My understanding is that the long term goal was to remove the property, and always respect valid EXIF orientation data whenever the image is used. If that is indeed the goal, then we should not be adding parameters to canvas methods.

[svgeesus]

This is mainly about EXIF after the image data. Browsers did not want to be required to honor this as it would force a re-layout, especially problematic on large images which have already been displayed.

[css-meeting-bot]

The CSS Working Group just discussed [css-images] Should CSS decorative images respect EXIF-orientation by default.

The full IRC log of that discussion

<noamr> ChrisL: it wasn't clear what we've resolved on
<noamr> ChrisL: what this means that if you have EXIF that comes before the image data we must respect it
<noamr> ChrisL: for consistency we should probably ignore it when it comes after
<schenney> q+
<noamr> smfr: you mean in the order in which the image is encoded?
<noamr> ChrisL: yes, which is not always possible
<noamr> q+
<astearns> ack schenney
<noamr> schenney: I agree with ignoring after the image, but the q was whether the orientation should look at the image-orientation property on the element
<noamr> schenney: chrome does this but safari does not
<noamr> ChrisL: there wasn't a discussion on this, can we discuss now?
<noamr> schenney: one, I implemented this a while ago, the usage data for image-orientation and it's rising, but not in a huge amount
<noamr> schenney: people use it to avoid applying the image metadata
<smfr> q+
<noamr> schenney: from that point of view the current chrome behavior is preferrable for web devs. but the long term is perhaps to not have that property
<noamr> schenney: we'll get pushback if we do that
<noamr> ... if we don't honor image-orientation
<ChrisL> q+ how to say "ignore rotation" in that case?
<TabAtkins> scribe+
<bramus> scribe+
<astearns> ack noamr
<TabAtkins> noamr: even now, image-orietnation is broken in the sense that it doesn't work on cross-origin images
<schenney> q+
<TabAtkins> noamr: which is a real big breakage for people
<TabAtkins> noamr: I find it a broken CSS property because of that
<ChrisL> +1 to URL modifier
<TabAtkins> noamr: having it apply to CSS images, it's a bit weird you have something for the whole lement and apply it to all images. i'd expect it to be a url modifier or something, a keyword that says you read the exif
<TabAtkins> noamr: we have url() modifiers now for other things, we can put it there. i think that's the right place
<astearns> ack smfr
<noamr> scribe+
<TabAtkins> (+1 to that, it was clumsy to have it apply that way)
<noamr> smfr: it should be per-image, maybe on the image function
<noamr> smfr: background: webkit started supporting orientation by default on regular images, we just didn't get to CSS images
<noamr> smfr: I think we can change it to match the chromium behavior
<astearns> ack how
<Zakim> how, you wanted to say "ignore rotation" in that case?
<PaulG> q+
<noamr> CLilley: URL modifier is perhaps a good way to go
<astearns> ack schenney
<noamr> schenney: URL modifier would address the original reason why image orientation is ignored on cross-origin
<fantasai> I url modifiers should be for generic modifiers. We should use image() for image-specific modifiers.
<fantasai> And we should have more of them.
<astearns> ack dbaron
<noamr> dbaron: I'm more hesitant about modifiers. they make sense for features we want permanently. my impression is that this feature is more fore compatibility during transition
<noamr> dbaron: if we want to end up always honoring the metadata, we should only add as many features as we need for that, rather than add more granularity
<noamr> astearns: if we resolve on ignoring EXIF at the end, does that help with the transition?
<noamr> dbaron: it wasn't clear to me which cases the resolution applies to
<astearns> ack PaulG
<noamr> PaulG: for a11y I'd be concerned that if people relied on this feature for WCAG orientation, I wouldn't want to get rid of it
<noamr> PaulG: I suspect we'd want to keep that
<astearns> ack fantasai
<noamr> fantasai: modifiers should be in image() and not in url()
<noamr> q+
<astearns> ack noamr
<TabAtkins> Yeah since this is explicitly an image *file*, I'm fine with it living on url()
<ChrisL> q+
<noamr> astearns: we can probably resolve on ignoring metadata that's after the metadata
<noamr> schenney: I thought it was resolved a while ago
<astearns> ack ChrisL
<noamr> ChrisL: it was unclear enough that it's hard to say what a PASS is
<smfr> q+
<noamr> q+
<astearns> ack smfr
<noamr> smfr: we might not be able to do this in the implementation
<astearns> ack noamr
<astearns> ack fantasai
<noamr> fantasai: HTML should add that, but in CSS we should say how we handle images with EXIF, regardless of the host language
<smfr> q+
<noamr> it should at least be in both
<fantasai> s/should add that/could add special rules/
<astearns> ack smfr
<noamr> smfr: odd that CSS would define things around the encoding of an image
<noamr> q+
<fantasai> It doesn't need to be in both. It's a rendering question, it needs to be in CSS. HTML can optionally say stuff, but it doesn't need to -- we do.
<dbaron> s/HTML should add that/HTML could add that/
<TabAtkins> noamr: We talked before in WHATWG about separqting some of those image details to as eaprate spec
<TabAtkins> noamr: it never happened because nobody had time, but I think right now, relying on parts of HTML that defines how images are decoded, relying on that inside of CSs, is the best we'
<ChrisL> relevant PNG WG issue is https://github.com/w3c/png/issues/310
<TabAtkins> noamr: We should behave the same way for HTML and CSS
<noamr> astearns: it is claimed that all browsers ignore post-image-data orientation
<noamr> astearns: smfr would you validate?
<noamr> smfr: will validate
<noamr> astearns: will take it back to the issue about encoding order
<noamr> ... to validate that all browsers do this today
<noamr> ChrisL: please check with JPEG with PNG, I saw that Safari does something different
<noamr> smfr: that's why we need more time
<noamr> (In PNG this is encoded in XMP IIRC)
<noamr> astearns: for how we override the metadata
<ChrisL> No, it can be but there is also an exif chunk
<noamr> smfr: this is *this* issue, but we discussed a different one
<TabAtkins> noamr: also there's a proposal for a url() modifier. not clear yet. But at least not taking the info from image-orientation
<TabAtkins> schenney: so proposal is phase out a per-image way of specifying it, and phase-out image-orientation property
<TabAtkins> phase in an image-specific way to do *CSS images*.
<TabAtkins> dbaron: replaced elements, form HTML, would still obey image-orientation
<TabAtkins> astearns: so let's continue that part of the discussion in this issue
<noamr> astearns: action item for smfr to validate the tests

[smfr]

I filed https://github.com/w3c/csswg-drafts/issues/11114 on the issue about respecting EXIF depending on where it occurs in the file.