Should orientation metadata (such as exif) be used when drawing HTMLImageElements/SVGImageElements to canvas?

[weinig]

Its possible it is specified elsewhere, but https://html.spec.whatwg.org/multipage/canvas.html#dom-context-2d-drawimage doesn't seem to indicate whether orientation metadata (such as exif) should be respected when using drawImage.

Should it?

And if it should, should a CSS image-orientation property (e.g. image-orientation: none) computed for that image affect the rendering?

(my guess/suggestion here on what makes the most sense is that orientation metadata should be respected, but CSS image-orientation: should not be, just like the CSS to change the image's width and height is not respected. For users that want more control, ImageBitmap provides nobs).

[annevk]

Pretty sure that EXIF tags should be respected (and as per https://github.com/w3c/csswg-drafts/issues/5165#issuecomment-654127723 user agents should act as if it's baked in for opaque images to avoid leaking orientation in addition to width/height).

See also #7210 and #9693.

cc @whatwg/canvas

[kdashg]

Yes, this all sounds correct.

[noamr]

Pretty sure that EXIF tags should be respected (and as per w3c/csswg-drafts#5165 (comment) user agents should act as if it's baked in for opaque images to avoid leaking orientation in addition to width/height).

See also #7210 and #9693.

cc @whatwg/canvas

I believe https://github.com/w3c/csswg-drafts/issues/5165 is not relevant here because:

1. the only way the orientation can leak is by using image-orientation: none. I don't think that CSS property should be respected, as anyway the image element that's drawn to a canvas does not necessarily originate from a CSS-selectable DOM context. So without that CSS property the orientation is anyway "baked into the image".
2. drawing such an image into a canvas would taint the canvas anyway, so this information won't be leaked, no?

[weinig]

I think that’s right @noamr.

Would be good to clarify in the spec that:

• exif is respected
• CSS overrides are not

(perhaps a normative note about how ImageBitmap can be used if more control is required)

[schenney-chromium]

I did the Chromium implementation of image orientation from exif. Looking at the code I wrote back then, Chromium respects the image-orientation: none CSS property on the canvas element itself, otherwise always respects orientation and always respects orientation on cross origin content (or any content that would taint). The logic was that if canvas authors did not want orientation, they would have a consistent desire for all images in the canvas and hence the canvas element was a good place to put the CSS property.

I'm fine with removing that and always orienting according to exif.

[noamr]

I did the Chromium implementation of image orientation from exif. Looking at the code I wrote back then, Chromium respects the image-orientation: none CSS property on the canvas element itself, otherwise always respects orientation and always respects orientation on cross origin content (or any content that would taint). The logic was that if canvas authors did not want orientation, they would have a consistent desire for all images in the canvas and hence the canvas element was a good place to put the CSS property.

I'm fine with removing that and always orienting according to exif.

I think it's fine the way it is from a security standpoint, as the canvas pixels would be tainted anyway. However, I'd really like to see all of this in the HTML/CSS standards. None of this is spec'ed anywhere.