Demo app: Images rotated if added from iOS / macOS photo library

[robertknight]

When loading an image into the web demo app directly from the iOS / macOS photo library, the images frequently end up rotated, though they are displayed in the correct orientation in the Photo Library itself. Due to https://github.com/robertknight/tesseract-wasm/issues/29, the OCR then doesn't work as expected. If I first export the image to a JPEG by dragging the image from Photos => Finder and then Finder => web app, it works as expected.

If the demo app automatically detected the orientation of the image and rotated it, that would solve the problem.

[robertknight]

https://github.com/robertknight/tesseract-wasm/pull/34 can help by providing a way to detect non-portrait orientation and then providing the user with a way to correct it.

[robertknight]

Update: This only happens in Safari, so I think it is just another case of the issue where Safari does not respect EXIF orientation information when loading images via createImageBitmap and then drawing them on the canvas. My guess as to why it works when exporting the image from Photos to disk (ie. Photos => Finder => web app) is that during the initial export, the orientation is applied to the actual order of pixels in the image.

See also https://github.com/whatwg/html/issues/7210#issuecomment-945237128.

[robertknight]

Possibly related WebKit issues:

https://bugs.webkit.org/show_bug.cgi?id=203482 (Orientation not preserved after image resizing) https://bugs.webkit.org/show_bug.cgi?id=207088 (EXIF metadata lost when using file upload)

I should note that I have the "High efficiency" (HEIF/HEVC) option selected in the Camera Formats settings in iOS.

[robertknight]

An option for Safari might be to handle reading and applying the EXIF orientation data from the file ourselves, eg. using https://github.com/exif-js/exif-js, this is assuming that the relevant EXIF data is present. The orientation is preserved when dragging from Photos => the NanoJPEG demo app: https://github.com/evanw/skew-nanojpeg, so it looks like it is possible.

[holtwick]

I remember that I also had this problem a long time ago. My solution was then based on this code. Maybe it is helpful. But just reading the EXIF data should also do.

https://stackoverflow.com/questions/7584794/accessing-jpeg-exif-rotation-data-in-javascript-on-the-client-side/32490603#32490603

[robertknight]

Thanks. Those answers suggest it might be possible to read EXIF info with a smaller amount of code than the full EXIF JS. It would be even better if the issue was fixed in WebKit though!