Cropping sometimes results in the wrong part of image (rotation/orientation?)

[aleksejrs]

What

• Sometimes it's just a little off (maybe cropping is lossless?), but sometimes, especially when the selection extends beyond an edge, it's shifted completely off of the actual selection.

  Part of

• Image editing

[TaciteOFF]

It has already been reported.

Selecting pixels outside of the picture lead to this.

That's indeed troublesome

[aleksejrs]

It's not always because outside pixels are selected visibly by the user: issue #440.

[aleksejrs]

I am not sure what I really mean in my last comment. If it's not a duplicate of #440, then maybe it's related to the Orientation tag in the uploaded image file?

[CharlesNepote]

Maybe I have found an example of what you mean with this product: https://world.openfoodfacts.org/cgi/product.pl?type=edit&code=3439495402643

To reproduce the issue based on this product:

• edit the product
• click on "rotate left"
• select the right zone
• click on "validate and/or resize the image"
• => the resulted image is displayed partly and with a wrong orientation

Tested either with Firefox and Chrome.

I think this is due to exif metadata. When I download the photo and try to open it with various software:

• some are opening it with a good orientation: Gimp, ImageMagick
• some are displaying it horizontally: Gthumb, ristretto, Firefox, etc.

In the exif metadata we can find: exif:Orientation: 6.

[CharlesNepote]

Here is another example: https://world.openfoodfacts.org/cgi/product.pl?type=edit&code=3415581571004 https://world.openfoodfacts.org/images/products/341/558/157/1004/3.jpg

exiftool -Orientation 3.jpg
Orientation                     : Rotate 90 CW

or

exiftool --printConv -Orientation 2.jpg
Orientation                     : 6

A normal photo is giving:

exiftool -Orientation 3.jpg
Orientation                     : Horizontal (normal)

or

exiftool --printConv -Orientation 3.jpg
Orientation                     : 1

Solution

We could run exiftool -r -n -Orientation=1 /path/to/my/files to reset all photos orientation.

At first, we could evaluate the number of photos which are concerned with: exiftool -r -if 'defined $orientation and $orientation# > 1' -p '$filename' ./ | wc -l

[hangy]

Are we storing the original images with EXIF info? I think that should already be stripped by https://github.com/openfoodfacts/openfoodfacts-server/blob/f14d0a4590c928b96eef23d9541a7ee4226f9211/lib/ProductOpener/Images.pm#L779C15-L779C15 prior to saving, because we don't want private metadata like location to be exposed. Maybe that's not working as expected based on the comment?

[CharlesNepote]

Thank you @hangy! I can see this code is very old (2014!). Finding all the photos with exif orientation will perhaps explain where the issue comes from (when did it start? etc.).

We have performance issues on the server nowadays, so I will perform tests later.

[hangy]

Here is another example: https://world.openfoodfacts.org/cgi/product.pl?type=edit&code=3415581571004 https://world.openfoodfacts.org/images/products/341/558/157/1004/3.jpg

Finding all the photos with exif orientation will perhaps explain where the issue comes from (when did it start? etc.).

The linked image was actually added in 2013, so the issue might affect only a relatively small amount of old images

[alexgarel]

Might be related: report from slack

It looks like image resizing is broken on https://world.openfoodfacts.org/ Looking inside the Response tab inside Devtools, the POST to product_image_crop.pl comes back with the following error: Can't locate object method "Resize" via package "Exception 410: no images defined Image::Magick::Q16' @ error/Q16.xs/XS_Image__Magick__Q16_Clone/3028" (perhaps you forgot to load "Exception 410: no images definedImage::Magick::Q16' @ error/Q16.xs/XS_ImageMagickQ16_Clone/3028"?) at /srv/off/lib/ProductOpener/Images.pm line 1612. Comes when attempting to rotate right and validate on this product: https://world.openfoodfacts.org/cgi/product.pl?type=edit&code=9300684270913#ingredients

I can't reproduce the error message, but the image is wrongly cropped, without the rotation:

Selected:

Result:

Request parameters:

Image link: https://images.openfoodfacts.org/images/products/930/068/427/0913/2.jpg

[CharlesNepote]

Might be related: report from slack [...]

Yes @alexgarel that's exactly what we're talking about.

At least twice a month I'm facing this issue. The last one: https://world.openfoodfacts.org/cgi/product.pl?type=edit&code=8413176932402 => The product has been created in 2022 so I guess it's not a very old bug back in 2013.

@alexgarel to better understand and measure the problem we could use exiftool, at least to see the number of files and their dates (made some tests on my local computer, it's working):

nice exiftool -r -q -q -if 'defined $orientation and $orientation# > 1' -printFormat '$DateTimeOriginal,$filename' ./ > ort_issue.csv
wc -l ort_issue.csv
sort ort_issue.csv | less

But I don't know exactly where to launch these commands: what's the directory for images?

[alexgarel]

what's the directory for images?

/mnt/off/images/products…

It might be better to do the tests on your computer though (avoid installing too much packages on the server, as it won't be documented)