Open g123k opened 1 day ago
alexgarel
commented
3 hours ago This relates to https://github.com/openfoodfacts/openfoodfacts-server/issues/9191
Thanks @g123k for spoting the reason (or at least one reason !), because I wasn't able to reproduce so far !
teolemon
commented
3 hours ago @alexgarel @raphael0202 @g123k what if that accounted for the drop in images ? we'd need to check when HEIC behaviors were tweaked by Apple/The Mobile App
g123k
commented
2 hours ago
@alexgarel
@raphael0202
@g123k what if that accounted for the drop in images ? we'd need to check when HEIC behaviors were tweaked by Apple/The Mobile App
The issue with the app is only when we pick from the gallery. I'm not sure we have that amount of users.
alexgarel
commented
2 hours ago This is the supported formats we have for ImageMagick on the server, so HEIC should be supported, but maybe there are more than the two subformat listed below ?
$ identify-im6 -list format|grep -i heic
AVIF* HEIC rw+ AV1 Image File Format (1.11.0)
HEIC* HEIC rw+ Apple High efficiency Image Format (1.11.0)FTR, the full list.
$ identify-im6 -list format
Format Module Mode Description
-------------------------------------------------------------------------------
3FR DNG r-- Hasselblad CFV/H3D39II
3G2 VIDEO r-- Media Container
3GP VIDEO r-- Media Container
AAI* AAI rw+ AAI Dune image
AI PDF rw- Adobe Illustrator CS2
APNG VIDEO rw+ Animated Portable Network Graphics
ART* ART rw- PFS: 1st Publisher Clip Art
ARW DNG r-- Sony Alpha Raw Image Format
AVI VIDEO r-- Microsoft Audio/Visual Interleaved
AVIF* HEIC rw+ AV1 Image File Format (1.11.0)
AVS* AVS rw+ AVS X image
BGR* BGR rw+ Raw blue, green, and red samples
BGRA* BGR rw+ Raw blue, green, red, and alpha samples
BGRO* BGR rw+ Raw blue, green, red, and opacity samples
BIE* JBIG rw- Joint Bi-level Image experts Group interchange format (2.1)
BMP* BMP rw- Microsoft Windows bitmap image
BMP2* BMP rw- Microsoft Windows bitmap image (V2)
BMP3* BMP rw- Microsoft Windows bitmap image (V3)
BRF* BRAILLE -w- BRF ASCII Braille format
CAL* CALS rw- Continuous Acquisition and Life-cycle Support Type 1
Specified in MIL-R-28002 and MIL-PRF-28002
CALS* CALS rw- Continuous Acquisition and Life-cycle Support Type 1
Specified in MIL-R-28002 and MIL-PRF-28002
CANVAS* XC r-- Constant image uniform color
CAPTION* CAPTION r-- Caption
CIN* CIN rw- Cineon Image File
CIP* CIP -w- Cisco IP phone image format
CLIP* CLIP rw+ Image Clip Mask
CMYK* CMYK rw+ Raw cyan, magenta, yellow, and black samples
CMYKA* CMYK rw+ Raw cyan, magenta, yellow, black, and alpha samples
CR2 DNG r-- Canon Digital Camera Raw Image Format
CR3 DNG r-- Canon Digital Camera Raw Image Format
CRW DNG r-- Canon Digital Camera Raw Image Format
CUR* ICON rw- Microsoft icon
CUT* CUT r-- DR Halo
DATA* INLINE rw+ Base64-encoded inline images
DCM* DCM r-- Digital Imaging and Communications in Medicine image
DICOM is used by the medical community for images like X-rays. The
specification, "Digital Imaging and Communications in Medicine
(DICOM)", is available at http://medical.nema.org/. In particular,
see part 5 which describes the image encoding (RLE, JPEG, JPEG-LS),
and supplement 61 which adds JPEG-2000 encoding.
DCR DNG r-- Kodak Digital Camera Raw Image File
DCX* PCX rw+ ZSoft IBM PC multi-page Paintbrush
DDS* DDS rw+ Microsoft DirectDraw Surface
DFONT* TTF r-- Multi-face font package (Freetype 2.10.4)
DJVU* DJVU r-- Deja vu
See http://www.djvuzone.org/ for details about the DJVU format. The
DJVU 1.2 specification is available there and at
ftp://swrinde.nde.swri.edu/pub/djvu/documents/.
DNG DNG r-- Digital Negative
DOT DOT --- Graphviz
DPX* DPX rw- SMPTE 268M-2003 (DPX 2.0)
Digital Moving Picture Exchange Bitmap, Version 2.0.
See SMPTE 268M-2003 specification at http://www.smtpe.org
DXT1* DDS rw+ Microsoft DirectDraw Surface
DXT5* DDS rw+ Microsoft DirectDraw Surface
EPDF PDF rw- Encapsulated Portable Document Format
EPI PS rw- Encapsulated PostScript Interchange format
EPS PS rw- Encapsulated PostScript
EPS2 PS2 -w- Level II Encapsulated PostScript
EPS3 PS3 -w+ Level III Encapsulated PostScript
EPSF PS rw- Encapsulated PostScript
EPSI PS rw- Encapsulated PostScript Interchange format
EPT EPT rw- Encapsulated PostScript with TIFF preview
EPT2 EPT rw- Encapsulated PostScript Level II with TIFF preview
EPT3 EPT rw+ Encapsulated PostScript Level III with TIFF preview
ERF DNG r-- Epson Raw Format
EXR EXR rw- High Dynamic-range (HDR) (OpenEXR 2.5.4)
FAX* FAX rw+ Group 3 FAX
FAX machines use non-square pixels which are 1.5 times wider than
they are tall but computer displays use square pixels, therefore
FAX images may appear to be narrow unless they are explicitly
resized using a geometry of "150x100%".
FILE* URL r-- Uniform Resource Locator (file://)
FITS* FITS rw- Flexible Image Transport System
FRACTAL* PLASMA r-- Plasma fractal image
FTP* URL r-- Uniform Resource Locator (ftp://)
FTS* FITS rw- Flexible Image Transport System
G3* FAX rw- Group 3 FAX
G4* FAX rw- Group 4 FAX
GIF* GIF rw+ CompuServe graphics interchange format
GIF87* GIF rw- CompuServe graphics interchange format (version 87a)
GRADIENT* GRADIENT r-- Gradual linear passing from one shade to another
GRAY* GRAY rw+ Raw gray samples
GRAYA* GRAY rw+ Raw gray and alpha samples
GROUP4* TIFF rw- Raw CCITT Group4
GV DOT --- Graphviz
H* MAGICK -w- Image expressed as a 'C/C++' char array
HALD* HALD r-- Identity Hald color lookup table image
HDR* HDR rw+ Radiance RGBE image format
HEIC* HEIC rw+ Apple High efficiency Image Format (1.11.0)
HISTOGRAM* HISTOGRAM -w- Histogram of the image
HRZ* HRZ rw- Slow Scan TeleVision
HTM* HTML -w- Hypertext Markup Language and a client-side image map
HTML* HTML -w- Hypertext Markup Language and a client-side image map
HTTP* URL r-- Uniform Resource Locator (http://)
HTTPS* URL r-- Uniform Resource Locator (https://)
ICB* TGA rw- Truevision Targa image
ICO* ICON rw+ Microsoft icon
ICON* ICON rw- Microsoft icon
IIQ DNG r-- Phase One Raw Image Format
INFO INFO -w+ The image format and characteristics
INLINE* INLINE rw+ Base64-encoded inline images
IPL* IPL rw+ IPL Image Sequence
ISOBRL* BRAILLE -w- ISO/TR 11548-1 format
ISOBRL6* BRAILLE -w- ISO/TR 11548-1 format 6dot
J2C* JP2 rw- JPEG-2000 Code Stream Syntax (2.4.0)
J2K* JP2 rw- JPEG-2000 Code Stream Syntax (2.4.0)
JBG* JBIG rw+ Joint Bi-level Image experts Group interchange format (2.1)
JBIG* JBIG rw+ Joint Bi-level Image experts Group interchange format (2.1)
JNG* PNG rw- JPEG Network Graphics
See http://www.libpng.org/pub/mng/ for details about the JNG
format.
JNX* JNX r-- Garmin tile format
JP2* JP2 rw- JPEG-2000 File Format Syntax (2.4.0)
JPC* JP2 rw- JPEG-2000 Code Stream Syntax (2.4.0)
JPE* JPEG rw- Joint Photographic Experts Group JFIF format (libjpeg-turbo 2.0.6)
JPEG* JPEG rw- Joint Photographic Experts Group JFIF format (libjpeg-turbo 2.0.6)
JPG* JPEG rw- Joint Photographic Experts Group JFIF format (libjpeg-turbo 2.0.6)
JPM* JP2 rw- JPEG-2000 Code Stream Syntax (2.4.0)
JPS* JPEG rw- Joint Photographic Experts Group JFIF format (libjpeg-turbo 2.0.6)
JPT* JP2 rw- JPEG-2000 File Format Syntax (2.4.0)
JSON JSON -w+ The image format and characteristics
K25 DNG r-- Kodak Digital Camera Raw Image Format
KDC DNG r-- Kodak Digital Camera Raw Image Format
LABEL* LABEL r-- Image label
M2V VIDEO rw+ MPEG Video Stream
M4V VIDEO rw+ Raw VIDEO-4 Video
MAC* MAC r-- MAC Paint
MAGICK* MAGICK rw- Predefined Magick Image (LOGO, ROSE, etc.); output same as 'H'
MAP* MAP rw- Colormap intensities and indices
MASK* MASK rw+ Image Clip Mask
MAT MAT rw+ MATLAB level 5 image format
MATTE* MATTE -w+ MATTE format
MEF DNG r-- Mamiya Raw Image File
MIFF* MIFF rw+ Magick Image File Format
MKV VIDEO rw+ Multimedia Container
MNG* PNG rw+ Multiple-image Network Graphics (libpng 1.6.37)
See http://www.libpng.org/pub/mng/ for details about the MNG
format.
MONO* MONO rw- Raw bi-level bitmap
MOV VIDEO rw+ MPEG Video Stream
MP4 VIDEO rw+ VIDEO-4 Video Stream
MPC* MPC rw+ Magick Persistent Cache image format
MPG VIDEO rw+ MPEG Video Stream
MRW DNG r-- Sony (Minolta) Raw Image File
MSL* MSL rw+ Magick Scripting Language
MSVG* SVG rw+ ImageMagick's own SVG internal renderer
MTV* MTV rw+ MTV Raytracing image format
MVG* MVG rw- Magick Vector Graphics
NEF DNG r-- Nikon Digital SLR Camera Raw Image File
NRW DNG r-- Nikon Digital SLR Camera Raw Image File
NULL* NULL rw- Constant image of uniform color
ORF DNG r-- Olympus Digital Camera Raw Image File
OTB* OTB rw- On-the-air bitmap
OTF* TTF r-- Open Type font (Freetype 2.10.4)
PAL* UYVY rw- 16bit/pixel interleaved YUV
PALM* PALM rw+ Palm pixmap
PAM* PNM rw+ Common 2-dimensional bitmap format
PANGO* PANGO r-- Pango Markup Language (Pangocairo 1.46.2)
PATTERN* PATTERN r-- Predefined pattern
PBM* PNM rw+ Portable bitmap format (black and white)
PCD* PCD rw- Photo CD
PCDS* PCD rw- Photo CD
PCL PCL rw+ Printer Control Language
PCT* PICT rw- Apple Macintosh QuickDraw/PICT
PCX* PCX rw- ZSoft IBM PC Paintbrush
PDB* PDB rw+ Palm Database ImageViewer Format
PDF PDF rw+ Portable Document Format
PDFA PDF rw+ Portable Document Archive Format
PEF DNG r-- Pentax Electronic File
PES* PES r-- Embrid Embroidery Format
PFA* TTF r-- Postscript Type 1 font (ASCII) (Freetype 2.10.4)
PFB* TTF r-- Postscript Type 1 font (binary) (Freetype 2.10.4)
PFM* PFM rw+ Portable float format
PGM* PNM rw+ Portable graymap format (gray scale)
PGX* PGX rw- JPEG 2000 uncompressed format
PICON* XPM rw- Personal Icon
PICT* PICT rw- Apple Macintosh QuickDraw/PICT
PIX* PIX r-- Alias/Wavefront RLE image format
PJPEG* JPEG rw- Joint Photographic Experts Group JFIF format (libjpeg-turbo 2.0.6)
PLASMA* PLASMA r-- Plasma fractal image
PNG* PNG rw- Portable Network Graphics (libpng 1.6.37)
See http://www.libpng.org/ for details about the PNG format.
PNG00* PNG rw- PNG inheriting bit-depth, color-type from original if possible
PNG24* PNG rw- opaque or binary transparent 24-bit RGB (zlib 1.2.11)
PNG32* PNG rw- opaque or transparent 32-bit RGBA
PNG48* PNG rw- opaque or binary transparent 48-bit RGB
PNG64* PNG rw- opaque or transparent 64-bit RGBA
PNG8* PNG rw- 8-bit indexed with optional binary transparency
PNM* PNM rw+ Portable anymap
POCKETMOD PDF rw+ Pocketmod Personal Organizer
PPM* PNM rw+ Portable pixmap format (color)
PREVIEW* PREVIEW -w- Show a preview an image enhancement, effect, or f/x
PS PS rw+ PostScript
PS2 PS2 -w+ Level II PostScript
PS3 PS3 -w+ Level III PostScript
PSB* PSD rw+ Adobe Large Document Format
PSD* PSD rw+ Adobe Photoshop bitmap
PTIF* TIFF rw+ Pyramid encoded TIFF
PWP* PWP r-- Seattle Film Works
RADIAL-GRADIENT* GRADIENT r-- Gradual radial passing from one shade to another
RAF DNG r-- Fuji CCD-RAW Graphic File
RAS* SUN rw+ SUN Rasterfile
RAW DNG r-- Raw
RGB* RGB rw+ Raw red, green, and blue samples
RGBA* RGB rw+ Raw red, green, blue, and alpha samples
RGBO* RGB rw+ Raw red, green, blue, and opacity samples
RGF* RGF rw- LEGO Mindstorms EV3 Robot Graphic Format (black and white)
RLA* RLA r-- Alias/Wavefront image
RLE* RLE r-- Utah Run length encoded image
RMF DNG r-- Raw Media Format
RW2 DNG r-- Panasonic Lumix Raw Image
SCR* SCR r-- ZX-Spectrum SCREEN$
SCT* SCT r-- Scitex HandShake
SFW* SFW r-- Seattle Film Works
SGI* SGI rw+ Irix RGB image
SHTML* HTML -w- Hypertext Markup Language and a client-side image map
SIX* SIXEL rw- DEC SIXEL Graphics Format
SIXEL* SIXEL rw- DEC SIXEL Graphics Format
SPARSE-COLOR* TXT -w+ Sparse Color
SR2 DNG r-- Sony Raw Format 2
SRF DNG r-- Sony Raw Format
STEGANO* STEGANO r-- Steganographic image
SUN* SUN rw+ SUN Rasterfile
SVG* SVG rw+ Scalable Vector Graphics (XML 2.9.10)
SVGZ* SVG rw+ Compressed Scalable Vector Graphics (XML 2.9.10)
TEXT* TXT rw+ Text
TGA* TGA rw- Truevision Targa image
THUMBNAIL* THUMBNAIL -w+ EXIF Profile Thumbnail
TIFF* TIFF rw+ Tagged Image File Format (LIBTIFF, Version 4.2.0)
TIFF64* TIFF rw+ Tagged Image File Format (64-bit) (LIBTIFF, Version 4.2.0)
TILE* TILE r-- Tile image with a texture
TIM* TIM r-- PSX TIM
TTC* TTF r-- TrueType font collection (Freetype 2.10.4)
TTF* TTF r-- TrueType font (Freetype 2.10.4)
TXT* TXT rw+ Text
UBRL* BRAILLE -w- Unicode Text format
UBRL6* BRAILLE -w- Unicode Text format 6dot
UIL* UIL -w- X-Motif UIL table
UYVY* UYVY rw- 16bit/pixel interleaved YUV
VDA* TGA rw- Truevision Targa image
VICAR* VICAR rw- VICAR rasterfile format
VID* VID rw+ Visual Image Directory
VIDEO VIDEO rw+ MPEG Video Stream
VIFF* VIFF rw+ Khoros Visualization image
VIPS* VIPS rw+ VIPS image
VST* TGA rw- Truevision Targa image
WBMP* WBMP rw- Wireless Bitmap (level 0) image
WEBM VIDEO rw+ Open Web Media
WEBP* WEBP rw+ WebP Image Format (libwebp 0.6.1 [020E])
WMF* WMF r-- Windows Meta File
WMV VIDEO rw+ Windows Media Video
WMZ* WMF r-- Compressed Windows Meta File
WPG* WPG r-- Word Perfect Graphics
X* X rw+ X Image
X3F DNG r-- Sigma Camera RAW Picture File
XBM* XBM rw- X Windows system bitmap (black and white)
XC* XC r-- Constant image uniform color
XCF* XCF r-- GIMP image
XPM* XPM rw- X Windows system pixmap (color)
XPS XPS r-- Microsoft XML Paper Specification
XV* VIFF rw+ Khoros Visualization image
XWD* XWD rw- X Windows system window dump (color)
YCbCr* YCbCr rw+ Raw Y, Cb, and Cr samples
YCbCrA* YCbCr rw+ Raw Y, Cb, Cr, and alpha samples
YUV* YUV rw- CCIR 601 4:1:1 or 4:2:2This is the supported formats we have for ImageMagick on the server, so HEIC should be supported, but maybe there are more than the two subformat listed below ?
$ identify-im6 -list format|grep -i heic AVIF* HEIC rw+ AV1 Image File Format (1.11.0) HEIC* HEIC rw+ Apple High efficiency Image Format (1.11.0)
Have you tried with my image?
This is the output of the identify command:
Image:
Filename: /Users/g123k/Downloads/IMG_1413 (1).HEIC
Permissions: rw-r--r--
Format: HEIC (High Efficiency Image Format)
Mime type: image/heic
Class: DirectClass
Geometry: 3024x4032+0+0
Resolution: 72x72
Print size: 42x56
Units: PixelsPerInch
Colorspace: sRGB
Type: TrueColor
Base type: Undefined
Endianness: Undefined
Depth: 8-bit
Channels: 3.0
Channel depth:
Red: 8-bit
Green: 8-bit
Blue: 8-bit
Channel statistics:
Pixels: 12192768
Red:
min: 0 (0)
max: 255 (1)
mean: 130.521 (0.511846)
median: 158 (0.619608)
standard deviation: 78.4264 (0.307555)
kurtosis: -1.53487
skewness: -0.253401
entropy: 0.947105
Green:
min: 0 (0)
max: 255 (1)
mean: 129.326 (0.507161)
median: 154 (0.603922)
standard deviation: 77.4462 (0.303711)
kurtosis: -1.53972
skewness: -0.235447
entropy: 0.950048
Blue:
min: 0 (0)
max: 255 (1)
mean: 118.182 (0.463458)
median: 131 (0.513725)
standard deviation: 73.9968 (0.290184)
kurtosis: -1.53039
skewness: -0.0741756
entropy: 0.949895
Image statistics:
Overall:
min: 0 (0)
max: 255 (1)
mean: 126.01 (0.494155)
median: 147.667 (0.579085)
standard deviation: 76.6231 (0.300483)
kurtosis: -1.53499
skewness: -0.187675
entropy: 0.949016
Rendering intent: Perceptual
Gamma: 0.454545
Chromaticity:
red primary: (0.64,0.33,0.03)
green primary: (0.3,0.6,0.1)
blue primary: (0.15,0.06,0.79)
white point: (0.3127,0.329,0.3583)
Matte color: grey74
Background color: white
Border color: srgb(223,223,223)
Transparent color: black
Interlace: None
Intensity: Undefined
Compose: Over
Page geometry: 3024x4032+0+0
Dispose: Undefined
Iterations: 0
Compression: Undefined
Orientation: TopLeft
Profiles:
Profile-exif: 2898 bytes
Profile-icc: 536 bytes
Properties:
date:create: 2024-11-26T09:07:31+00:00
date:modify: 2024-11-26T09:07:23+00:00
date:timestamp: 2024-11-26T09:08:52+00:00
exif:ApertureValue: 193685/85136
exif:BrightnessValue: 19433/4544
exif:ColorSpace: 65535
exif:DateTime: 2024:11:22 05:48:26
exif:DateTimeDigitized: 2024:11:22 05:48:26
exif:DateTimeOriginal: 2024:11:22 05:48:26
exif:DigitalZoomRatio: 42983/24207
exif:ExifOffset: 230
exif:ExifVersion: 0232
exif:ExposureBiasValue: 0/1
exif:ExposureMode: 0
exif:ExposureProgram: 2
exif:ExposureTime: 1/101
exif:Flash: 16
exif:FNumber: 11/5
exif:FocalLength: 111/50
exif:FocalLengthIn35mmFilm: 24
exif:GPSAltitude: 280023/7102
exif:GPSAltitudeRef: .
exif:GPSDateStamp: 2024:11:22
exif:GPSDestBearing: 621901/2469
exif:GPSDestBearingRef: T
exif:GPSHPositioningError: 9663/1546
exif:GPSImgDirection: 621901/2469
exif:GPSImgDirectionRef: T
exif:GPSInfo: 2582
exif:GPSLatitude: 48/1,54/1,392/100
exif:GPSLatitudeRef: N
exif:GPSLongitude: 2/1,18/1,5488/100
exif:GPSLongitudeRef: E
exif:GPSSpeed: 0/1
exif:GPSSpeedRef: K
exif:GPSTimeStamp: 4/1,48/1,2337/100
exif:LensMake: Apple
exif:LensModel: iPhone 14 Pro Max back triple camera 2.22mm f/2.2
exif:LensSpecification: 111/50, 111/50, 111/50, 111/50
exif:Make: Apple
exif:MakerNote: Apple iOS
exif:MeteringMode: 3
exif:Model: iPhone 14 Pro Max
exif:OffsetTime: +01:00
exif:OffsetTimeDigitized: +01:00
exif:OffsetTimeOriginal: +01:00
exif:PhotographicSensitivity: 125
exif:PixelXDimension: 4032
exif:PixelYDimension: 3024
exif:SceneType: .
exif:SensingMethod: 2
exif:ShutterSpeedValue: 65172/9791
exif:Software: 18.1
exif:SubjectArea: 2118, 2118, 2118, 2118
exif:SubSecTimeDigitized: 975
exif:SubSecTimeOriginal: 975
exif:WhiteBalance: 0
icc:copyright: Copyright Apple Inc., 2022
icc:description: Display P3
signature: 49715c82977f8af00483614b2e7ca012b556d53ee04860b5360042b7b210dfa9
unknown: iPhone 14 Pro Max
Artifacts:
verbose: true
Tainted: False
Filesize: 1.08926MiB
Number pixels: 12.1928M
Pixel cache type: Memory
Pixels per second: 71.3087MP
User time: 0.450u
Elapsed time: 0:01.170
Version: ImageMagick 7.1.1-41 Q16-HDRI aarch64 22504 https://imagemagick.orgYes I got the same thing, geometry is correctly identified:
$ identify-im6 -verbose /tmp/IMG_1413.HEIC
Image:
Filename: /tmp/IMG_1413.HEIC
Format: HEIC (Apple High efficiency Image Format)
Mime type: image/x-heic
Class: DirectClass
Geometry: 3024x4032+0+0
Resolution: 72x72
Print size: 42x56
Units: PixelsPerInch
...(the division by zero is because of wrong geometry detection)
alexgarel
commented
2 hours ago @g123k do you know what the API call should look like ?
alexgarel
commented
2 hours ago I have tried to use this image on a dev environment, with the web form, and it works.
alexgarel
commented
1 hour ago I'm on a track to find products that would have a size w or h of 0
I have to run this query on the parquet file:
SELECT code, images FROM food WHERE len(list_filter(images, img -> img.sizes.full.h = 0 OR img.sizes.full.w = 0)) > 0 LIMIT 10;I may try also with 400 in place of full.
alexgarel
commented
1 hour ago I uploaded the food.parquet from hugging face, then I did:
CREATE TABLE images AS SELECT code,images FROM read_parquet('food.parquet');
SELECT code, images FROM images WHERE len(list_filter(images, img -> img.sizes.full.h is null OR img.sizes.full.w is null)) > 0 LIMIT 10;
SELECT code, images FROM images WHERE len(list_filter(images, img -> img.sizes."400".h is null OR img.sizes."400".w is null)) > 0 LIMIT 10;both requests returns some data.
There are only 21 problematic products (I only look at uploaded images, not selected):
SELECT code FROM images WHERE len(list_filter(images, img -> img.imgid is NULL AND (img.sizes.full.h is null OR img.sizes.full.w is null OR img.sizes."400".h is null OR img.sizes."400".w is null))) > 0;
┌───────────────┐
│ code │
│ varchar │
├───────────────┤
│ 2000000029566 │
│ 24022088 │
│ 24049948 │
│ 24082655 │
│ 26053592 │
│ 3017760756297 │
│ 3046920028004 │
│ 3250390224747 │
│ 3250392108656 │
│ 3564700004784 │
│ 3760029248001 │
│ 5000159461313 │
│ 5038512005010 │
│ 5060283760003 │
│ 5449000050205 │
│ 5601151161456 │
│ 5601151354452 │
│ 5604450260226 │
│ 8003340073545 │
│ 8410376039207 │
│ 8480017516114 │
├───────────────┤
│ 21 rows │
└───────────────┘Looking at it closely:
.excel
SELECT code, list_filter(images, img -> img.imgid is NULL AND (img.sizes.full.h is null OR img.sizes.full.w is null OR img.sizes."400".h is null OR img.sizes."400".w is null)) FROM images WHERE len(list_filter(images, img -> img.imgid is NULL AND (img.sizes.full.h is null OR img.sizes.full.w is null OR img.sizes."400".h is null OR img.sizes."400".w is null))) > 0;They are all but one false positive…
stephanegigandet
commented
56 minutes ago I tried HEIC as well and it seems to be working (at first I thought we may be missing delegate for HEIC).
All the images from one product that triggered the error: https://openfoodfacts.sentry.io/issues/3901370484/?project=5376745&referrer=slack (4002809025761) can be read by ImageMagick well....
stephanegigandet
commented
46 minutes ago @g123k can you try it again, I added a bit more logs
alexgarel
commented
45 minutes ago @stephanegigandet could it be some race condition ? Like we try to crop an image that is not yet in the product, and auto-vivification use undef values instead of clean values ? Or that the mobile app, because of the task queue ask for crop before upload is complete ?
If we have undef, perl raise the division by zero error.
$ 1 / undef
Use of uninitialized value in division (/) at (eval 308) line 5.
Runtime error: Illegal division by zero at (eval 308) line 5.So the issue is that we are trying to crop an image that hasn't been correctly written to the product:
https://world.openfoodfacts.org/api/v3/product/4061464973442 : there is no image 5 (even though we do have the image file on the disk)
but the app is trying to select it:
off@off:/srv/off/logs$ grep "image_crop" modperl_error_log | grep "w: ," image_crop.pl - source_path: /srv/off/html/images/products/406/146/497/3442/5.jpg - product_id: 4061464973442 - imgid: 5 - crop_size: 400 - x1: , y1: , x2: , y2: , w: , h:
So we get undef values for $w and $h:
my $w = $new_product_ref->{images}{$imgid}{sizes}{$coordinates_image_size}
{w};
my $h = $new_product_ref->{images}{$imgid}{sizes}{$coordinates_image_size}
{h};
Image 5 was deleted in September: https://world.openfoodfacts.org/product/4061464973442/haselnuss-schnitte-choceur (or more exactly, it was removed from the images structure, but we still have the file on disk... )
stephanegigandet
commented
28 minutes ago So I guess we should just add a check in process_image_crop that we do have the image in the images structure, with non zero sizes.
g123k
commented
28 minutes ago If that may help you, I've reuploaded the image to https://world.openfoodfacts.org/product/0098765432186/test-product It was at 12:27
Hi everyone!
From the mobile app, I try to upload the following picture: drive.google.com/file/d/1iMwOScee68OgXPQGDxhHr_tqJ81NThN6/view?usp=sharing
However, I receive the following error:
Date/time: 2024-11-24T21:31:22.000261