search

immich-app / immich

High performance self-hosted photo and video management solution.
https://immich.app
GNU Affero General Public License v3.0
39.06k stars 1.85k forks source link

Heif thumbnail generate failed with bad image dimensions on decode #6849

Open CatTail opened 5 months ago

CatTail commented 5 months ago

The bug

After add external library I saw lots of HEIF image thumbnails are not properly generated, after I try to refresh the thumbnail, I was able to see the error logs from immich mircoservices

image 
immich_microservices     | [Nest] 7  - 02/02/2024, 6:49:15 AM     LOG [PersonService] Creating new person for face 388648e7-ff80-4333-93eb-b24302b6d414
immich_microservices     | [Nest] 7  - 02/02/2024, 6:49:16 AM   ERROR [JobService] Unable to run job handler (thumbnailGeneration/generate-jpeg-thumbnail): Error: /mnt/media/dsc07956.heif: bad seek to 5510386
immich_microservices     | heifload: bad image dimensions on decode
immich_microservices     | [Nest] 7  - 02/02/2024, 6:49:16 AM   ERROR [JobService] Error: /mnt/media/dsc07956.heif: bad seek to 5510386
immich_microservices     | heifload: bad image dimensions on decode
immich_microservices     |     at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
immich_microservices     |     at MediaRepository.resize (/usr/src/app/dist/infra/repositories/media.repository.js:41:14)
immich_microservices     |     at MediaService.generateThumbnail (/usr/src/app/dist/domain/media/media.service.js:121:44)
immich_microservices     |     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
immich_microservices     |     at async MediaService.handleGenerateJpegThumbnail (/usr/src/app/dist/domain/media/media.service.js:108:28)
immich_microservices     |     at async /usr/src/app/dist/domain/job/job.service.js:116:37
immich_microservices     |     at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
immich_microservices     |     at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)

Below is the image produce the error test.heif.zip

The OS that Immich Server is running on

Ubuntu 22.04

Version of Immich Server

v1.94.1

Version of Immich Mobile App

v1.94.1

Platform with the issue

Your docker-compose.yml content

version: "3.8"

#
# WARNING: Make sure to use the docker-compose.yml of the current release:
#
# https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
#
# The compose file on main may not be compatible with the latest release.
#

name: immich

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    command: [ "start.sh", "immich" ]
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - ${ORIGINAL_LOCATION}:/mnt/media/original:ro
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env
    ports:
      - 2283:3001
    depends_on:
      - redis
      - database
    restart: always

  immich-microservices:
    container_name: immich_microservices
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    # extends:
    #   file: hwaccel.yml
    #   service: hwaccel
    command: [ "start.sh", "microservices" ]
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - ${ORIGINAL_LOCATION}:/mnt/media/original:ro
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env
    depends_on:
      - redis
      - database
    restart: always

  immich-machine-learning:
    container_name: immich_machine_learning
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
    volumes:
      - model-cache:/cache
    env_file:
      - .env
    restart: always

  redis:
    container_name: immich_redis
    image: redis:6.2-alpine@sha256:c5a607fb6e1bb15d32bbcf14db22787d19e428d59e31a5da67511b49bb0f1ccc
    restart: always

  database:
    container_name: immich_postgres
    image: tensorchord/pgvecto-rs:pg14-v0.1.11@sha256:0335a1a22f8c5dd1b697f14f079934f5152eaaa216c09b61e293be285491f8ee
    env_file:
      - .env
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
    volumes:
      - pgdata:/var/lib/postgresql/data
    restart: always

volumes:
  pgdata:
    driver: local
    driver_opts:
      type: none
      device: /media/immich/pgdata
      o: bind
  model-cache:
    driver: local
    driver_opts:
      type: none
      device: /media/immich/model-cache
      o: bind

Your .env content

# You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables

# The location where your uploaded files are stored
UPLOAD_LOCATION=/media/photos/upload
ORIGINAL_LOCATION=/media/photos/migration

# The Immich version to use. You can pin this to a specific version like "v1.71.0"
IMMICH_VERSION=release

Reproduction steps

I write a minimal script to reproduce the issue

const sharp = require('sharp');

sharp('test.heif')
.pipelineColorspace('rgb16')
  .toBuffer()

Additional information

No response

alextran1502 commented 5 months ago

looks like those files are corrupted somehow?

CatTail commented 5 months ago

looks like those files are corrupted somehow?

Can you help to define what is corrupted image, I'm not an expert in image format but this image can be properly displayed in my Mac and iPhone. I use additional two tools to extract it's metadata, can you help to provide more insight on why it's corrupted?

Note the image is attachment in above already.

$ heif-info test.heif
MIME type: image/heic
main brand: heix
compatible brands: mif1, heix, miaf, MiHA, jpeg, SHIF

image: 7008x4672 (id=7), primary
  colorspace: YCbCr, 4:2:0
  bit depth: 10
  thumbnail: 1616x1080
  thumbnail: 320x212
  thumbnail: 160x120
  color profile: nclx
  alpha channel: no
  depth channel: no
metadata:
  Exif: 39270 bytes
  XMP: 2948 bytes
transformations:
  angle (ccw): 90
region annotations:
  none
properties:
$ exiftool test.heif
ExifTool Version Number         : 12.40
File Name                       : test.heif
Directory                       : .
File Size                       : 5.3 MiB
File Modification Date/Time     : 2024:02:02 20:12:29+00:00
File Access Date/Time           : 2024:02:02 20:12:29+00:00
File Inode Change Date/Time     : 2024:02:02 20:12:29+00:00
File Permissions                : -rwx------
File Type                       : HEIF
File Type Extension             : heif
MIME Type                       : image/heif
Major Brand                     : High Efficiency Image Format still image (.HEIF)
Minor Version                   : 0.0.0
Compatible Brands               : mif1, heix, miaf, MiHA, jpeg, SHIF
Handler Type                    : Picture
Primary Item Reference          : 7
Meta Image Size                 : 7008x4672
Exif Byte Order                 : Little-endian (Intel, II)
Make                            : SONY
Camera Model Name               : ILCE-7M4
X Resolution                    : 350
Y Resolution                    : 350
Resolution Unit                 : inches
Software                        : ILCE-7M4 v2.00
Modify Date                     : 2023:03:18 11:03:38
Y Cb Cr Positioning             : Centered
F Number                        : 4.0
Exposure Program                : Program AE
ISO                             : 3200
Sensitivity Type                : Recommended Exposure Index
Recommended Exposure Index      : 3200
Exif Version                    : 0232
Create Date                     : 2023:03:18 11:03:38
Offset Time                     : -08:00
Offset Time Original            : -08:00
Offset Time Digitized           : -08:00
Components Configuration        : Y, Cb, Cr, -
Compressed Bits Per Pixel       : 1
Brightness Value                : 1.696875
Exposure Compensation           : 0
Max Aperture Value              : 4.0
Metering Mode                   : Multi-segment
Light Source                    : Unknown
Flash                           : Off, Did not fire
Brightness                      : 0
Long Exposure Noise Reduction   : On (unused)
High ISO Noise Reduction        : Normal
HDR                             : Off; Uncorrected image
WB Shift AB GM                  : 0 0
WB Shift AB GM Precise          : 0.00 0.00
Creative Style                  : Off
Color Temperature               : Auto
Color Compensation Filter       : 0
Scene Mode                      : Auto
Zone Matching                   : ISO Setting Used
Dynamic Range Optimizer         : Auto
Image Stabilization             : On
Color Mode                      : Off
Full Image Size                 : 7008x4672
Preview Image Size              : 1616x1080
File Format                     : Unknown (4 0 1 0)
Flash Exposure Compensation     : 0
White Balance Fine Tune         : 0
White Balance                   : Auto
Sony Model ID                   : ILCE-7M4
Multi Frame Noise Reduction     : Off
Picture Effect                  : Off
Soft Skin Effect                : Off
Vignetting Correction           : Auto
Lateral Chromatic Aberration    : Auto
Distortion Correction Setting   : Auto
Lens Type                       : E-Mount, T-Mount, Other Lens or no lens
Lens Spec                       : FE PZ 16-35mm F4 G
Auto Portrait Framed            : No
Flash Action                    : Did not fire
Electronic Front Curtain Shutter: On
Focus Mode                      : AF-S
AF Area Mode Setting            : Wide
Flexible Spot Position          : 0 0
AF Point Selected               : n/a
AF Points Used                  : (none)
AF Tracking                     : Face tracking
Focal Plane AF Points Used      : 0
Multi Frame NR Effect           : Normal
Focus Location                  : 7008 4672 4380 4078
Variable Low Pass Filter        : n/a
Priority Set In AWB             : Standard
Metering Mode 2                 : Multi-segment
Exposure Standard Adjustment    : 0
RAW File Type                   : n/a
Quality                         : Fine
Pixel Shift Info                : n/a
Shadows                         : 0
Highlights                      : 0
Fade                            : 0
Sharpness Range                 : +3
Clarity                         : +1
Focus Frame Size                : 569x570
JPEG-HEIF Switch                : HEIF
Flash Level                     : Normal
Release Mode                    : Normal
Sequence Number                 : Single
Anti-Blur                       : On (Shooting)
Intelligent Auto                : On
Sequence File Number            : 1
Sequence Length                 : 1 file
Camera Orientation              : Rotate 270 CW
Quality 2                       : HEIF
ISO Setting                     : Unknown (255)
ISO Auto Min                    : Auto
ISO Auto Max                    : Auto
Ambient Temperature             : 22 C
AF Area Mode                    : Face Tracking
Focus Position 2                : 179
Tag 9416 0000                   :   3
Sony ISO                        : 3270
Stops Above Base ISO            : 5.0
Sony Exposure Time 2            : 1/161
Exposure Time                   : 1/160
Sony F Number 2                 : 4.0
Sony Max Aperture Value         : 4.0
Sequence Image Number           : 1
Lens Format                     : Full-frame
Lens Mount                      : E-mount
Lens Type 2                     : Unknown E-mount lens or other lens
Distortion Corr Params          : 1 0 -3 -6 -10 -15 -20 -25 -29 -30 -28 -22 -11 5 26 54
Picture Profile                 : Gamma Still - Standard/Neutral (PP2)
Focal Length                    : 28.5 mm
Min Focal Length                : 16.0 mm
Max Focal Length                : 35.0 mm
Vignetting Corr Params          : 0 64 160 384 800 1536 2432 3424 4448 5504 6528 7520 8480 9376 10176 11008
Chromatic Aberration Corr Params: 187 257 305 332 337 311 283 287 295 289 272 242 201 150 91 35 182 104 44 0 -26 -34 -28 2 40 68 100 140 192 258 344 442
Battery Temperature             : 27.8 C
Battery Level                   : 61%
Lens Mount 2                    : E-mount
Lens Type 3                     : Unknown (32871)
Camera E-mount Version          : 1.80
Lens E-mount Version            : 1.60
Lens Firmware Version           : Ver.01.001
Shutter                         : Mechanical (2997 5410 6743)
Flash Status                    : No Flash present
Shutter Count                   :   8002
Shutter Count 2                 : 8002
Sony Exposure Time              : 1/161
Sony F Number                   : 4.0
Release Mode 2                  : Normal
Internal Serial Number          : f8fe0000d909
User Comment                    :
Sub Sec Time                    : 268
Sub Sec Time Original           : 268
Sub Sec Time Digitized          : 268
Flashpix Version                : 0100
Color Space                     : sRGB
Exif Image Width                : 7008
Exif Image Height               : 4672
File Source                     : Digital Camera
Scene Type                      : Directly photographed
Custom Rendered                 : Normal
Exposure Mode                   : Auto
Digital Zoom Ratio              : 1
Focal Length In 35mm Format     : 28 mm
Scene Capture Type              : Standard
Contrast                        : Normal
Saturation                      : Normal
Sharpness                       : Normal
Lens Info                       : 16-35mm f/4
Lens Model                      : FE PZ 16-35mm F4 G
Composite Image                 : Unknown
XMP Toolkit                     : Image::ExifTool 12.40
Date/Time Original              : 2023:03:18 19:03:38
Rating                          : 0
Rotation                        : 90
Color Representation            : nclx 1 13 5
Image Pixel Depth               : 10 10 10
HEVC Configuration Version      : 1
General Profile Space           : Conforming
General Tier Flag               : Main Tier
General Profile IDC             : Main 10
Gen Profile Compatibility Flags : High Throughput
Constraint Indicator Flags      : 13 8 0 0 0 0
General Level IDC               : 150 (level 5.0)
Min Spatial Segmentation IDC    : 0
Parallelism Type                : 0
Chroma Format                   : 4:2:0
Bit Depth Luma                  : 10
Bit Depth Chroma                : 10
Average Frame Rate              : 0
Constant Frame Rate             : Unknown
Num Temporal Layers             : 1
Temporal ID Nested              : No
Image Width                     : 7008
Image Height                    : 4672
Image Spatial Extent            : 7008x4672
Media Data Size                 : 5506282
Media Data Offset               : 4096
Aperture                        : 4.0
Image Size                      : 7008x4672
Lens ID                         : FE PZ 16-35mm F4 G
Megapixels                      : 32.7
Scale Factor To 35 mm Equivalent: 1.0
Shutter Speed                   : 1/160
Create Date                     : 2023:03:18 11:03:38.268-08:00
Date/Time Original              : 2023:03:18 19:03:38.268-08:00
Modify Date                     : 2023:03:18 11:03:38.268-08:00
Focus Distance 2                : 2.07 m
Circle Of Confusion             : 0.031 mm
Field Of View                   : 65.5 deg
Focal Length                    : 28.5 mm (35 mm equivalent: 28.0 mm)
Hyperfocal Distance             : 6.64 m
Light Value                     : 6.3
alextran1502 commented 5 months ago

Do you know the originality of this HEIC file? is it coming from other app that downloaded it?

CatTail commented 5 months ago

Do you know the originality of this HEIC file? is it coming from other app that downloaded it?

Yes, the photo was originally taken from my camera ILCE-7M4, then uploaded to Google Photos and I export it with Google Takeout. There is no manual modification to that photo.

haossu commented 5 months ago

same errror, same camera...

CatTail commented 5 months ago

Hi @alextran1502 I'm wondering if there is a way for me to use original photo instead of thumbnail in mobile and web as a workaround to this issue, this issue is kind of blocker to me as lots of my photos are not able to displayed properly.

alextran1502 commented 5 months ago

@CatTail On the mobile app, go to settings > advance > Prefer remote assets.

There is no option on the web app since the web cannot display HEIC content.

I didn't know A7M4 has HEIC content. Probably being encoded with some Sony specific data so it cannot be decoded correctly

duanshiqiang commented 4 months ago

I am facing same issue with photos taken via Sony A7M4, also actually the file extension is HIF in sd card which I need to manually change to HEIF for immich to recognize.

jrasm91 commented 4 months ago

You can open a feature request for hif support if you want.

mertalev commented 4 months ago

This seems relevant.

Aericio commented 2 months ago

I'm also having the same issue with photos taken on an ILCE-7M4. My HIF files come directly off the SD card. 

[Nest] 7  - 04/21/2024, 2:44:46 AM   ERROR [JobService] Unable to run job handler (thumbnailGeneration/generate-preview): Error: upload/upload/3ea9cc80-221a-4fa3-8661-2482c04d45c1/16/85/1685a7c3-8409-436f-8685-e6e3e99a52e4.HIF: bad seek to 4292616
heifload: bad image dimensions on decode
[Nest] 7  - 04/21/2024, 2:44:46 AM   ERROR [JobService] Error: upload/upload/3ea9cc80-221a-4fa3-8661-2482c04d45c1/16/85/1685a7c3-8409-436f-8685-e6e3e99a52e4.HIF: bad seek to 4292616
heifload: bad image dimensions on decode

I noticed that this issue only happens with photos that were shot vertically, whereas horizontal photos are able to be decoded without issue. When uploading vertical photos to Google Photos, the images are displayed horizontally rather than vertically. Only when opening using Sony's Imaging Edge Desktop app, the orientation of the images are correct.

@CatTail's sample image was also shot vertically. Attempting to upload their image on my instance caused the same error.

gongdao123 commented 2 weeks ago

I'm having this issue too, any update on this issue? This is a really great project BTW.

alextran1502 commented 2 weeks ago

@gongdao123 are you on ios 18 beta?

entonjackson commented 2 weeks ago

Same here image

latest version client and server same look in browser (chrome)

EDIT: Found a fix for me. I rebooted my server, deleted the affected images through web immich (also from trash). After that again reboot and then reuploaded from backup. Before the reboots the server was up for a very long time. And as it's not the most powerful machine, it could be a performance issue...

gongdao123 commented 2 weeks ago

@gongdao123 are you on ios 18 beta?

No, I'm uploaded via Chrome. And view it using Chrome as well.

alextran1502 commented 2 weeks ago

@gongdao123 can you attach a sample file from your library that can be displayed here?

gongdao123 commented 2 weeks ago

Sure thing. @alextran1502

DSC04673.HIF.zip

junolym commented 2 weeks ago

I'm also having the same issue with photos taken on an ILCE-7CM2. Here are the logs

heifload: bad image dimensions on decode at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19) at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14) at MediaService.generateThumbnail (/usr/src/app/dist/services/media.service.js:158:48) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async MediaService.handleGeneratePreview (/usr/src/app/dist/services/media.service.js:135:29) at async /usr/src/app/dist/services/job.service.js:148:36 at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28) at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)

gongdao123 commented 20 hours ago

Me too. The image with this issue is shot with ILCE-7CM2.

Any update on this issue? @alextran1502

mertalev commented 13 hours ago

I believe this is an open issue in libheif and stems from the HEIF here being a bit quirky (but not invalid). But per that issue, the actual error seems to come from how libheif is used. Since libvips is what uses libheif, it might require a change in their handler.