Error generating thumbnails in v1.118

[AFumi39]

The bug

Uploading jpeg images, immich v1.118.0 can't generate the preview images (thumbnails, etc), but the downloaded image is working. See the attachment

The OS that Immich Server is running on

Docker

Version of Immich Server

V1.118.0

Version of Immich Mobile App

1.117.0

Platform with the issue

• [X] Server
• [ ] Web
• [ ] Mobile

Your docker-compose.yml content

#
# WARNING: Make sure to use the docker-compose.yml of the current release:
#
# https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
#
# The compose file on main may not be compatible with the latest release.
#

name: immich

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION}
    # extends:
    #   file: hwaccel.transcoding.yml
    #   service: cpu # set to one of [nvenc, quicksync, rkmpp, vaapi, vaapi-wsl] for accelerated transcoding
    volumes:
      # Do not edit the next line. If you want to change the media storage location on your system, edit the value of UPLOAD_LOCATION in the .env file
      - ${UPLOAD_LOCATION}:/usr/src/app/upload:rw
      - /etc/localtime:/etc/localtime:ro
      - /volume1/docker/immich-app/ext-library:/mnt/media/ext-library:rw
    env_file:
      - stack.env
    ports:
      - 2283:2283
    depends_on:
      - redis
      - database
    restart: always
    healthcheck:
      disable: false
    labels:
      - 'wud.tag.include=v^([0-9]\d*)\.([0-9]\d*)\.([0-9]\d*)$$'
      - 'wud.watch.digest=true'
      - 'wud.link.template=https://github.com/immich-app/immich/releases/tag/$${raw}'
      - 'wud.display.name=Immich server'
      - 'wud.display.icon=mdi:image-multiple-outline'

  immich-machine-learning:
    container_name: immich_machine_learning
    # For hardware acceleration, add one of -[armnn, cuda, openvino] to the image tag.
    # Example tag: ${IMMICH_VERSION:-release}-cuda
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION}
    # extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/ml-hardware-acceleration
    #   file: hwaccel.ml.yml
    #   service: cpu # set to one of [armnn, cuda, openvino, openvino-wsl] for accelerated inference - use the `-wsl` version for WSL2 where applicable
    volumes:
      - model-cache:/cache:rw
    env_file:
      - stack.env
    restart: always
    healthcheck:
      disable: false
    labels:
      - 'wud.watch=false'

  redis:
    container_name: immich_redis
    image: docker.io/redis:6.2-alpine@sha256:e3b17ba9479deec4b7d1eeec1548a253acc5374d68d3b27937fcfe4df8d18c7e
    healthcheck:
      test: redis-cli ping || exit 1
    restart: always
    labels:
      - 'wud.watch=false'

  database:
    container_name: immich_postgres
    image: docker.io/tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
      POSTGRES_INITDB_ARGS: '--data-checksums'
    volumes:
      # Do not edit the next line. If you want to change the database storage location on your system, edit the value of DB_DATA_LOCATION in the .env file
      - ${DB_DATA_LOCATION}:/var/lib/postgresql/data:rw
    healthcheck:
      test: pg_isready --dbname='${DB_DATABASE_NAME}' --username='${DB_USERNAME}' || exit 1; Chksum="$$(psql --dbname='${DB_DATABASE_NAME}' --username='${DB_USERNAME}' --tuples-only --no-align --command='SELECT COALESCE(SUM(checksum_failures), 0) FROM pg_stat_database')"; echo "checksum failure count is $$Chksum"; [ "$$Chksum" = '0' ] || exit 1
      interval: 5m
      #start_interval: 30s
      start_period: 5m
    command: ["postgres", "-c", "shared_preload_libraries=vectors.so", "-c", 'search_path="$$user", public, vectors', "-c", "logging_collector=on", "-c", "max_wal_size=2GB", "-c", "shared_buffers=512MB", "-c", "wal_compression=on"]
    restart: always
    labels:
      - 'wud.watch=false'

  backup:
    container_name: immich_db_dumper
    image: prodrigestivill/postgres-backup-local:14
    restart: always
    env_file:
      - stack.env
    environment:
      POSTGRES_HOST: database
      POSTGRES_CLUSTER: 'TRUE'
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_DB: ${DB_DATABASE_NAME}
      SCHEDULE: "@daily"
      POSTGRES_EXTRA_OPTS: '--clean --if-exists'
      BACKUP_DIR: /db_dumps
    volumes:
      - /volume1/backup/immich_db_backup:/db_dumps:rw
    depends_on:
      - database
    labels:
      - 'wud.watch=false'

volumes:
  model-cache:

Your .env content

UPLOAD_LOCATION=/volume1/docker/immich-app/library
DB_DATA_LOCATION=/volume1/docker/immich-app/postgres
TZ=Europe/Rome
IMMICH_VERSION=v1.118.0
DB_PASSWORD=***
DB_USERNAME=postgres
DB_DATABASE_NAME=immich

Reproduction steps

1. 2. 3. ...

Relevant log output

st] 7  - 10/15/2024, 4:33:48 PM   ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-thumbnails): Error: VipsJpeg: ./lib/jpegli/decode_marker.cc:285: DHT marker: no Huffman table found

[Nest] 7  - 10/15/2024, 4:33:48 PM   ERROR [Microservices:JobService] Error: VipsJpeg: ./lib/jpegli/decode_marker.cc:285: DHT marker: no Huffman table found

    at Sharp.toBuffer (/usr/src/app/node_modules/sharp/lib/output.js:163:17)

    at MediaRepository.decodeImage (/usr/src/app/dist/repositories/media.repository.js:57:68)

    at MediaService.generateImageThumbnails (/usr/src/app/dist/services/media.service.js:153:63)

    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

    at async MediaService.handleGenerateThumbnails (/usr/src/app/dist/services/media.service.js:101:25)

    at async /usr/src/app/dist/services/job.service.js:163:36

    at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)

    at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)

[Nest] 7  - 10/15/2024, 4:33:48 PM   ERROR [Microservices:JobService] Object:

{

  "id": "e35ef6e8-4b48-4441-83b3-a36613c54e77"

}

Additional information

No response

[Blotz]

I'm also getting this error. Here are some sample images which cause the error.

images.zip

[marderbot]

Seems to happen with all or nearly all images received using Whatsapp, maybe because Whatsapp strips all metadata including huffman tables and the new library doesn't use a default one in that case?

[Blotz]

I dont think its all whatsapp images. I managed to upload some images without issue

[AFumi39]

I think this can be the reason. Even if some WA images are processed correctly

[marderbot]

you have the option to send HD images or no in whatsapp, maybe that affects the metadata they use?

[AFumi39]

I just checked, but no, that's not the case 🤔 2 similar images sent by the same person, in the same day: 1 ok, the other one no preview

[nesalkiN]

I see the same and the asset I looked at happened to be from Whatsapp.

[Nest] 7  - 10/15/2024, 4:25:08 PM   ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-thumbnails): Error: VipsJpeg: ./lib/jpegli/decode_marker.cc:285: DHT marker: no Huffman table found
[Nest] 7  - 10/15/2024, 4:25:08 PM   ERROR [Microservices:JobService] Error: VipsJpeg: ./lib/jpegli/decode_marker.cc:285: DHT marker: no Huffman table found
    at Sharp.toBuffer (/usr/src/app/node_modules/sharp/lib/output.js:163:17)
    at MediaRepository.decodeImage (/usr/src/app/dist/repositories/media.repository.js:57:68)
    at MediaService.generateImageThumbnails (/usr/src/app/dist/services/media.service.js:153:63)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async MediaService.handleGenerateThumbnails (/usr/src/app/dist/services/media.service.js:101:25)
    at async /usr/src/app/dist/services/job.service.js:163:36
    at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
    at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)

[mertalev]

This is technically out of spec for JPEG and the jpegli decoder doesn't handle it (unlike other decoders). I've raised the issue upstream and will proceed based on their response.

[Xsoft32]

Same issue here, all whatsapp images and some other ones are showing "error loading image" both in the thumbnail and in the actual preview, downloading the image or opening from the server volume work fine. Trying to reload the thumbnail gives the same log error.

[chriswa81]

Exactly the same for me!

[simoaica]

Same for me

[chriswa81]

Could the error be fixed? Is the bug fix included in the next update?

Thanks.

[sammyke007]

Ok, I was already wondering why my rerun of all thumbnails kept throwing errors and a lot of faces lost their thumbnail in the people section.

[mertalev]

I patched jpegli with a fix and tested with the sample images here. We'll make a hotfix for it.

[mertalev]

Should be fixed as of 1.118.2

[chriswa81]

I cannot confirm this. Some photos can now be loaded, but not all of them.

[mertalev]

You should run thumbnail generation in the Jobs page.

[chriswa81]

You should run thumbnail generation in the Jobs page.

Has already been carried out several times. Unfortunately without success!

[mertalev]

Can you share the logs?

[chriswa81]

Can you share the logs?

Here is my log: [Nest] 17 - 10/16/2024, 8:06:35 PM LOG [Api:EventRepository] Websocket Disconnect: ZEOICqCensSkOWO8AAAj [Nest] 17 - 10/16/2024, 8:09:39 PM LOG [Api:EventRepository] Websocket Connect: YfQyhfjuxdIjD131AAAl [Nest] 17 - 10/16/2024, 8:10:53 PM LOG [Api:EventRepository] Websocket Disconnect: YfQyhfjuxdIjD131AAAl [Nest] 17 - 10/16/2024, 8:10:56 PM LOG [Api:EventRepository] Websocket Connect: F3MSUJgizUQ_Nxh-AAAn [Nest] 17 - 10/16/2024, 8:12:09 PM LOG [Api:EventRepository] Websocket Disconnect: F3MSUJgizUQ_Nxh-AAAn [Nest] 17 - 10/16/2024, 8:16:06 PM LOG [Api:EventRepository] Websocket Connect: jzeQeltVe3ctPwD3AAAp [Nest] 17 - 10/16/2024, 8:24:59 PM LOG [Api:EventRepository] Websocket Disconnect: jzeQeltVe3ctPwD3AAAp [Nest] 7 - 10/16/2024, 9:07:53 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-thumbnails): Error: VipsJpeg: Message codes are not supported, error message is in msg_parm.s string [Nest] 7 - 10/16/2024, 9:07:53 PM ERROR [Microservices:JobService] Error: VipsJpeg: Message codes are not supported, error message is in msg_parm.s string at Sharp.toBuffer (/usr/src/app/node_modules/sharp/lib/output.js:163:17) at MediaRepository.decodeImage (/usr/src/app/dist/repositories/media.repository.js:57:68) at MediaService.generateImageThumbnails (/usr/src/app/dist/services/media.service.js:153:63) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async MediaService.handleGenerateThumbnails (/usr/src/app/dist/services/media.service.js:101:25) at async /usr/src/app/dist/services/job.service.js:163:36 at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28) at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24) [Nest] 7 - 10/16/2024, 9:07:53 PM ERROR [Microservices:JobService] Object: { "id": "d63c87fa-ca26-40a5-bb34-8aeb7cb41526" } [Nest] 7 - 10/16/2024, 9:07:53 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-thumbnails): Error: VipsJpeg: Message codes are not supported, error message is in msg_parm.s string [Nest] 7 - 10/16/2024, 9:07:53 PM ERROR [Microservices:JobService] Error: VipsJpeg: Message codes are not supported, error message is in msg_parm.s string at Sharp.toBuffer (/usr/src/app/node_modules/sharp/lib/output.js:163:17) at MediaRepository.decodeImage (/usr/src/app/dist/repositories/media.repository.js:57:68) at MediaService.generateImageThumbnails (/usr/src/app/dist/services/media.service.js:153:63) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async MediaService.handleGenerateThumbnails (/usr/src/app/dist/services/media.service.js:101:25) at async /usr/src/app/dist/services/job.service.js:163:36 at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28) at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24) [Nest] 7 - 10/16/2024, 9:07:53 PM ERROR [Microservices:JobService] Object: { "id": "ccbe3879-539c-4344-97e5-2daa3a9f351e" } [Nest] 7 - 10/16/2024, 9:07:53 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-thumbnails): Error: VipsJpeg: Message codes are not supported, error message is in msg_parm.s string [Nest] 7 - 10/16/2024, 9:07:53 PM ERROR [Microservices:JobService] Error: VipsJpeg: Message codes are not supported, error message is in msg_parm.s string at Sharp.toBuffer (/usr/src/app/node_modules/sharp/lib/output.js:163:17) at MediaRepository.decodeImage (/usr/src/app/dist/repositories/media.repository.js:57:68) at MediaService.generateImageThumbnails (/usr/src/app/dist/services/media.service.js:153:63) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async MediaService.handleGenerateThumbnails (/usr/src/app/dist/services/media.service.js:101:25) at async /usr/src/app/dist/services/job.service.js:163:36 at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28) at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24) [Nest] 7 - 10/16/2024, 9:07:53 PM ERROR [Microservices:JobService] Object: { "id": "b9cfd38d-8d54-44c3-86a9-4baeb79ec16e" } [Nest] 7 - 10/16/2024, 9:07:53 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-thumbnails): Error: VipsJpeg: Message codes are not supported, error message is in msg_parm.s string [Nest] 7 - 10/16/2024, 9:07:53 PM ERROR [Microservices:JobService] Error: VipsJpeg: Message codes are not supported, error message is in msg_parm.s string at Sharp.toBuffer (/usr/src/app/node_modules/sharp/lib/output.js:163:17) at MediaRepository.decodeImage (/usr/src/app/dist/repositories/media.repository.js:57:68) at MediaService.generateImageThumbnails (/usr/src/app/dist/services/media.service.js:153:63) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async MediaService.handleGenerateThumbnails (/usr/src/app/dist/services/media.service.js:101:25) at async /usr/src/app/dist/services/job.service.js:163:36 at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28) at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24) [Nest] 7 - 10/16/2024, 9:07:53 PM ERROR [Microservices:JobService] Object: { "id": "e747e00a-754f-404e-83f1-2fe97ec9f1a7" } [Nest] 7 - 10/16/2024, 9:07:53 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-thumbnails): Error: VipsJpeg: Message codes are not supported, error message is in msg_parm.s string [Nest] 7 - 10/16/2024, 9:07:53 PM ERROR [Microservices:JobService] Error: VipsJpeg: Message codes are not supported, error message is in msg_parm.s string at Sharp.toBuffer (/usr/src/app/node_modules/sharp/lib/output.js:163:17) at MediaRepository.decodeImage (/usr/src/app/dist/repositories/media.repository.js:57:68) at MediaService.generateImageThumbnails (/usr/src/app/dist/services/media.service.js:153:63) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async MediaService.handleGenerateThumbnails (/usr/src/app/dist/services/media.service.js:101:25) at async /usr/src/app/dist/services/job.service.js:163:36 at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28) at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24) [Nest] 7 - 10/16/2024, 9:07:53 PM ERROR [Microservices:JobService] Object: { "id": "73b77901-87ce-4d3a-8d94-bca45c2fa284" } [Nest] 17 - 10/16/2024, 11:31:57 PM LOG [Api:EventRepository] Websocket Connect: i-jEEen4sbM9itOFAAAr [Nest] 17 - 10/16/2024, 11:32:15 PM LOG [Api:EventRepository] Websocket Disconnect: i-jEEen4sbM9itOFAAAr [Nest] 17 - 10/16/2024, 11:33:54 PM LOG [Api:EventRepository] Websocket Connect: G4MabHPMcEgEuH5qAAAt [Nest] 7 - 10/16/2024, 11:34:03 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-thumbnails): Error: VipsJpeg: Message codes are not supported, error message is in msg_parm.s string [Nest] 7 - 10/16/2024, 11:34:03 PM ERROR [Microservices:JobService] Error: VipsJpeg: Message codes are not supported, error message is in msg_parm.s string at Sharp.toBuffer (/usr/src/app/node_modules/sharp/lib/output.js:163:17) at MediaRepository.decodeImage (/usr/src/app/dist/repositories/media.repository.js:57:68) at MediaService.generateImageThumbnails (/usr/src/app/dist/services/media.service.js:153:63) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async MediaService.handleGenerateThumbnails (/usr/src/app/dist/services/media.service.js:101:25) at async /usr/src/app/dist/services/job.service.js:163:36 at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28) at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24) [Nest] 7 - 10/16/2024, 11:34:03 PM ERROR [Microservices:JobService] Object: { "id": "d63c87fa-ca26-40a5-bb34-8aeb7cb41526" }

[chriswa81]

Here an example Image without thumbnail: example.zip

[mertalev]

Thanks for the sample, I'll take a look. But this is different than the Whatsapp issue, so I think you should make a separate issue for this.

[chriswa81]

I have opened a new error case for this: #13522

[AFumi39]

I can confirm that this bug is fixed for all my images @mertalev Thank you 🙇‍♂️

[sammyke007]

+1, fixed in v1.118.2 and rerun of the thumbnail job. Tnx!

[simoaica]

fixed in v1.118.2 and rerun of the thumbnail job. 👍🙏