search

immich-app / immich

High performance self-hosted photo and video management solution.
https://immich.app
GNU Affero General Public License v3.0
44.76k stars 2.17k forks source link

import HEIF photo failure on 1.109.x immich server #11247

Open LoadingZhang opened 1 month ago

LoadingZhang commented 1 month ago

i got error after update from 108 -> 109 in website, rollback 108 then get back to normal.

[Nest] 7  - 07/20/2024, 10:08:29 PM   ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-preview): Error: Input file has corrupt header: magickload: Magick: UnableToLoadModule '/usr/local/lib/ImageMagick-7.1.1/modules-Q16HDRI/coders/heic.la': file not found @ error/module.c/OpenModule/1293 (null)
[Nest] 7  - 07/20/2024, 10:08:29 PM   ERROR [Microservices:JobService] Error: Input file has corrupt header: magickload: Magick: UnableToLoadModule '/usr/local/lib/ImageMagick-7.1.1/modules-Q16HDRI/coders/heic.la': file not found @ error/module.c/OpenModule/1293 (null)
    at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
    at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
    at MediaService.generateThumbnail (/usr/src/app/dist/services/media.service.js:164:48)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async MediaService.handleGeneratePreview (/usr/src/app/dist/services/media.service.js:135:29)
    at async /usr/src/app/dist/services/job.service.js:148:36
    at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
    at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7  - 07/20/2024, 10:08:29 PM   ERROR [Microservices:JobService] Object:
{
  "id": "6797f51d-30bc-4d82-b950-463f62729377",
  "source": "upload"
}

file: IMG_20240720_230151.zip

The OS that Immich Server is running on

Debian WSL2

Version of Immich Server

v1.109.x

Version of Immich Mobile App

null

Platform with the issue

Your docker-compose.yml content

immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    extends:
      file: hwaccel.transcoding.yml
      service: vaapi-wsl # set to one of [nvenc, quicksync, rkmpp, vaapi, vaapi-wsl] for accelerated transcoding
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /etc/timezone:/etc/timezone:ro
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
    devices: ["/dev/dri"]
    env_file: [".env"]
    ports: ["2283:3001"]
    depends_on:
      - redis
      - database
    restart: on-failure

Your .env content

Upload again if you need to

Reproduction steps

upload HEIF file from web or app

Relevant log output

No response

Additional information

No response

ebrearley commented 1 month ago

I think I'm experiencing the same problem, but going back to 108 doesn't resolve the problem for me. For each upload, I see an error like this:


[Nest] 8  - 07/24/2024, 6:23:53 AM   ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-preview): Error: Input file has corrupt header: magickload: Magick: UnableToLoadModule '/usr/local/lib/ImageMagick-7.1.1/modules-Q16HDRI/coders/heic.la': file not found @ error/module.c/OpenModule/1293 (null)
[Nest] 8  - 07/24/2024, 6:23:53 AM   ERROR [Microservices:JobService] Error: Input file has corrupt header: magickload: Magick: UnableToLoadModule '/usr/local/lib/ImageMagick-7.1.1/modules-Q16HDRI/coders/heic.la': file not found @ error/module.c/OpenModule/1293 (null)
    at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
    at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
    at MediaService.generateThumbnail (/usr/src/app/dist/services/media.service.js:164:48)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async MediaService.handleGeneratePreview (/usr/src/app/dist/services/media.service.js:135:29)
    at async /usr/src/app/dist/services/job.service.js:148:36
    at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
    at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 8  - 07/24/2024, 6:23:53 AM   ERROR [Microservices:JobService] Object:
{
  "id": "e097f3c9-a6a4-48c1-9fb6-a273599be169"
}
zprde commented 1 month ago

I met the same problem with the same config and version.

alextran1502 commented 1 month ago

If you guys temporarily disable hardware acceleration, does it work normally?

zprde commented 1 month ago

If you guys temporarily disable hardware acceleration, does it work normally?

Yes, it works for me by remobving the hardware acceleration.

w00tlarr commented 1 month ago

I met the same problem with the same config and version.

I'm also hit with this issue :-( Removed all my HW acceleration config in the immich-server docker-compose and that worked.

mertalev commented 1 month ago

The vaapi-wsl config sets LD_LIBRARY_PATH=/usr/lib/wsl/lib, so this is probably messing with other libraries.

dodg3r commented 1 month ago

If you guys temporarily disable hardware acceleration, does it work normally?

Hi.

I´m facing the same problem but I´m not using HW acceleration. Do you want more info from me or do you already have a solution :)

I´m running v1.110.0

alextran1502 commented 1 month ago

@dodg3r if you can include a sample file, that would be great

dodg3r commented 1 month ago

It seems that about 20 of my uploaded files are corrupt. I need to investigate where in the process they became corrupt.

w00tlarr commented 1 month ago

I met the same problem with the same config and version.

I'm also hit with this issue :-( Removed all my HW acceleration config in the immich-server docker-compose and that worked.

I fixed my issue. My docker compose has this: environment:

Works now after removing that.

LoadingZhang commented 1 month ago

If you guys temporarily disable hardware acceleration, does it work normally?

remove vaapi in docker compose worked.

jakub-antos commented 1 month ago

1.111.0 with iOS 18 beta 4 and latest app.

No thumbnails no image in Web (HEIC only)

2024-08-05 14:58:27 [Nest] 7 - 08/05/2024, 12:58:27 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-preview): Error: Input file has corrupt header: magickload: Magick: UnableToLoadModule '/usr/local/lib/ImageMagick-7.1.1/modules-Q16HDRI/coders/heic.la': file not found @ error/module.c/OpenModule/1293 (null) 2024-08-05 14:58:27 [Nest] 7 - 08/05/2024, 12:58:27 PM ERROR [Microservices:JobService] Error: Input file has corrupt header: magickload: Magick: UnableToLoadModule '/usr/local/lib/ImageMagick-7.1.1/modules-Q16HDRI/coders/heic.la': file not found @ error/module.c/OpenModule/1293 (null)

alextran1502 commented 1 month ago

@jakub-antos are you using any hardware acceleration? Look like they are not compatible with the newer version of HEIC

jakub-antos commented 1 month ago

@alextran1502 No im not using HW acceleration, pure CPU AMD Ryzen Embedded R1505G, disabled HW Accel in Transcoding, no additional options in

immich-microservices: container_name: immich_microservices image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release} extends: file: hwaccel.yml service: hwaccel command: [ "start.sh", "microservices" ] volumes:

alextran1502 commented 1 month ago

@jakub-antos your docker compose file is out of date, there is no microservices anymore. I assume any photo you upload will not have thumbnail generated, not just HEIC

jakub-antos commented 1 month ago

@alextran1502 Thank you so much! It works perfectly, and everything is back to normal!

akropp commented 3 weeks ago

Why is this closed? It still doesn't work with hardware acceleration. Shouldn't the solution be to fix the path issue?

mertalev commented 3 weeks ago

You're right, the issue is still present. I think changing the LD_LIBRARY_PATH env in the vaapi-wsl config to /usr/lib/jellyfin-ffmpeg/lib:/usr/lib/wsl/lib should fix it.

akropp commented 3 weeks ago

Yes, that did it!

--Adam

On Wed, Aug 21, 2024 at 9:47 PM Mert @.***> wrote:

You're right, the issue is still present. I think changing the LD_LIBRARY_PATH env in the vaapi-wsl config to /usr/lib/jellyfin-ffmpeg/lib:/usr/lib/wsl/lib should fix it.

— Reply to this email directly, view it on GitHub https://github.com/immich-app/immich/issues/11247#issuecomment-2303503810, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHIXJVJ4RDTLAYHIA7KKVCLZSU7LDAVCNFSM6AAAAABLGB7LV2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMBTGUYDGOBRGA . You are receiving this because you commented.Message ID: @.***>

NicholasFlamy commented 2 weeks ago

You're right, the issue is still present. I think changing the LD_LIBRARY_PATH env in the vaapi-wsl config to /usr/lib/jellyfin-ffmpeg/lib:/usr/lib/wsl/lib should fix it.

Should this be PRed to the file provided in the repo?

(I confirmed this issue on my system.)

mertalev commented 2 weeks ago

That would be good, yes. Better would be to add the env to the image directly here and remove that env from the vaapi-wsl config. It'll avoid this happening again if the expected LD_LIBRARY_PATH changes.

NicholasFlamy commented 2 weeks ago

If that's better then would you be able to PR that? I'm not certain I know what to do there.