immich-app / immich

High performance self-hosted photo and video management solution.
https://immich.app
GNU Affero General Public License v3.0
49.83k stars 2.64k forks source link

Useless downstream jobs triggered when video preview image cannot be generated #12053

Open C-Otto opened 2 months ago

C-Otto commented 2 months ago

The bug

My library contains two videos for which no preview can be generated (see #11133). Because the preview images are missing for those two videos, the repair page shows two offline paths (/usr/src/app/upload/thumbs/[...]-preview.jpeg). I'm not sure if this is intended, though.

More importantly, the preview image is used for further jobs, one of which creating the thumbnail image (/usr/src/app/upload/thumbs/[...]-thumbnail.webp). Even though the preview image does not exist, the job trying to create the thumbnail image is run. This job fails, leaving a nasty error in the logs and an empty file on disk. This empty file shows up as an "untracked file" on the repair page.

According to my logs the "thumbhash" job also failed. I don't know, but assume, that other jobs (metadata, machine learning, ...) might also have failed.

If, for whatever reason, the preview image cannot be created, no downstream job should be triggered.

The OS that Immich Server is running on

Debian Stable

Version of Immich Server

v1.112.1

Platform with the issue

Reproduction steps

  1. Have video for which no preview can be generated (I can provide one upon request)
  2. Check "Repair" page and server logs

Relevant log output

[Nest] 6  - 08/26/2024, 1:52:45 PM   ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-thumbhash): Error: Input file is missing: upload/thumbs/[...]-preview.jpeg
[Nest] 6  - 08/26/2024, 1:52:45 PM   ERROR [Microservices:JobService] Error: Input file is missing: upload/thumbs/[...]-preview.jpeg
    at Sharp.toBuffer (/usr/src/app/node_modules/sharp/lib/output.js:161:17)
    at MediaRepository.generateThumbhash (/usr/src/app/dist/repositories/media.repository.js:141:14)
    at MediaService.handleGenerateThumbhash (/usr/src/app/dist/services/media.service.js:223:54)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async /usr/src/app/dist/services/job.service.js:148:36
    at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
    at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 6  - 08/26/2024, 1:52:45 PM   ERROR [Microservices:JobService] Object:
{
  "id": "f8fee794-d90e-4bb5-b93a-0cc64e49f288"
}

[Nest] 6  - 08/26/2024, 1:52:46 PM   ERROR [Microservices:MediaRepository] ffmpeg version 6.0.1-Jellyfin Copyright (c) 2000-2023 the FFmpeg developers
  built with gcc 12 (Debian 12.2.0-14)
  configuration: --prefix=/usr/lib/jellyfin-ffmpeg --target-os=linux --extra-version=Jellyfin --disable-doc --disable-ffplay --disable-ptx-compression --disable-static --disable-libxcb --disable-sdl2 --disable-xlib --enable-lto --enable-gpl --enable-version3 --enable-shared --enable-gmp --enable-gnutls --enable-chromaprint --enable-opencl --enable-libdrm --enable-libass --enable-libfreetype --enable-libfribidi --enable-libfontconfig --enable-libbluray --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libopenmpt --enable-libdav1d --enable-libsvtav1 --enable-libwebp --enable-libvpx --enable-libx264 --enable-libx265 --enable-libzvbi --enable-libzimg --enable-libfdk-aac --arch=arm64 --cross-prefix=/usr/bin/aarch64-linux-gnu- --toolchain=hardened --enable-cross-compile --enable-rkmpp --enable-rkrga
  libavutil      58.  2.100 / 58.  2.100
  libavcodec     60.  3.100 / 60.  3.100
  libavformat    60.  3.100 / 60.  3.100
  libavdevice    60.  1.100 / 60.  1.100
  libavfilter     9.  3.100 /  9.  3.100
C-Otto commented 2 months ago

Note that the job shows a success log entry despite ffmpeg failing with an error:

https://github.com/immich-app/immich/blob/fe672d4f35d1899ee09d58f773cc1f2b46af5d36/server/src/services/media.service.ts#L250-L260

LOG [Microservices:MediaService] Successfully generated JPEG video preview for asset [...]
ERROR [Microservices:MediaRepository] ffmpeg version 6.0.1-Jellyfin Copyright (c) 2000-2023 the FFmpeg developers
  built with gcc 12 (Debian 12.2.0-14)
  configuration: --prefix=/usr/lib/jellyfin-ffmpeg --target-os=linux --extra-version=Jellyfin --disable-doc --disable-ffplay --disable-ptx-compression --disable-static --disable-libxcb --disable-sdl2 --disable-xlib --enable-lto --enable-gpl --enable-version3 --enable-shared --enable-gmp --enable-gnutls --enable-chromaprint --enable-opencl --enable-libdrm --enable-libass --enable-libfreetype --enable-libfribidi --enable-libfontconfig --enable-libbluray --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libopenmpt --enable-libdav1d --enable-libsvtav1 --enable-libwebp --enable-libvpx --enable-libx264 --enable-libx265 --enable-libzvbi --enable-libzimg --enable-libfdk-aac --arch=arm64 --cross-prefix=/usr/bin/aarch64-linux-gnu- --toolchain=hardened --enable-cross-compile --enable-rkmpp --enable-rkrga
  libavutil      58.  2.100 / 58.  2.100
  libavcodec     60.  3.100 / 60.  3.100
  libavformat    60.  3.100 / 60.  3.100
  libavdevice    60.  1.100 / 60.  1.100
  libavfilter     9.  3.100 /  9.  3.100
  libswscale      7.  1.100 /  7.  1.100
  libswresample   4. 10.100 /  4. 10.100
  libpostproc    57.  1.100 / 57.  1.100
[h264 @ 0x4dd261c0180] Reinit context to 1920x1088, pix_fmt: yuv420p
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/home/user/pictures/[...].mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: isommp42
    creation_time   : 2018-09-14T08:35:50.000000Z
  Duration: 00:00:06.04, start: 0.000000, bitrate: 4434 kb/s
  Stream #0:0[0x1](und): Video: h264 (High), 1 reference frame (avc1 / 0x31637661), yuv420p(tv, bt709/bt709/smpte170m, progressive, left), 1920x1080 (1920x1088) [SAR 1:1 DAR 16:9], 4339 kb/s, 29.90 fps, 29.92 tbr, 90k tbn (default)
    Metadata:
      creation_time   : 2018-09-14T08:35:50.000000Z
      handler_name    : ISO Media file produced by Google Inc. Created on: 09/14/2018.
      vendor_id       : [0][0][0][0]
  Stream #0:1[0x2](eng): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 125 kb/s (default)
    Metadata:
      creation_time   : 2018-09-14T08:35:50.000000Z
      handler_name    : ISO Media file produced by Google Inc. Created on: 09/14/2018.
      vendor_id       : [0][0][0][0]
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> webp (libwebp_anim))
Press [q] to stop, [?] for help
[h264 @ 0x4dd261c1580] Reinit context to 1920x1088, pix_fmt: yuv420p
[Parsed_thumbnail_1 @ 0x4dd26100540] batch size: 12 frames
[Parsed_scale_5 @ 0x4dd26100480] w:-2 h:250 flags:'lanczos+accurate_rnd+full_chroma_int' interl:0
[graph 0 input from stream 0:0 @ 0x4dd26101140] w:1920 h:1080 pixfmt:yuv420p tb:1/90000 fr:359/12 sar:1/1
[Parsed_fps_0 @ 0x4dd26101080] fps=12/1
[Parsed_scale_5 @ 0x4dd26100480] w:1920 h:1080 fmt:yuv420p sar:1/1 -> w:444 h:250 fmt:yuv420p sar:1000/999 flags:0x00042200
EOF in input file 0
Terminating demuxer thread 0
[Parsed_fps_0 @ 0x4dd26101080] Set first pts to 63
Output #0, webp, to 'upload/thumbs/[...]-thumbnail.webp':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: isommp42
    encoder         : Lavf60.3.100
  Stream #0:0(und): Video: webp, 1 reference frame, yuv420p, 444x250 (0x0) [SAR 1000:999 DAR 16:9], q=2-31, 200 kb/s, 12 fps, 1k tbn (default)
    Metadata:
      creation_time   : 2018-09-14T08:35:50.000000Z
      handler_name    : ISO Media file produced by Google Inc. Created on: 09/14/2018.
      vendor_id       : [0][0][0][0]
      encoder         : Lavc60.3.100 libwebp_anim
frame=    0 fps=0.0 q=0.0 size=       0kB time=-577014:32:22.77 bitrate=  -0.0kbits/s speed=N/A    
No more output streams to write to, finishing.
[libwebp encoder @ 0x4dd26110780] WebPAnimEncoderAssemble() failed with error: 1
[vost#0:0/libwebp_anim @ 0x4dd260d2b80] Error submitting video frame to the encoder
[Parsed_fps_0 @ 0x4dd26101080] 2 frames in, 0 frames out; 2 frames dropped, 0 frames duplicated.
[out#0/webp @ 0x4dd261009c0] All streams finished
[out#0/webp @ 0x4dd261009c0] Terminating muxer thread
[AVIOContext @ 0x4dd261a0540] Statistics: 0 bytes written, 0 seeks, 0 writeouts
[AVIOContext @ 0x4dd261a0180] Statistics: 3346283 bytes read, 0 seeks
Conversion failed!