Certain DNG files cannot be processed

[KingOfDog]

The bug

Certain DNG files of mine can't be processed by Immich. They only show up as broken images, while the metadata is correctly displayed. It appears to mostly occur with DNG files that were created by Lightroom, e.g. through HDR or Panorama stitching, of ARW raw photos from my Sony a7 IV camera. See the example image here.

I've tracked down the issue to the libraw library not being able to parse the DNG files. If I build libraw from source with the Adobe DNG SDK linked, on the other hand, it is actually able to read out details from the file. raw-identify then says that the file is a Sony ILCE-7M4 image.

Would it be possible to include the Adobe DNG SDK in Immich server in any way? Or is there an easier way to get those files to show up?

The OS that Immich Server is running on

Unraid 6.12.10

Version of Immich Server

1.116.2

Version of Immich Mobile App

1.116.1 build.161

Platform with the issue

• [X] Server
• [ ] Web
• [ ] Mobile

Your docker-compose.yml content

name: immich

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    # extends:
    #   file: hwaccel.transcoding.yml
    #   service: cpu # set to one of [nvenc, quicksync, rkmpp, vaapi, vaapi-wsl] for accelerated transcoding
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /mnt/user/KingOfDog/Photos/:/mnt/photos
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env
    ports:
      - 2283:3001
    depends_on:
      - redis
      - database
    restart: always

  immich-machine-learning:
    container_name: immich_machine_learning
    # For hardware acceleration, add one of -[armnn, cuda, openvino] to the image tag.
    # Example tag: ${IMMICH_VERSION:-release}-cuda
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
    # extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/ml-hardware-acceleration
    #   file: hwaccel.ml.yml
    #   service: cpu # set to one of [armnn, cuda, openvino, openvino-wsl] for accelerated inference - use the `-wsl` version for WSL2 where applicable
    volumes:
      - model-cache:/cache
    env_file:
      - .env
    restart: always

  redis:
    container_name: immich_redis
    image: docker.io/redis:6.2-alpine@sha256:d6c2911ac51b289db208767581a5d154544f2b2fe4914ea5056443f62dc6e900
    healthcheck:
      test: redis-cli ping || exit 1
    restart: always

  database:
    container_name: immich_postgres
    image: docker.io/tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
      POSTGRES_INITDB_ARGS: '--data-checksums'
    volumes:
      - ${DB_DATA_LOCATION}:/var/lib/postgresql/data
    healthcheck:
      test: pg_isready --dbname='${DB_DATABASE_NAME}' || exit 1; Chksum="$$(psql --dbname='${DB_DATABASE_NAME}' --username='${DB_USERNAME}' --tuples-only --no-align --command='SELECT SUM(checksum_failures) FROM pg_stat_database')"; echo "checksum failure count is $$Chksum"; [ "$$Chksum" = '0' ] || exit 1
      interval: 5m
      # start_interval: 30s
      start_period: 5m
    command: ["postgres", "-c" ,"shared_preload_libraries=vectors.so", "-c", 'search_path="$$user", public, vectors', "-c", "logging_collector=on", "-c", "max_wal_size=2GB", "-c", "shared_buffers=512MB", "-c", "wal_compression=on"]
    restart: always

volumes:
  model-cache:

Your .env content

# The location where your uploaded files are stored
UPLOAD_LOCATION=/mnt/user/immich/upload
# The location where your database files are stored
DB_DATA_LOCATION=/mnt/user/immich/database

# To set a timezone, uncomment the next line and change Etc/UTC to a TZ identifier from this list: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List
# TZ=Etc/UTC

# The Immich version to use. You can pin this to a specific version like "v1.71.0"
IMMICH_VERSION=release

# Connection secret for postgres. You should change it to a random password
DB_PASSWORD=*****

# The values below this line do not need to be changed
###################################################################################
DB_USERNAME=postgres
DB_DATABASE_NAME=immich

Reproduction steps

1. Upload the file to Immich
2. The image won't show up

Relevant log output

[Nest] 7  - 09/29/2024, 10:32:17 AM   ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-preview): Error: Input file has corrupt header: magickload: Magick: Unsupported file format or not RAW file `/mnt/photos/2024/2024-09/2024-09-19_Rückfahrt nach Trondheim/Sony a7 IV Photo/_A7F9420-HDR.dng' @ error/dng.c/ReadDNGImage/504 (null)
[Nest] 7  - 09/29/2024, 10:32:17 AM   ERROR [Microservices:JobService] Error: Input file has corrupt header: magickload: Magick: Unsupported file format or not RAW file `/mnt/photos/2024/2024-09/2024-09-19_Rückfahrt nach Trondheim/Sony a7 IV Photo/_A7F9420-HDR.dng' @ error/dng.c/ReadDNGImage/504 (null)
    at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:90:19)
    at MediaRepository.generateThumbnail (/usr/src/app/dist/repositories/media.repository.js:69:14)
    at MediaService.generateThumbnail (/usr/src/app/dist/services/media.service.js:174:48)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async MediaService.handleGeneratePreview (/usr/src/app/dist/services/media.service.js:139:29)
    at async /usr/src/app/dist/services/job.service.js:171:36
    at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
    at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7  - 09/29/2024, 10:32:17 AM   ERROR [Microservices:JobService] Object:
{
  "id": "4c24f2af-dab2-4634-8236-2a46c01716c6"
}

Additional information

No response

[alexisharara]

Same issue with Apple now supporting JPEG-XL format on iOS 18 which stores them in a .DNG container

[mertalev]

Would it be possible to include the Adobe DNG SDK in Immich server in any way?

We build libraw ourselves, so this is possible. IIRC DNG SDK didn't have good support for Linux (only Windows and Mac), so it's a bit of a pain to get it working. A PR for the base image would be welcome if you're able to make it work in Linux.

[MikevE1995]

Same issue on Galaxy S24 Ultra and Windows pc

[thomashollier]

Same here, using v1.117.

[devtechk]

Hopefullly I'm not alone... some dng are displayed well other not... like yours. I hope upgrade will adjust it

[andresledo]

Same issue with Galaxy S23 Ultra images from expert raw

[jedie]

Same here with v1.119.1 and DNG from different camery types: Pentax K-1, Pentax K-5 II, Pentax K-01 and more ... All of them display only the very low res preview thumbnail :(

[mertalev]

There's an option in the Image Settings to use the embedded preview where possible. The result should be better if you enable that, re-run thumbnail generation on all assets and clear browser cache once it's done.

[MikevE1995]

This fixed it for me, thanks!

[jedie]

There's an option in the Image Settings to use the embedded preview where possible. The result should be better if you enable that, re-run thumbnail generation on all assets and clear browser cache once it's done.

But this fix only low res thumbnails... What i mean in https://github.com/immich-app/immich/issues/13029#issuecomment-2465799903 above: In detail view of one picture is also only the low res thumbnail used! This will be not fixed, isn't it?