NEF files only show a small thumbnail

[zynexiz]

The bug

Since v1.118.0 (I think) my NEF (Nikon RAW) files only displays a low resolution image, even when opening the picture itself. When right clicking on the image, and selecting "Open image in new tab", only the thumbnail is shown. Not the original picture (or in this case, the webp version). This worked in previous versions to my knowledge.

The OS that Immich Server is running on

Arch Linux

Version of Immich Server

v1.118.2

Version of Immich Mobile App

v1.118.0

Platform with the issue

• [X] Server
• [X] Web
• [ ] Mobile

Your docker-compose.yml content

N/A

Your .env content

N/A

Reproduction steps

1. Upload a NEF file
2. Wait for it to convert to WebP (my setting)
3. View the image in the web UI
4. Select "Open image in new tab"

Relevant log output

No relevant output.

Additional information

No response

[bo0tzz]

Do you have "prefer embedded preview" set in https://my.immich.app/admin/system-settings?isOpen=image+thumbnail-settings+preview-settings ?

[zynexiz]

No, I unchecked that one before because of this issue. Then I got it to work before.

[bo0tzz]

Can you try rerunning the thumbnail job for one of these assets?

[zynexiz]

Done that to, I even tried removing the image and reupload it. Same issue. It's a bit odd that it uses the thumbnail image for the "normal" sized image.

[mertalev]

Can you provide a zipped sample image?

[zynexiz]

Shared a couple of pictures to review here; https://zoik.se:10444/share/ho5jT8UYtJ1IeSjUZTbt6iyY0ZgqIA2jo2PqjkhxuOApCpB_l4h9ZBZ9TjPz5XNOfog

If you open the image from the interface, you only get a small thumbnail version.

[C-Otto]

On my instance those images show up just fine (I downloaded them from your site and uploaded them to mine).

[zynexiz]

On my instance those images show up just fine (I downloaded them from your site and uploaded them to mine).

That's odd, you use v1.118.0 to?

[zynexiz]

Found the issue, I needed to have prefer embedded preview enabled. BUT, I can't regenerate the thumbnails (that's why I didn't saw any difference with it on or off). It only works when I reuploaded the image, which is a huge hassle (have a lot of NEFs). Shouldn't regenerate thumbnails redo it over again?

[EDIT] I did a regeneration of all images in my catalogue, and it seemed to fix most of the images, Some still doesn't work though, and is till low res. Tried upload that image to my admin account, and then it got processed correctly.

[C-Otto]

I run v1.118.2 and I did NOT configure the "prefer embedded preview" thingy.

[zynexiz]

I run v1.118.2 and I did NOT configure the "prefer embedded preview" thingy.

Same as my settings before then. When I enabled prefer embedded preview, and rescaned my collection if fixed most of them (but not all). Now there's an issue with portrait mode that are displayed as landscape (and it messes with face recognition).

[EDIT] I tried disable prefer embedded preview, and uploaded two images to my admin account again. This time it was the same issue, so for me to get it to work I need to have prefer embedded preview enabled.

[zynexiz]

Realized that it was a browser caching issue that made some of the images not showing correctly. All is working now, but I have to have prefer embedded preview enabled, else it won't work :smile:

There's still two issues regarding NEF, one is the rotation (which mess with face recognition) and person thumbnail image. Have a couple of people only occurring on NEF images, and the thumbnail of that person don't work. It only works for people within JPEGs it seems.

okt 20 15:02:38 kvm-immich immich-server[312666]: [Nest] 312666  - 2024-10-20 15:02:38   ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-person-thumbnail): Error: extract_area: bad extract area
okt 20 15:02:38 kvm-immich immich-server[312666]: [Nest] 312666  - 2024-10-20 15:02:38   ERROR [Microservices:JobService] Error: extract_area: bad extract area
okt 20 15:02:38 kvm-immich immich-server[312666]:     at Sharp.toFile (/usr/lib/immich/app/server/node_modules/sharp/lib/output.js:90:19)
okt 20 15:02:38 kvm-immich immich-server[312666]:     at MediaRepository.generateThumbnail (/usr/lib/immich/app/server/dist/repositories/media.repository.js:65:14)
okt 20 15:02:38 kvm-immich immich-server[312666]:     at PersonService.handleGeneratePersonThumbnail (/usr/lib/immich/app/server/dist/services/person.service.js:431:36)
okt 20 15:02:38 kvm-immich immich-server[312666]:     at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
okt 20 15:02:38 kvm-immich immich-server[312666]:     at async /usr/lib/immich/app/server/dist/services/job.service.js:163:36
okt 20 15:02:38 kvm-immich immich-server[312666]:     at async Worker.processJob (/usr/lib/immich/app/server/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
okt 20 15:02:38 kvm-immich immich-server[312666]:     at async Worker.retryIfFailed (/usr/lib/immich/app/server/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
okt 20 15:02:38 kvm-immich immich-server[312666]: [Nest] 312666  - 2024-10-20 15:02:38   ERROR [Microservices:JobService] Object:
okt 20 15:02:38 kvm-immich immich-server[312666]: {
okt 20 15:02:38 kvm-immich immich-server[312666]:   "id": "e70ee5cf-b012-44c6-aad2-c94a8df48a33"
okt 20 15:02:38 kvm-immich immich-server[312666]: }

[C-Otto]

This might be related to #218. Could you provide a sample NEF file where the face detection is off?

[zynexiz]

Can try this one; https://zoik.se:10444/share/BZz4_PRrbIXoJSZjTh4Pv0lvYVx-0kpKxy-Y_IsMfkIs4SLXP7uXBM4Miv4nVA1XCXU

If I looking at the area, it fits with the position for a rotated image. The issue only occurs in images in portrait mode. If the image was displayed in portrait mode, the would be showing the correct area.

[zynexiz]

I have an idea of whats going on here. Previously the preview images that was shot in portrait mode, was generated with portrait (but low resolution) mode. Now that I enabled "prefer embedded preview", and regenerated all previews, all portrait shots was not rotated but generated in landscape mode. This means that the area is still from the old preview, thus the offsets are off.

It seems that not using "prefer embedded preview" adheres to the landscape/portrait mode, while generating from embedded preview does not. This is quite logical that the area gets thrown off when the image displayed is changed like this :smile: The face detection area is still from the old preview.

[C-Otto]

It's correct for me (and it also shows portrait, not landscape). The face detection algorithm uses the (Immich) preview image that is computed as part of the thumbnail job. I don't know what's going on on your system, but I assume that you have a bunch of outdated data. How painful would it be to re-compute all thumbnails?

[zynexiz]

It's correct for me (and it also shows portrait, not landscape). The face detection algorithm uses the (Immich) preview image that is computed as part of the thumbnail job. I don't know what's going on on your system, but I assume that you have a bunch of outdated data. How painful would it be to re-compute all thumbnails?

Did you use "prefer embedded preview" when trying? If I don't use it, the image is rotated correctly, but with a very low resolution. Using "prefer embedded preview" does not rotate it for me.

Not very painfill, did it when I enabled "prefer embedded preview". That's when I realized the rotation issue,

[C-Otto]

No, I don't use "prefer embedded preview". I also don't have the low resolution issue.

[zynexiz]

No, I don't use "prefer embedded preview". I also don't have the low resolution issue.

The rotation issue is related to "prefer embedded preview". It's a bit odd that the resolution works for you, but not here on the same version.

[zynexiz]

Just disabled prefer embedded preview and tried the image I shared. The rotation is correct, but as you can see the resolution is terrible.

[C-Otto]

Can you reproduce this, i.e. by uploading an image the server hasn't seen before, or deleting one (also from trash) before uploading it again?

[zynexiz]

Can you reproduce this, i.e. by uploading an image the server hasn't seen before, or deleting one (also from trash) before uploading it again?

I used the admin account (no images there) and it give the same result every time here.

[C-Otto]

Asset data should not be shared between accounts, but I'd rather play it safe: please try to find a new image according to my description above. For such a new image, which files do you see in the thumbs directory? If the asset id (shown in the URL) is xxyyzzz, you should find a directory thumbs/AAA/xx/yy with files xxyyzzz.* in it (where AAA is the user ID owning the image). Sorry, that's a bit complicated :)

[zynexiz]

I will pack up my gear and take a new image and try it :smile:

[zynexiz]

Same issue still, low resolution. I have two files in the thumbs directory;

-rw-rw---- 1 immich store 4060 20 okt 15.47 5a7181cc-8be8-4040-a190-943b5ff3fb51-preview.webp
-rw-rw---- 1 immich store 4060 20 okt 15.47 5a7181cc-8be8-4040-a190-943b5ff3fb51-thumbnail.webp

The have the same size, so I checked the files and they have identical hash sums.

[kvm-usr@kvm-immich 71]$ sha256sum 5a7181cc-8be8-4040-a190-943b5ff3fb51-preview.webp 
fe8bdca584740a775a5ef0af8cbf900d7f62955b46930b52bd8554e42bda5bb2  5a7181cc-8be8-4040-a190-943b5ff3fb51-preview.webp
[kvm-usr@kvm-immich 71]$ sha256sum 5a7181cc-8be8-4040-a190-943b5ff3fb51-thumbnail.webp 
fe8bdca584740a775a5ef0af8cbf900d7f62955b46930b52bd8554e42bda5bb2  5a7181cc-8be8-4040-a190-943b5ff3fb51-thumbnail.webp

[C-Otto]

4060 bytes is almost nothing. Could you provide those files (I guess one of them suffices)? What are your thumbnail/preview settings (in detail)?

[zynexiz]

This is the settings I'm using for the images.

Do you mean the thumbnail generated on the server? The 4k one?

[C-Otto]

I'd like to have a look at 5a7181cc-8be8-4040-a190-943b5ff3fb51-preview.webp, yes. I use the same settings, just 1440p instead of 720p "Upplösning" for "Förhandsvisningsinställningar". What a beautiful language :)

[zynexiz]

There is the file (new name due to new upload) with both preview and thumbnail file. Same sha256 sum on both files here to.

preview.zip

Thanks, it's Swedish :smile:

[C-Otto]

That's a 120x160 pixel preview, far from 720p. Weird.

[zynexiz]

That's a 120x160 pixel preview, far from 720p. Weird.

Yeh, very weird indeed. Even that the files are identical down to the last bit is quite odd to. Feels like it's just making a copy of it?

[C-Otto]

The preview/thumbnail images are generated using the sharp library (https://github.com/lovell/sharp). I don't see any recent code change in that area (a version update in May, though). Do you use an uncommon CPU architecture? Do you think your server setup is special in any way?

[zynexiz]

Nah, quite normal I would say. Run a HPE Microserver with Intel processor, although I use it as a Type 1 virtualization server. The Immich instance run in a KVM environment. Immich is installed using AUR (https://aur.archlinux.org/packages/immich-server), but the pull the image from the tar file here.

[C-Otto]

Would you be able to install a second clean Immich instance (possibly stopping the first in the meantime to avoid them using the same port) and try with several versions, starting from v1.116.0 or so? I'd help a lot to narrow down the time frame of when the regression happened. That's a lot of work, though.

[zynexiz]

Sure, I can try that. My afternoon is saved :laughing:

[C-Otto]

Thanks. Please make sure you have a current DB backup first.

[WangHaoyu0621]

It seems that I have the same problem when downloading .NEF through app The resolution of the downloaded file is too low

[C-Otto]

@WangHaoyu0621 this isn't about downloads at all. Please open a new issue.

[zynexiz]

Thanks. Please make sure you have a current DB backup first.

I did a clean start, so no need for backup :smile: Although, I have a hard time to get it running. I skiped the AUR package, getting an older release was a mess. But I can't get the docker image to start (not a pro in Docker tho). Seems like the immich_server just keeps restarting (the others ones are running). Don't get anything useful from docker compose logs either.

mmich_server            | [Nest] 17  - 10/20/2024, 6:56:36 PM     LOG [Api:EventRepository] Initialized websocket server
immich_server            | DEBUG: cgroup v2 detected.
immich_server            | DEBUG: No CPU limits set.
immich_server            | Detected CPU Cores: 4
immich_server            | Starting api worker
immich_server            | Starting microservices worker
immich_server            | [Nest] 7  - 10/20/2024, 6:56:44 PM     LOG [Microservices:EventRepository] Initialized websocket server
immich_server            | [Nest] 17  - 10/20/2024, 6:56:44 PM     LOG [Api:EventRepository] Initialized websocket server
immich_server            | DEBUG: cgroup v2 detected.
immich_server            | DEBUG: No CPU limits set.
immich_server            | Detected CPU Cores: 4
immich_server            | Starting api worker
immich_server            | Starting microservices worker
immich_server            | DEBUG: cgroup v2 detected.
immich_server            | DEBUG: No CPU limits set.
immich_server            | Detected CPU Cores: 4
immich_server            | Starting api worker
immich_server            | Starting microservices worker
immich_server            | [Nest] 7  - 10/20/2024, 6:57:02 PM     LOG [Microservices:EventRepository] Initialized websocket server
immich_server            | [Nest] 17  - 10/20/2024, 6:57:02 PM     LOG [Api:EventRepository] Initialized websocket server
immich_server            | [Nest] 7  - 10/20/2024, 6:57:08 PM     LOG [Microservices:SystemConfigService] LogLevel=verbose (set via IMMICH_LOG_LEVEL)
immich_server            | [Nest] 7  - 10/20/2024, 6:57:08 PM   DEBUG [Microservices:StorageTemplateService] Compiling new storage template: {{y}}/{{y}}-{{MM}}-{{dd}}/{{filename}}
immich_server            | DEBUG: cgroup v2 detected.
immich_server            | DEBUG: No CPU limits set.
immich_server            | Detected CPU Cores: 4
immich_server            | Starting api worker
immich_server            | Starting microservices worker
immich_server            | [Nest] 7  - 10/20/2024, 6:57:15 PM     LOG [Microservices:EventRepository] Initialized websocket server
immich_server            | [Nest] 7  - 10/20/2024, 6:57:15 PM     LOG [Microservices:SystemConfigService] LogLevel=verbose (set via IMMICH_LOG_LEVEL)
immich_server            | [Nest] 7  - 10/20/2024, 6:57:15 PM   DEBUG [Microservices:StorageTemplateService] Compiling new storage template: {{y}}/{{y}}-{{MM}}-{{dd}}/{{filename}}
immich_server            | [Nest] 17  - 10/20/2024, 6:57:16 PM     LOG [Api:EventRepository] Initialized websocket server
immich_server            | DEBUG: cgroup v2 detected.
immich_server            | DEBUG: No CPU limits set.
immich_server            | Detected CPU Cores: 4
immich_server            | Starting api worker
immich_server            | Starting microservices worker
immich_server            | DEBUG: cgroup v2 detected.
immich_server            | DEBUG: No CPU limits set.
immich_server            | Detected CPU Cores: 4
immich_server            | Starting api worker
immich_server            | Starting microservices worker
immich_server            | [Nest] 6  - 10/20/2024, 6:57:29 PM     LOG [Microservices:EventRepository] Initialized websocket server
immich_server            | [Nest] 6  - 10/20/2024, 6:57:30 PM     LOG [Microservices:SystemConfigService] LogLevel=verbose (set via IMMICH_LOG_LEVEL)
immich_server            | [Nest] 6  - 10/20/2024, 6:57:30 PM   DEBUG [Microservices:StorageTemplateService] Compiling new storage template: {{y}}/{{y}}-{{MM}}-{{dd}}/{{filename}}
immich_server            | [Nest] 16  - 10/20/2024, 6:57:30 PM     LOG [Api:EventRepository] Initialized websocket server
immich_server            | DEBUG: cgroup v2 detected.
immich_server            | DEBUG: No CPU limits set.
immich_server            | Detected CPU Cores: 4
immich_server            | Starting api worker
immich_server            | Starting microservices worker
immich_server            | DEBUG: cgroup v2 detected.
immich_server            | DEBUG: No CPU limits set.
immich_server            | Detected CPU Cores: 4
immich_server            | Starting api worker
immich_server            | Starting microservices worker
immich_server            | [Nest] 6  - 10/20/2024, 6:57:49 PM     LOG [Microservices:EventRepository] Initialized websocket server
immich_server            | DEBUG: cgroup v2 detected.
immich_server            | DEBUG: No CPU limits set.
immich_server            | Detected CPU Cores: 4
immich_server            | Starting api worker
immich_server            | Starting microservices worker
immich_server            | [Nest] 7  - 10/20/2024, 6:57:59 PM     LOG [Microservices:EventRepository] Initialized websocket server
immich_server            | DEBUG: cgroup v2 detected.
immich_server            | DEBUG: No CPU limits set.
immich_server            | Detected CPU Cores: 4
immich_server            | Starting api worker
immich_server            | Starting microservices worker
immich_server            | DEBUG: cgroup v2 detected.
immich_server            | DEBUG: No CPU limits set.
immich_server            | Detected CPU Cores: 4
immich_server            | Starting api worker
immich_server            | Starting microservices worker
immich_server            | [Nest] 7  - 10/20/2024, 6:58:32 PM     LOG [Microservices:EventRepository] Initialized websocket server
immich_server            | DEBUG: cgroup v2 detected.
immich_server            | DEBUG: No CPU limits set.
immich_server            | Detected CPU Cores: 4
immich_server            | Starting api worker
immich_server            | Starting microservices worker
immich_server            | DEBUG: cgroup v2 detected.
immich_server            | DEBUG: No CPU limits set.
immich_server            | Detected CPU Cores: 4
immich_server            | Starting api worker
immich_server            | Starting microservices worker

Followed the guide for Docker installation, used the docker-compose.yml and the .env file (just added IMMICH_VERSION=v1.116.0 and IMMICH_LOG_LEVEL=verbose to it).

Any idea?

[C-Otto]

No, sorry.

[zynexiz]

Tried Docker installation with both newest and v1.116.0, same issue with both of them for some reason.