search

immich-app / immich

High performance self-hosted photo and video management solution.
https://immich.app
GNU Affero General Public License v3.0
45.19k stars 2.19k forks source link

[BUG] Error importing geodata when running container with a non-root user #5404

Closed amitrea closed 9 months ago

amitrea commented 9 months ago

The bug

After running the latest version of Immich (v1.89.0) and inspecting the logs of container immich-microservices I get the below error. I usually run containers with a non root user (uid:guid 1000:1000) for security reasons. This error occurred only with this release.

❯ podman logs -f immich-microservices
[Nest] 3  - 11/30/2023, 9:00:13 AM     LOG [NestFactory] Starting Nest application...
[Nest] 3  - 11/30/2023, 9:00:13 AM     LOG [InstanceLoader] TypeOrmModule dependencies initialized +44ms
[Nest] 3  - 11/30/2023, 9:00:13 AM     LOG [InstanceLoader] BullModule dependencies initialized +0ms
[Nest] 3  - 11/30/2023, 9:00:13 AM     LOG [InstanceLoader] ConfigHostModule dependencies initialized +1ms
[Nest] 3  - 11/30/2023, 9:00:13 AM     LOG [InstanceLoader] DiscoveryModule dependencies initialized +0ms
[Nest] 3  - 11/30/2023, 9:00:13 AM     LOG [InstanceLoader] ScheduleModule dependencies initialized +1ms
[Nest] 3  - 11/30/2023, 9:00:13 AM     LOG [InstanceLoader] ConfigModule dependencies initialized +5ms
[Nest] 3  - 11/30/2023, 9:00:13 AM     LOG [InstanceLoader] BullModule dependencies initialized +0ms
[Nest] 3  - 11/30/2023, 9:00:13 AM     LOG [InstanceLoader] BullModule dependencies initialized +1ms
[Nest] 3  - 11/30/2023, 9:00:13 AM     LOG [InstanceLoader] TypeOrmCoreModule dependencies initialized +262ms
[Nest] 3  - 11/30/2023, 9:00:13 AM     LOG [InstanceLoader] TypeOrmModule dependencies initialized +1ms
[Nest] 3  - 11/30/2023, 9:00:13 AM     LOG [InstanceLoader] InfraModule dependencies initialized +5ms
[Nest] 3  - 11/30/2023, 9:00:13 AM     LOG [InstanceLoader] DomainModule dependencies initialized +34ms
[Nest] 3  - 11/30/2023, 9:00:13 AM     LOG [InstanceLoader] MicroservicesModule dependencies initialized +0ms
[Nest] 3  - 11/30/2023, 9:00:13 AM     LOG [MetadataRepository] Initializing metadata repository
[Nest] 3  - 11/30/2023, 9:00:13 AM     LOG [MetadataRepository] Importing geodata to database from file
[Nest] 3  - 11/30/2023, 9:00:54 AM   FATAL [MetadataRepository] Error importing geodata
[Nest] 3  - 11/30/2023, 9:00:54 AM   FATAL [MetadataRepository] Error: EACCES: permission denied, open '/usr/src/resources/admin1CodesASCII.txt'
[Nest] 3  - 11/30/2023, 9:00:54 AM   ERROR [MetadataService] Unable to initialize reverse geocoding: Error: EACCES: permission denied, open '/usr/src/resources/admin1CodesASCII.txt'
Error: EACCES: permission denied, open '/usr/src/resources/admin1CodesASCII.txt'
[Nest] 3  - 11/30/2023, 9:00:55 AM     LOG [SearchService] Running bootstrap
[Nest] 3  - 11/30/2023, 9:00:55 AM     LOG [TypesenseRepository] Schema up to date: assets/assets-v10
[Nest] 3  - 11/30/2023, 9:00:55 AM     LOG [TypesenseRepository] Schema up to date: albums/albums-v2
[Nest] 3  - 11/30/2023, 9:00:55 AM     LOG [TypesenseRepository] Schema up to date: faces/faces-v1
[Nest] 3  - 11/30/2023, 9:00:55 AM     LOG [TypesenseRepository] Alias mapping: [{"collection_name":"faces-v1","name":"faces"},{"collection_name":"albums-v2","name":"albums"},{"collection_name":"assets-v10","name":"assets"}]
[Nest] 3  - 11/30/2023, 9:00:55 AM     LOG [TypesenseRepository] Collections needing migration: {"assets":false,"albums":false,"faces":false}
[Nest] 3  - 11/30/2023, 9:00:55 AM     LOG [NestApplication] Nest application successfully started +16ms
[Nest] 3  - 11/30/2023, 9:00:55 AM     LOG [ImmichMicroservice] Immich Microservices is listening on http://[::1]:3002 [v1.89.0] [PRODUCTION]

The OS that Immich Server is running on

Fedora 38

Version of Immich Server

v1.89.0

Version of Immich Mobile App

*

Platform with the issue

Your docker-compose.yml content

...

  immich-microservices:
    image: ghcr.io/immich-app/immich-server:release
    container_name: immich-microservices
    command: ["./start-microservices.sh"]
    restart: unless-stopped
    user: 1000:1000
    group_add:
    - "105"
    env_file:
      - immich.env
    devices:
      - /dev/dri:/dev/dri:rw
    volumes:
      # - /data/immich/geocoding:/usr/src/app/.reverse-geocoding-dump:rw,z
      - /data/immich/upload:/usr/src/app/upload/upload:rw,z
      - /data/immich/library:/usr/src/app/upload/library:rw,z
      - /data/immich/thumbs:/usr/src/app/upload/thumbs:rw,z
      - /data/immich/profile:/usr/src/app/upload/profile:rw,z
      - /data/immich/encoded-video:/usr/src/app/upload/encoded-video:rw,z
      - /etc/localtime:/etc/localtime:ro,z
      - /data/certs/root_ca.crt:/etc/ssl/certs/root_ca.pem:ro,z
    depends_on:
      postgresqldb:
        condition: service_started
      redis:
        condition: service_started
      typesense:
        condition: service_started
      immich-server:
        condition: service_started

...

Your .env content

N/A

Reproduction steps

1) Deploy latest release (v1.89.0)
2) Inspected every Immich container logs
3) Catch the error above in immich-microservices container.

Additional information

I logged in immich-microservices container and checked the file permissions.

root@immich-microservices:/usr/src/app# ls -lah /usr/src/resources/
total 34M
drwxr-xr-x. 2 root root  102 Nov 25 03:05 .
drwxr-xr-x. 1 root root   17 Nov 25 03:27 ..
-rw-------. 1 root root 136K Nov 24 04:55 admin1CodesASCII.txt
-rw-------. 1 root root 2.2M Nov 24 04:55 admin2Codes.txt
-rw-r--r--. 1 root root  31M Nov 24 05:48 cities500.txt
-rw-r--r--. 1 root root   25 Nov 25 03:05 geodata-date.txt

It doesn't make sense to have the two files above admin1CodesASCII.txt and admin2Codes.txt non-readable by others.

One solution would be to change the permissions of the two files admin1CodesASCII.txt and admin2Codes.txt, in Dockerfile

amitrea commented 9 months ago

After changing manually the two file permissions in immich-server (less important but for consistency) and immich-microservices the importing of geodata worked.

$ chmod 644 /usr/src/resources/*

Restart both containers immich-server, immich-microservices:

$ podman restart immich-server
$ podman restart immich-microservices
$ podman logs -f immich-microservices

...
[Nest] 3  - 11/30/2023, 10:37:24 AM     LOG [NestFactory] Starting Nest application...
[Nest] 3  - 11/30/2023, 10:37:24 AM     LOG [InstanceLoader] TypeOrmModule dependencies initialized +26ms
[Nest] 3  - 11/30/2023, 10:37:24 AM     LOG [InstanceLoader] BullModule dependencies initialized +1ms
[Nest] 3  - 11/30/2023, 10:37:24 AM     LOG [InstanceLoader] ConfigHostModule dependencies initialized +0ms
[Nest] 3  - 11/30/2023, 10:37:24 AM     LOG [InstanceLoader] DiscoveryModule dependencies initialized +0ms
[Nest] 3  - 11/30/2023, 10:37:24 AM     LOG [InstanceLoader] ScheduleModule dependencies initialized +0ms
[Nest] 3  - 11/30/2023, 10:37:24 AM     LOG [InstanceLoader] ConfigModule dependencies initialized +4ms
[Nest] 3  - 11/30/2023, 10:37:24 AM     LOG [InstanceLoader] BullModule dependencies initialized +0ms
[Nest] 3  - 11/30/2023, 10:37:24 AM     LOG [InstanceLoader] BullModule dependencies initialized +0ms
[Nest] 3  - 11/30/2023, 10:37:24 AM     LOG [InstanceLoader] TypeOrmCoreModule dependencies initialized +113ms
[Nest] 3  - 11/30/2023, 10:37:24 AM     LOG [InstanceLoader] TypeOrmModule dependencies initialized +0ms
[Nest] 3  - 11/30/2023, 10:37:24 AM     LOG [InstanceLoader] InfraModule dependencies initialized +2ms
[Nest] 3  - 11/30/2023, 10:37:24 AM     LOG [InstanceLoader] DomainModule dependencies initialized +14ms
[Nest] 3  - 11/30/2023, 10:37:24 AM     LOG [InstanceLoader] MicroservicesModule dependencies initialized +0ms
[Nest] 3  - 11/30/2023, 10:37:24 AM     LOG [MetadataRepository] Initializing metadata repository
[Nest] 3  - 11/30/2023, 10:37:24 AM     LOG [MetadataRepository] Importing geodata to database from file
[Nest] 3  - 11/30/2023, 10:37:45 AM     LOG [MetadataRepository] Geodata import completed
[Nest] 3  - 11/30/2023, 10:37:45 AM     LOG [MetadataService] Initialized local reverse geocoder
[Nest] 3  - 11/30/2023, 10:37:45 AM     LOG [SearchService] Running bootstrap
[Nest] 3  - 11/30/2023, 10:37:45 AM     LOG [TypesenseRepository] Schema up to date: assets/assets-v10
[Nest] 3  - 11/30/2023, 10:37:45 AM     LOG [TypesenseRepository] Schema up to date: albums/albums-v2
[Nest] 3  - 11/30/2023, 10:37:45 AM     LOG [TypesenseRepository] Schema up to date: faces/faces-v1
[Nest] 3  - 11/30/2023, 10:37:45 AM     LOG [TypesenseRepository] Alias mapping: [{"collection_name":"faces-v1","name":"faces"},{"collection_name":"albums-v2","name":"albums"},{"collection_name":"assets-v10","name":"assets"}]
[Nest] 3  - 11/30/2023, 10:37:45 AM     LOG [TypesenseRepository] Collections needing migration: {"assets":false,"albums":false,"faces":false}
[Nest] 3  - 11/30/2023, 10:37:45 AM     LOG [NestApplication] Nest application successfully started +13ms
[Nest] 3  - 11/30/2023, 10:37:45 AM     LOG [ImmichMicroservice] Immich Microservices is listening on http://[::1]:3002 [v1.89.0] [PRODUCTION]
bo0tzz commented 9 months ago

@zackpollard should we just run this chmod in the base image build?

amitrea commented 9 months ago

Yes. In "Additional information" section I pointed to the Dockerfile and line.

zackpollard commented 9 months ago

@zackpollard should we just run this chmod in the base image build?

Yes that makes sense, I'm not really sure why the file is created as 600 by docker at all tbh... Interesting default.

amitrea commented 9 months ago

Maybe there's a user mask somewhere which I cannot see.

I will try to provide a PR asap in the base server image repo.

baloo2 commented 9 months ago

I am experiencing exactly the same problem on my Synology. Stopped Immich project till fixed. Hopefully very soon...

otbutz commented 9 months ago

Workaround:

docker compose exec -it --user root immich-microservices bash
chmod 644 /usr/src/resources/*
docker compose restart immich-microservices
amitrea commented 9 months ago

It seems that Docker ADD URL_SRC DST run with umask 077.

I replaced all ADD commands + the RUN command related to geodata into one RUN command, using wget for downloading, with a small improvement on the size of dev base server image: from 2.31 GB to 2.30 GB. The prod base server image is 856 MB, which I think it is the same in both cases (with copy and wget). The permissions on the files with this change are OK.

Please notice that the remote geodata files are dated with today, so probably they changed since the last release.

In several minutes I will come with the PR.

zackpollard commented 9 months ago

https://github.com/immich-app/base-images/pull/21 should fix this, which will be included in the next Immich release. Please tag me if this persists after the next release has been published.

yodatak commented 9 months ago

Some command to run if k3s is use to fix permission

kubectl get pod  -n photos | grep micro
kubectl -n photos describe pod immich-microservices-b59d867bd-f2lln | grep 'Container ID:'
sudo k3s ctr task exec -t --exec-id myshell --user root YOURCONTENAIRIDHERE bash
chmod 644 /usr/src/resources/*
Nertskull commented 9 months ago

I had the same. And I tried the above work around with the chmod 644 which seems to have fixed the problem for the ASCII file.

But now I get the following error:

immich_microservices     | [Nest] 7  - 12/03/2023, 7:05:39 AM   ERROR [MetadataService] Unable to initialize reverse geocoding: Error: EACCES: permission denied, open '/usr/src/resources/geodata-date.txt'
immich_microservices     | Error: EACCES: permission denied, open '/usr/src/resources/geodata-date.txt'

I tried chmod with 777 and still the same error.

Ultimately this may not matter and may be fixed with the next release update. But just thought I'd document this. I'll check again once the next release is out.

amitrea commented 9 months ago

Log into the immich-microservices container with user root: $ docker exec -it --user root immich-microservices bash

Run this: $ ls -la /usr/src/resources/

Post the output you have.

The output should look like:

-rw-r--r--. 1 root root   138529 Nov 24 04:55 admin1CodesASCII.txt
-rw-r--r--. 1 root root  2251114 Nov 24 04:55 admin2Codes.txt
-rw-r--r--. 1 root root 32483181 Nov 24 05:48 cities500.txt
-rw-r--r--. 1 root root       25 Nov 25 03:05 geodata-date.txt

If the output is correct please try to restart immich-microservices container (this is neccessary if you change the files permissions): $ docker restart immich-microservices

Check the logs for immich-microservices container: $ docker logs -f immich-microservices