Assets automatically moved to trash

Thloluvin commented 1 month ago

The bug

Hello,

My assest are moved trash bin after a few hours once they have been added. This affects only assets added from an external library. A rescan is doing nothing as the assets still remain in the trash bin. Only removing the library and readding starts scanning and adding again the assets. But a few hours laters the assets are moved again to the trash bin. All Albums are emptied, so it shows that there are 0 Assets inside. Restoring the assets from trash bin is not working, all assets remain there. I am running immich about two years and an never had this problem in the past and I thought the problem was caused due to the breaking change in 1.118 and so I updated compose file to the latest version I found on Github. Since latest version I am running Immich in portainer, this problem was before there and is still present. I have now deactivated trash features in admin settings and will check now if this problem remains.

BR Richard

The OS that Immich Server is running on

Ubuntu 22.10

Version of Immich Server

v1.118.2

Version of Immich Mobile App

v1.118.0 build .163

Platform with the issue

[X] Server
[ ] Web
[ ] Mobile

Your docker-compose.yml content

#
# https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
#

name: immich

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    # extends:
    #   file: hwaccel.transcoding.yml
    #   service: cpu # set to one of [nvenc, quicksync, rkmpp, vaapi, vaapi-wsl] for accelerated transcoding
    volumes:
      # Do not edit the next line. If you want to change the media storage location on your system, edit the value of UPLOAD_LOCATION in the .env file
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
      - /path/4_MA-Verein:/usr/src/app/ma:ro
      - /path/5_Musical:/usr/src/app/musical:ro
      - /path/6_KulturhausWartberg:/usr/src/app/khw:ro
      - /path/7_Geburtstage:/usr/src/app/bdays:ro
      - /path/8_Hochzeiten&JGA:/usr/src/app/hochzeit:ro
      - /path/9_Konzerte&Festivals:/usr/src/app/konzerte:ro
      - /path/10_Lanpartys:/usr/src/app/lan:ro
      - /path/11_Lost_Places:/usr/src/app/lostplaces:ro
      - /path/12_Partys:/usr/src/app/partys:ro
      - /path/13_Radfahren:/usr/src/app/radfahren:ro
      - /path/14_Sylvester:/usr/src/app/sylvester:ro
      - /path/15_Urlaube:/usr/src/app/urlaub:ro
      - /path/16_Wandern:/usr/src/app/wandern:ro

    env_file:
      - stack.env
    ports:
      - 2283:2283
    depends_on:
      - redis
      - database
    restart: always
    healthcheck:
      disable: false
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.immich.entrypoints=websecure"
      - "traefik.http.routers.immich.rule=Host(`immich.domain.eu`)"
      - "traefik.http.routers.immich.tls=true"
      - "traefik.http.routers.immich.tls.certresolver=http_resolver"
      - "traefik.http.routers.immich.service=immich"
      - "traefik.http.services.immich.loadbalancer.server.port=2283"
      - "traefik.docker.network=proxy"
      - "traefik.http.routers.immich.middlewares=default@file"
    networks:
      - immich
      - proxy

  immich-machine-learning:
    container_name: immich_machine_learning
    # For hardware acceleration, add one of -[armnn, cuda, openvino] to the image tag.
    # Example tag: ${IMMICH_VERSION:-release}-cuda
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
    # extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/ml-hardware-acceleration
    #   file: hwaccel.ml.yml
    #   service: cpu # set to one of [armnn, cuda, openvino, openvino-wsl] for accelerated inference - use the `-wsl` version for WSL2 where applicable
    volumes:
      - model-cache:/cache
    env_file:
      - stack.env
    restart: always
    healthcheck:
      disable: false
    networks:
      - immich

  redis:
    container_name: immich_redis
    image: docker.io/redis:6.2-alpine@sha256:2ba50e1ac3a0ea17b736ce9db2b0a9f6f8b85d4c27d5f5accc6a416d8f42c6d5
    healthcheck:
      test: redis-cli ping || exit 1
    restart: always
    networks:
      - immich

  database:
    container_name: immich_postgres
    image: docker.io/tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
      POSTGRES_INITDB_ARGS: '--data-checksums'
    volumes:
      # Do not edit the next line. If you want to change the database storage location on your system, edit the value of DB_DATA_LOCATION in the .env file
      - ${DB_DATA_LOCATION}:/var/lib/postgresql/data
    healthcheck:
      test: pg_isready --dbname='${DB_DATABASE_NAME}' --username='${DB_USERNAME}' || exit 1; Chksum="$$(psql --dbname='${DB_DATABASE_NAME}' --username='${DB_USERNAME}' --tuples-only --no-align --command='SELECT COALESCE(SUM(checksum_failures), 0) FROM pg_stat_database')"; echo "checksum failure count is $$Chksum"; [ "$$Chksum" = '0' ] || exit 1
      interval: 5m
      start_interval: 30s
      start_period: 5m
    command: ["postgres", "-c", "shared_preload_libraries=vectors.so", "-c", 'search_path="$$user", public, vectors', "-c", "logging_collector=on", "-c", "max_wal_size=2GB", "-c", "shared_buffers=512MB", "-c", "wal_compression=on"]
    restart: always
    networks:
      - immich

volumes:
  model-cache:
networks:
  proxy:
    external: true
  immich:

Your .env content

IMMICH_VERSION=release
DB_DATA_LOCATION=path/database
DB_DATABASE_NAME=DB_Name
DB_USERNAME=DB_User
DB_PASSWORD=DB_Pass
UPLOAD_LOCATION=/path/upload
LOG_LEVEL=warn

Reproduction steps

adding assets
waiting a few hours
assets are moved to trash

Relevant log output

No response

Additional information

My immich config: immich-config.json

Thloluvin commented 1 month ago

Today I have checked again and all assets are again removed

And here is the log output:

[Nest] 7 - 10/25/2024, 12:00:00 AM ERROR [Microservices:JobService] Unable to run job handler (library/library-queue-sync-files): EntityNotFoundError: Could not find any entity of type "LibraryEntity" matching: { "where": { "id": "ae8799c0-544f-4331-8074-fcab6f54e2f0" }, "relations": { "owner": true }, "withDeleted": false } [Nest] 7 - 10/25/2024, 12:00:00 AM ERROR [Microservices:JobService] EntityNotFoundError: Could not find any entity of type "LibraryEntity" matching: { "where": { "id": "ae8799c0-544f-4331-8074-fcab6f54e2f0" }, "relations": { "owner": true }, "withDeleted": false } at /usr/src/app/node_modules/typeorm/entity-manager/EntityManager.js:652:39 at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async LibraryService.handleQueueSyncFiles (/usr/src/app/dist/services/library.service.js:433:25) at async /usr/src/app/dist/services/job.service.js:163:36 at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28) at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24) [Nest] 7 - 10/25/2024, 12:00:00 AM ERROR [Microservices:JobService] Object: { "id": "ae8799c0-544f-4331-8074-fcab6f54e2f0" }

[Nest] 7 - 10/25/2024, 12:00:00 AM ERROR [Microservices:JobService] Unable to run job handler (library/library-queue-sync-assets): EntityNotFoundError: Could not find any entity of type "LibraryEntity" matching: { "where": { "id": "ae8799c0-544f-4331-8074-fcab6f54e2f0" }, "relations": { "owner": true }, "withDeleted": false } [Nest] 7 - 10/25/2024, 12:00:00 AM LOG [Microservices:LibraryService] Refreshing library 880857de-cb9f-4856-ab86-4677053b9522 for new assets [Nest] 7 - 10/25/2024, 12:00:00 AM ERROR [Microservices:JobService] EntityNotFoundError: Could not find any entity of type "LibraryEntity" matching: { "where": { "id": "ae8799c0-544f-4331-8074-fcab6f54e2f0" }, "relations": { "owner": true }, "withDeleted": false } at /usr/src/app/node_modules/typeorm/entity-manager/EntityManager.js:652:39 at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async LibraryService.handleQueueSyncAssets (/usr/src/app/dist/services/library.service.js:475:25) at async /usr/src/app/dist/services/job.service.js:163:36 at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28) at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24) [Nest] 7 - 10/25/2024, 12:00:00 AM ERROR [Microservices:JobService] Object: { "id": "ae8799c0-544f-4331-8074-fcab6f54e2f0" }

[Nest] 7 - 10/25/2024, 12:00:00 AM LOG [Microservices:LibraryService] Refreshing library 880857de-cb9f-4856-ab86-4677053b9522 for new assets [Nest] 7 - 10/25/2024, 12:00:00 AM ERROR [Microservices:JobService] Unable to run job handler (library/library-queue-sync-assets): EntityNotFoundError: Could not find any entity of type "LibraryEntity" matching: { "where": { "id": "ae8799c0-544f-4331-8074-fcab6f54e2f0" }, "relations": { "owner": true }, "withDeleted": false } [Nest] 7 - 10/25/2024, 12:00:00 AM ERROR [Microservices:JobService] EntityNotFoundError: Could not find any entity of type "LibraryEntity" matching: { "where": { "id": "ae8799c0-544f-4331-8074-fcab6f54e2f0" }, "relations": { "owner": true }, "withDeleted": false } at /usr/src/app/node_modules/typeorm/entity-manager/EntityManager.js:652:39 at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async LibraryService.handleQueueSyncAssets (/usr/src/app/dist/services/library.service.js:475:25) at async /usr/src/app/dist/services/job.service.js:163:36 at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28) at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24) [Nest] 7 - 10/25/2024, 12:00:00 AM ERROR [Microservices:JobService] Object: { "id": "ae8799c0-544f-4331-8074-fcab6f54e2f0" }

alextran1502 commented 1 month ago

This indicates that your external library gets unmounted at some point

maxbraun91 commented 1 month ago

I experienced the same issue today: all assets are moved to trash. Don't know, if external library got disconnected at some point... What is the best way to move the assets back from trash and also keep the assets that were added to albums? And is there any way to prevent this behavior? Thanks!

Thloluvin commented 1 month ago

Thank you Alex

I am running my host with a SW Raid5 (mdadm) setup with hdds for my data, This contains my pictures and added in Immich as external drives. Raid is running unchanged without any problems for years and is accessible 24/7, so I am wondering why this is now unmounted at some point. Immich is running on a SSD drive, upload dir and DB are there.

So what i can try now is

to copy some pics to my drive where immich is running and try to add this as external drive to check if it will work
to move my upload dir to my raid, to see if the problem still exists there with uploaded pictures
check syslog for mdadm related output

I will give feedback after that, this will take one or two days Further suggestions are welcome

BR

Thloluvin commented 1 month ago

updated to 1.119.x and now i can not add anymore a new paths to an external library. A message appears in the screen that the path validation failed. Checked inside the container and all volumes and all files inside are accessable. When I try to add a library for fractions of a second a message in red is visible. So I tried to make a screenshot, because the time is to short to read this message. A delay would be helpful to read this message. This message says "This import path already exists". This is not possible, because I set up Immich completely new and this is the first path I have added to this library.

lib

Without a solution I can not go further with my initial problem Any ideas?

alextran1502 commented 1 month ago

Can you use absolute path?

Thloluvin commented 1 month ago

Thank you Alex, this worked for me

maxbraun91 commented 1 month ago

And just to add: the assets seem to reappear after the next library scan, if they are available again.

Thloluvin commented 1 month ago

Ok, that also seems to be the solution to my original problem. I have not changed anything in the setup, the assets remain and are not moved to the trash. I think a reference to the "Absolute Paths" when adding the paths to a library would be a good hint to avoid future problems of this kind.

grafik

Many thanks again and keep up the excellent work

immich-app / immich