[BUG] Failed to upload 0 files []

[Waldorf3]

The bug

Attempting to import additional photos to an existing setup crashes with a "Failed to upload 0 files []" error.

The OS that Immich Server is running on

Debian 11, Docker

Version of Immich Server

v1.52.1

Version of Immich Mobile App

n/a

Platform with the issue

• [X] Server
• [ ] Web
• [ ] Mobile

Your docker-compose.yml content

version: "3.8"

services:
  immich-server:
    image: altran1502/immich-server:release
    entrypoint: ["/bin/sh", "./start-server.sh"]
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
    env_file:
      - .env
    environment:
      - NODE_ENV=production
    depends_on:
      - redis
      - database
    restart: unless-stopped

  immich-microservices:
    image: altran1502/immich-server:release
    entrypoint: ["/bin/sh", "./start-microservices.sh"]
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
    env_file:
      - .env
    environment:
      - NODE_ENV=production
    depends_on:
      - redis
      - database
    restart: unless-stopped

  immich-machine-learning:
    image: altran1502/immich-machine-learning:release
    entrypoint: ["/bin/sh", "./entrypoint.sh"]
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
    env_file:
      - .env
    environment:
      - NODE_ENV=production
    depends_on:
      - database
    restart: unless-stopped

  immich-web:
    image: altran1502/immich-web:release
    entrypoint: ["/bin/sh", "./entrypoint.sh"]
    env_file:
      - .env
    restart: unless-stopped

  redis:
    container_name: immich_redis
    image: redis:6.2
    restart: unless-stopped

  database:
    container_name: immich_postgres
    image: postgres:14
    env_file:
      - .env
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
      PG_DATA: /var/lib/postgresql/data
    volumes:
      - ./pgdata:/var/lib/postgresql/data
    restart: unless-stopped

  immich-proxy:
    container_name: immich_proxy
    image: altran1502/immich-proxy:release
    ports:
      - 8087:8080
    logging:
      driver: none
    depends_on:
      - immich-server
    restart: unless-stopped

  typesense:
    container_name: immich_typesense
    image: typesense/typesense:0.24.0
    environment:
      - TYPESENSE_API_KEY=${TYPESENSE_API_KEY}
      - TYPESENSE_DATA_DIR=/data
    logging:
      driver: none
    volumes:
      - ./tsdata:/data

Your .env content

###################################################################################
# Database
###################################################################################

DB_HOSTNAME=immich_postgres
DB_USERNAME=postgres
DB_PASSWORD=postgres
DB_DATABASE_NAME=immich

# Optional Database settings:
# DB_PORT=5432

###################################################################################
# Redis
###################################################################################

REDIS_HOSTNAME=immich_redis

# Optional Redis settings:
# REDIS_PORT=6379
# REDIS_DBINDEX=0
# REDIS_PASSWORD=
# REDIS_SOCKET=

###################################################################################
# Upload File Config
###################################################################################

UPLOAD_LOCATION=/media/private/immich

###################################################################################
# Log message level - [simple|verbose]
###################################################################################

LOG_LEVEL=simple

###################################################################################
# JWT SECRET
###################################################################################

# This JWT_SECRET is used to sign the authentication keys for user login
# You should set it to a long randomly generated value
# You can use this command to generate one: openssl rand -base64 128
JWT_SECRET=(redacted)
###################################################################################
# Reverse Geocoding
####################################################################################

# DISABLE_REVERSE_GEOCODING=false

# Reverse geocoding is done locally which has a small impact on memory usage
# This memory usage can be altered by changing the REVERSE_GEOCODING_PRECISION variable
# This ranges from 0-3 with 3 being the most precise
# 3 - Cities > 500 population: ~200MB RAM
# 2 - Cities > 1000 population: ~150MB RAM
# 1 - Cities > 5000 population: ~80MB RAM
# 0 - Cities > 15000 population: ~40MB RAM

# REVERSE_GEOCODING_PRECISION=3

####################################################################################
# WEB - Optional
####################################################################################

# Custom message on the login page, should be written in HTML form.
# For example PUBLIC_LOGIN_PAGE_MESSAGE="This is a demo instance of Immich.<br><br>Email: <i>demo@demo.de</i><br>Password: <i>demo</i>"

PUBLIC_LOGIN_PAGE_MESSAGE=

TYPESENSE_API_KEY=(redacted)

Reproduction steps

$ sudo docker run -it --rm -v /media/private/MainPhotoAlbum:/import ghcr.io/immich-app/immich-cli:latest upload --key (redacted) --server http://192.168.10.10:8087/api --recursive
Checking connectivity with Immich instance...
Server status: OK
Checking credentials...
Login status: OK
Successful authentication for user (redacted)
Indexing local assets...
Indexing complete, found 41893 local assets
Comparing local assets with those on the Immich instance...
A total of 1126 assets will be uploaded to the server
Do you want to start upload now? (y/n) y
Start uploading...
Upload Progress | █░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ | 2% || 1126/41893 || Current file [/import/ziggy/2020-12-31
Failed to upload 0 files  []

Additional information

docker logs show a lot of duplicate errors. There may be duplicate images in the import feed, but shouldn't the system be able to deal with that without crashing?

immich_postgres                   | 2023-03-30 09:59:48.768 UTC [122] ERROR:  duplicate key value violates unique constraint "UQ_userid_checksum"
immich_postgres                   | 2023-03-30 09:59:48.768 UTC [122] DETAIL:  Key ("ownerId", checksum)=(a75fb4b3-253c-4974-98b9-259a5a4a0905, \xa2e7020fe74bc2885662931ed8ac5b6a08820495) already exists.
immich_postgres                   | 2023-03-30 09:59:48.768 UTC [122] STATEMENT:  INSERT INTO "assets"("id", "deviceAssetId", "ownerId", "deviceId", "type", "originalPath", "resizePath", "webpPath", "encodedVideoPath", "createdAt", "updatedAt", "fileCreatedAt", "fileModifiedAt", "isFavorite", "mimeType", "checksum", "duration", "isVisible", "livePhotoVideoId") VALUES (DEFAULT, $1, $2, $3, $4, $5, $6, $7, $8, DEFAULT, DEFAULT, $9, $10, $11, $12, $13, $14, $15, DEFAULT) RETURNING "id", "webpPath", "encodedVideoPath", "createdAt", "updatedAt", "isFavorite", "isVisible"
immich_postgres                   | 2023-03-30 09:59:48.774 UTC [123] ERROR:  duplicate key value violates unique constraint "UQ_userid_checksum"
immich_postgres                   | 2023-03-30 09:59:48.774 UTC [123] DETAIL:  Key ("ownerId", checksum)=(a75fb4b3-253c-4974-98b9-259a5a4a0905, \x187021c65324d73cd0db100d087029ac2705c04c) already exists.
immich_postgres                   | 2023-03-30 09:59:48.774 UTC [123] STATEMENT:  INSERT INTO "assets"("id", "deviceAssetId", "ownerId", "deviceId", "type", "originalPath", "resizePath", "webpPath", "encodedVideoPath", "createdAt", "updatedAt", "fileCreatedAt", "fileModifiedAt", "isFavorite", "mimeType", "checksum", "duration", "isVisible", "livePhotoVideoId") VALUES (DEFAULT, $1, $2, $3, $4, $5, $6, $7, $8, DEFAULT, DEFAULT, $9, $10, $11, $12, $13, $14, $15, DEFAULT) RETURNING "id", "webpPath", "encodedVideoPath", "createdAt", "updatedAt", "isFavorite", "isVisible"
immich_postgres                   | 2023-03-30 09:59:48.786 UTC [123] ERROR:  duplicate key value violates unique constraint "UQ_userid_checksum"
immich_postgres                   | 2023-03-30 09:59:48.786 UTC [123] DETAIL:  Key ("ownerId", checksum)=(a75fb4b3-253c-4974-98b9-259a5a4a0905, \x7d933e446aad4d664207d9c9bdf358a0791f15d0) already exists.
immich_postgres                   | 2023-03-30 09:59:48.786 UTC [123] STATEMENT:  INSERT INTO "assets"("id", "deviceAssetId", "ownerId", "deviceId", "type", "originalPath", "resizePath", "webpPath", "encodedVideoPath", "createdAt", "updatedAt", "fileCreatedAt", "fileModifiedAt", "isFavorite", "mimeType", "checksum", "duration", "isVisible", "livePhotoVideoId") VALUES (DEFAULT, $1, $2, $3, $4, $5, $6, $7, $8, DEFAULT, DEFAULT, $9, $10, $11, $12, $13, $14, $15, DEFAULT) RETURNING "id", "webpPath", "encodedVideoPath", "createdAt", "updatedAt", "isFavorite", "isVisible"

[alextran1502]

Does the server actually crash? Can you still accessing the web?

If you try to run the CLI again, is it reproducible on multiple runs?

[Waldorf3]

Sorry, no the import process crash, not the server. My bad.

Yes, it is reproducable on multiple runs.

[etnoy]

Are you sure it's crashing? It says all files were uploaded (0 failed items)

[Waldorf3]

It says Upload Progress | █░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ | 2% || 1126/41893 ... that looks like it hasn't completed, no?

Also, every time I restart it, it says A total of 1126 assets will be uploaded to the server That would kinda suggest there still are assets to be uploaded, no?

[jilles]

I have the same issue. Did an initial upload of the original 1000 photos. This failed somewhere in betweeen (my own fault). Then I started the upload again for the rest (let's say 200). immich-cli then tells me it will upload 200 of 1000 and then says that it is done at 20% at 200/1000 with the bar also at 20%. Overall this looks like all files were done, but the bar and division is misleading if it is indeed finished...

[etnoy]

Would you be able to run a custom version of the cli script that give more logging? That could help us find the issue.

The CLI isn't very good right now and a rewrite is being worked on.

[jilles]

Sure, why not.

[Waldorf3]

$ sudo docker logs immich-immich-machine-learning-1
/bin/sh: 0: cannot open ./entrypoint.sh: No such file
/bin/sh: 0: cannot open ./entrypoint.sh: No such file
/bin/sh: 0: cannot open ./entrypoint.sh: No such file
/bin/sh: 0: cannot open ./entrypoint.sh: No such file
/bin/sh: 0: cannot open ./entrypoint.sh: No such file
/bin/sh: 0: cannot open ./entrypoint.sh: No such file
/bin/sh: 0: cannot open ./entrypoint.sh: No such file
/bin/sh: 0: cannot open ./entrypoint.sh: No such file

Seems the machine learning modules is crashing now. Is this related? v1.52.1

[alextran1502]

$ sudo docker logs immich-immich-machine-learning-1

/bin/sh: 0: cannot open ./entrypoint.sh: No such file

/bin/sh: 0: cannot open ./entrypoint.sh: No such file

/bin/sh: 0: cannot open ./entrypoint.sh: No such file

/bin/sh: 0: cannot open ./entrypoint.sh: No such file

/bin/sh: 0: cannot open ./entrypoint.sh: No such file

/bin/sh: 0: cannot open ./entrypoint.sh: No such file

/bin/sh: 0: cannot open ./entrypoint.sh: No such file

/bin/sh: 0: cannot open ./entrypoint.sh: No such file

Seems the machine learning modules is crashing now. Is this related? v1.52.1

I believe your docker-compose file is out of date

[bgriffen]

I also have the same error, which is the latest docker-compose file I should update mine to?

[alextran1502]

@bgriffen If you still have the same issue. Please open an issue or head over to Discord for faster support