[BUG] CLI error when uploading a folder with 8k images

[SitramSoft]

The bug

When I tried to upload a folder with over 8k images and videos using the new CLI, I got the following error. I think the node process ran out of memory then was killed by the OS. My laptop runs Arch and has 16 GB of RAM memory. The process consumed according to the Resource Monitor, a bit over 12 GB RAM before I got the error.

Uploading a folder that contained around 1.3k images and videos using the new CLI worked.

Below is a snippet from the content of the folder and the file count:

The OS that Immich Server is running on

Ubuntu 22.04.2 LTS

Version of Immich Server

v1.88.1

Version of Immich Mobile App

v1.88.0 build.112

Platform with the issue

• [X] Server
• [ ] Web
• [ ] Mobile

Your docker-compose.yml content

version: "3.9"

services:
#Immich - self hosted photos and videos management application - https://immich.app/
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    command: [ "start.sh", "immich" ]
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env
    ports:
      - 2283:3001
    labels:
      - "com.centurylinklabs.watchtower.enable=false"
    depends_on:
      - typesense
    restart: always

  immich-microservices:
    container_name: immich_microservices
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    # extends:
    #   file: hwaccel.yml
    #   service: hwaccel
    command: [ "start.sh", "microservices" ]
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env
    labels:
      - "com.centurylinklabs.watchtower.enable=false"
    depends_on:
      - typesense
    restart: always

  immich-machine-learning:
    container_name: immich_machine_learning
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
    volumes:
      - model-cache:/cache
    env_file:
      - .env
    labels:
      - "com.centurylinklabs.watchtower.enable=false"
    restart: always

  typesense:
    container_name: immich_typesense
    image: typesense/typesense:0.24.1@sha256:9bcff2b829f12074426ca044b56160ca9d777a0c488303469143dd9f8259d4dd
    environment:
      - TYPESENSE_API_KEY=${TYPESENSE_API_KEY}
      - TYPESENSE_DATA_DIR=/data
      # remove this to get debug messages
      - GLOG_minloglevel=1
    volumes:
      - tsdata:/data
    labels:
      - "com.centurylinklabs.watchtower.enable=false"
    cpus: 6
    restart: always

volumes:
  model-cache:
  tsdata:

Your .env content

COMPOSE_PROJECT_NAME=serenity_immich

# You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables

# The location where your uploaded files are stored
UPLOAD_LOCATION=/home/sitram/data/photos_immich

# The Immich version to use. You can pin this to a specific version like "v1.71.0"
IMMICH_VERSION=release

# Connection secrets for postgres and typesense. You should change these to random passwords
# Typesense
TYPESENSE_ENABLED=true
TYPESENSE_API_KEY=xxx
DB_PASSWORD=xxx

# Timezone
TZ=Europe/Bucharest

# The values below this line do not need to be changed
###################################################################################
DB_HOSTNAME=192.168.0.101
DB_USERNAME=xxx
DB_DATABASE_NAME=xxx

REDIS_HOSTNAME=192.168.0.101

Reproduction steps

1. immich login-key http://192.168.0.101:2283/api xxx
2. immich upload --dry-run  --recursive --album '2020 - Sabrina' /home/sitram/mounts/data/photos/Sabrina/2020

Additional information

Redis and database container are hosted in a common docker container for my entire homlab, that's why they are not part of the docker-compose.yml content

[esmoyer]

Similar boat, wanting to move from the AIO install to the new compose; but hitting this limit on resources; I also noticed that it doesn't give a progress bar like the old CLI tool did, not sure if that is a bug or by design, not a huge issue.

[commanderalpha]

Same problem here. First the CLI is consuming huge ammount of memory depending on the image count. In my case ~17.5 GB for aorund 11k images. Second issue: After the copy process is started, it gets slower and slower. I Aborted the process after around 1GB uploaded. Then the speed was around one pictures every 5 second. The first pictures copies fast as usual.

I tried this with the CLI directly in the Immich Container and also on my windows machine where i installed the CLI through npm.

[esmoyer]

Definitely something weird going on. Tried running on smaller individual folders and it will crash my entire Proxmox server with CPU stalls but am not able to see anything in any logs. Resources don't seem to spike, that I notice.

[alextran1502]

@esmoyer how many ram do you have on the machine you are trying to run the cli on?

[esmoyer]

4 CPU; 16GB RAM; 8GB Swap; it could very well be something else, but when I used the old CLI tool months ago it seemed to go a lot smoother. This is my first time using the new CLI tool, so not sure what to expect.

@esmoyer how many ram do you have on the machine you are trying to run the cli on?

[alextran1502]

@esmoyer can you update the CLI tool to the latest version 2.0.4 to see if it still causes the issue?

You can do npm i -g @immich/cli to update it

[SitramSoft]

@alextran1502 I can confirm that the immich upload command in --dry-run mode is successfully executed on two folders with a large amount of images and videos in them. Issue can be closed.

Thanks for a quick fix!

[esmoyer]

@alextran1502 seems to be working well now, thanks for the quick fix!