immich-app / immich

High performance self-hosted photo and video management solution.
https://immich.app
GNU Affero General Public License v3.0
44.56k stars 2.17k forks source link

[BUG] Error when uploading a video larger than 2Gib with CLI #5217

Closed SitramSoft closed 9 months ago

SitramSoft commented 9 months ago

The bug

I got an error with the new immich CLI when I upload a folder that contains a 23 Gib mp4 file from my wedding album.

image

The OS that Immich Server is running on

Ubuntu 22.04.2 LTS

Version of Immich Server

v1.88.1

Version of Immich Mobile App

v1.88.0 build.112

Platform with the issue

Your docker-compose.yml content

version: "3.9"

services:
#Immich - self hosted photos and videos management application - https://immich.app/
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    command: [ "start.sh", "immich" ]
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env
    ports:
      - 2283:3001
    labels:
      - "com.centurylinklabs.watchtower.enable=false"
    depends_on:
      - typesense
    restart: always

  immich-microservices:
    container_name: immich_microservices
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    # extends:
    #   file: hwaccel.yml
    #   service: hwaccel
    command: [ "start.sh", "microservices" ]
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env
    labels:
      - "com.centurylinklabs.watchtower.enable=false"
    depends_on:
      - typesense
    restart: always

  immich-machine-learning:
    container_name: immich_machine_learning
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
    volumes:
      - model-cache:/cache
    env_file:
      - .env
    labels:
      - "com.centurylinklabs.watchtower.enable=false"
    restart: always

  typesense:
    container_name: immich_typesense
    image: typesense/typesense:0.24.1@sha256:9bcff2b829f12074426ca044b56160ca9d777a0c488303469143dd9f8259d4dd
    environment:
      - TYPESENSE_API_KEY=${TYPESENSE_API_KEY}
      - TYPESENSE_DATA_DIR=/data
      # remove this to get debug messages
      - GLOG_minloglevel=1
    volumes:
      - tsdata:/data
    labels:
      - "com.centurylinklabs.watchtower.enable=false"
    cpus: 6
    restart: always

volumes:
  model-cache:
  tsdata:

Your .env content

COMPOSE_PROJECT_NAME=serenity_immich

# You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables

# The location where your uploaded files are stored
UPLOAD_LOCATION=/home/sitram/data/photos_immich

# The Immich version to use. You can pin this to a specific version like "v1.71.0"
IMMICH_VERSION=release

# Connection secrets for postgres and typesense. You should change these to random passwords
# Typesense
TYPESENSE_ENABLED=true
TYPESENSE_API_KEY=xxx
DB_PASSWORD=xxx

# Timezone
TZ=Europe/Bucharest

# The values below this line do not need to be changed
###################################################################################
DB_HOSTNAME=192.168.0.101
DB_USERNAME=xxx
DB_DATABASE_NAME=xxx

REDIS_HOSTNAME=192.168.0.101

Reproduction steps

1. immich login-key http://192.168.0.101:2283/api xxx
2. immich upload $DRY_RUN --recursive --album '2022.07.24 - Nunta Oli & Adi - Annes Events - Mosnita Veche' /home/sitram/mounts/data/photos/'2022.07.24 - Nunta Oli & Adi - Annes Events - Mosnita Veche'

Additional information

Redis and database container are hosted in a common docker container for my entire homlab, that's why they are not part of the docker-compose.yml content

AlexGuld commented 9 months ago

I get same error on a new install trying to upload via CLI.

TajAFK commented 9 months ago

Got the same error on ubuntu server

alextran1502 commented 9 months ago

Fixed and release please update the CLI with npm i -g @immich/cli to version 2.0.4

SitramSoft commented 9 months ago

The new version introduced a progress bar that shows remaining time, size and which file which file is being processed. This is quite amazing!

@alextran1502 you made some serious improvements to the CLI because the memory usage of the node process when the command runs has been drastically reduced. It consumes now around 80 MB.

I can confirm that the immich upload command no longer crashes in --dry-run mode when processing a large file.

Thanks for a quick fix!