immich-app / immich

High performance self-hosted photo and video management solution.
https://immich.app
GNU Affero General Public License v3.0
44.98k stars 2.18k forks source link

[BUG] Incorrect storage display size for Mac #4318

Open Rihcus opened 11 months ago

Rihcus commented 11 months ago

The bug

Storage counter in home ui broken.

image

My host mac has 256gb ssd and 1tb hard drive (usb)

The external 1tb hardrive contains the immich library and is using Mac OS Extended (Journaled) file system

Admin panal storage count is correct however:

image

The OS that Immich Server is running on

macOS 12.6.2 21G320 x86_64

Version of Immich Server

v1.80.0

Version of Immich Mobile App

v1.79.0

Platform with the issue

Your docker-compose.yml content

version: "3.8"

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    command: [ "start.sh", "immich" ]
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /Volumes/MACHD/PhotoLibrary/Library/Photos/Oneplus8/OP8_Camera_Backup/Camera:/mnt/media/Camera:ro
    env_file:
      - .env
    depends_on:
      - redis
      - database
      - typesense
    restart: always

  immich-microservices:
    container_name: immich_microservices
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    # extends:
    #   file: hwaccel.yml
    #   service: hwaccel
    command: [ "start.sh", "microservices" ]
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /Volumes/MACHD/PhotoLibrary/Library/Photos/Oneplus8/OP8_Camera_Backup/Camera:/mnt/media/Camera:ro
    env_file:
      - .env
    depends_on:
      - redis
      - database
      - typesense
    restart: always

  immich-machine-learning:
    container_name: immich_machine_learning
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
    volumes:
      - model-cache:/cache
    env_file:
      - .env
    restart: always

  immich-web:
    container_name: immich_web
    image: ghcr.io/immich-app/immich-web:${IMMICH_VERSION:-release}
    env_file:
      - .env
    restart: always

  typesense:
    container_name: immich_typesense
    image: typesense/typesense:0.24.1@sha256:9bcff2b829f12074426ca044b56160ca9d777a0c488303469143dd9f8259d4dd
    environment:
      - TYPESENSE_API_KEY=${TYPESENSE_API_KEY}
      - TYPESENSE_DATA_DIR=/data
      # remove this to get debug messages
      - GLOG_minloglevel=1
    volumes:
      - tsdata:/data
    restart: always

  redis:
    container_name: immich_redis
    image: redis:6.2-alpine@sha256:70a7a5b641117670beae0d80658430853896b5ef269ccf00d1827427e3263fa3
    restart: always

  database:
    container_name: immich_postgres
    image: postgres:14-alpine@sha256:28407a9961e76f2d285dc6991e8e48893503cc3836a4755bbc2d40bcc272a441
    env_file:
      - .env
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
    volumes:
      - pgdata:/var/lib/postgresql/data
    restart: always

  immich-proxy:
    container_name: immich_proxy
    image: ghcr.io/immich-app/immich-proxy:${IMMICH_VERSION:-release}
    environment:
      # Make sure these values get passed through from the env file
      - IMMICH_SERVER_URL
      - IMMICH_WEB_URL
    ports:
      - 2283:8080
    depends_on:
      - immich-server
      - immich-web
    restart: always

volumes:
  pgdata:
  model-cache:
  tsdata:

Your .env content

# You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables

# The location where your uploaded files are stored
UPLOAD_LOCATION=/Volumes/MACHD/PhotoLibrary/immich-upload

# The Immich version to use. You can pin this to a specific version like "v1.71.0"
IMMICH_VERSION=release

# Connection secrets for postgres and typesense. You should change these to random passwords
TYPESENSE_API_KEY=some-random-text
DB_PASSWORD=postgres

# The values below this line do not need to be changed
###################################################################################
DB_HOSTNAME=immich_postgres
DB_USERNAME=postgres
DB_DATABASE_NAME=immich

REDIS_HOSTNAME=immich_redis

Reproduction steps

1. Set up docker compose on macOS 12.6.2 21G320 x86_64 
2. Plug in an external usb hard drive (make sure its formated in Mac OS Extended (Journaled) file system).
3. Mount some external libraries to docker-compose.yml
4. Install immich and update the web ui accordingly
5. Run a library scan
6. Check the storage counter in the home ui and compare it to admin panal
...

Additional information

Possibly related issues:

bo0tzz commented 11 months ago

This is a docker bug that we can't really do anything about.

jrasm91 commented 11 months ago

I'd like to take a look at this one actually.

Mika571 commented 11 months ago

Also seeing this on macOS Monterey w/ docker desktop 4.24 image image

cedvan commented 8 months ago

I have exactly the same problem. After investigation, some images are missing in the originals "upload/library" folder.

WARNING, Immich may have deleted or not uploaded somes photos correctly. The "upload/thumbs" previews exists and the images are presents in the database. But their originals may be missing. Which explains this delta. Immich front display thumbs images and hide this problem. Try download image to test original is present or check picture path in "upload/library"

The storage "bottom left" indicates the true size of the library from the File system "upload/library". The servers stats indicates the size obtained from the database, so maybe bad

MatissAndersons commented 6 months ago

I just installed Immich on MacOS through docker compose. For me it was the same until I set the quotas for the users. I set the quota of 150gb and it shows 308.7mb usage which is exactly the same number as Finder app shows for library folder.

scottrbaxter commented 6 months ago

I just installed Immich on MacOS through docker compose. For me it was the same until I set the quotas for the users. I set the quota of 150gb and it shows 308.7mb usage which is exactly the same number as Finder app shows for library folder.

I've experienced exactly this with immich 1.97.0 on both macOS Sonoma 14.3.1 with Docker Desktop 4.26.1, as well as Synology's Container Manager "Docker 20.10.23-1437".

macOS - without quota

Screenshot 2024-03-06 at 5 06 34 PM

1000 GB quota

Screenshot 2024-03-06 at 5 07 58 PM

Synology - without quota (this is the only user, so 6.5GB should be the total storage used, which matches what the underlying system reports)

Screenshot 2024-03-06 at 5 01 20 PM

1000 GB quota

Screenshot 2024-03-06 at 5 01 36 PM

I can actually revert back to the incorrect storage size simply by changing my user quota from 1000 back to 0.

krakowski commented 6 months ago

This does also seem to affect Linux. Setting a quota (500 GiB in my case) fixes the calculated size, as mentioned in https://github.com/immich-app/immich/issues/4318#issuecomment-1972017633.

Screenshot from 2024-03-12 07-10-48Screenshot from 2024-03-12 07-10-57

mufeedali commented 5 months ago

Seems to affect every case where storage is using sshfs

image

Still present in v1.100.0

0x5e commented 1 week ago

I guess the total size is from linux machine on macos so it's not true. If we can't get it, it will be better display unknown instead of a wrong size.