immich-app / immich

High performance self-hosted photo and video management solution.
https://immich.app
GNU Affero General Public License v3.0
50.35k stars 2.67k forks source link

Immich autobackup fills up iPhone System Data #12594

Open sevvlor opened 1 month ago

sevvlor commented 1 month ago

The bug

The Immich backup feature that allows photos to be backed up to the remote server causes the system data on my iPhone 15 Pro Max with iOS 17.6.1 to be filled up completely. This causes the iPhone to notify me with "storage full" notifications.

Image describes the System Data that is filled. IMG_1877

The OS that Immich Server is running on

Debian 12

Version of Immich Server

1.114.0

Version of Immich Mobile App

1.114.0 build.173

Platform with the issue

Your docker-compose.yml content

#
# WARNING: Make sure to use the docker-compose.yml of the current release:
#
# https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
#
# The compose file on main may not be compatible with the latest release.
#

name: immich

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    # extends:
    #   file: hwaccel.transcoding.yml
    #   service: cpu # set to one of [nvenc, quicksync, rkmpp, vaapi, vaapi-wsl] for accelerated transcoding
    volumes:
      # Do not edit the next line. If you want to change the media storage location on your system, edit the value of UPLOAD_LOCATION in the .env file
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - stack.env
    ports:
      - 2283:3001
    depends_on:
      - redis
      - database
    restart: always

  immich-machine-learning:
    container_name: immich_machine_learning
    # For hardware acceleration, add one of -[armnn, cuda, openvino] to the image tag.
    # Example tag: ${IMMICH_VERSION:-release}-cuda
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
    # extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/ml-hardware-acceleration
    #   file: hwaccel.ml.yml
    #   service: cpu # set to one of [armnn, cuda, openvino, openvino-wsl] for accelerated inference - use the `-wsl` version for WSL2 where applicable
    volumes:
      - model-cache:/cache
    env_file:
      - stack.env
    restart: always

  redis:
    container_name: immich_redis
    image: docker.io/redis:6.2-alpine@sha256:e3b17ba9479deec4b7d1eeec1548a253acc5374d68d3b27937fcfe4df8d18c7e
    healthcheck:
      test: redis-cli ping || exit 1
    restart: always

  database:
    container_name: immich_postgres
    image: docker.io/tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
      POSTGRES_INITDB_ARGS: '--data-checksums'
    volumes:
      # Do not edit the next line. If you want to change the database storage location on your system, edit the value of DB_DATA_LOCATION in the .env file
      - ${DB_DATA_LOCATION}:/var/lib/postgresql/data
    healthcheck:
      test: pg_isready --dbname='${DB_DATABASE_NAME}' --username='${DB_USERNAME}' || exit 1; Chksum="$$(psql --dbname='${DB_DATABASE_NAME}' --username='${DB_USERNAME}' --tuples-only --no-align --command='SELECT COALESCE(SUM(checksum_failures), 0) FROM pg_stat_database')"; echo "checksum failure count is $$Chksum"; [ "$$Chksum" = '0' ] || exit 1
      interval: 5m
      start_interval: 30s
      start_period: 5m
    command: ["postgres", "-c" ,"shared_preload_libraries=vectors.so", "-c", 'search_path="$$user", public, vectors', "-c", "logging_collector=on", "-c", "max_wal_size=2GB", "-c", "shared_buffers=512MB", "-c", "wal_compression=on"]
    restart: always

volumes:
  model-cache:

Your .env content

UPLOAD_LOCATION=/immich/photos
DB_DATA_LOCATION=/immich/postgres
IMMICH_VERSION=release
DB_PASSWORD=hunter2
DB_USERNAME=postgres
DB_DATABASE_NAME=immich

Reproduction steps

  1. Enable or run the backup functionality
  2. See the System Data that's being filled

Relevant log output

No response

Additional information

No response

alextran1502 commented 1 month ago

Can you help me check how much data/document storage does the app use?

sevvlor commented 1 month ago

Can you help me check how much data/document storage does the app use?

It's currently using 35,2MB for documents and data. If you want I can re-enable the backup functionality and report back.

Edit: I have granted it access to the photo library and it shot up to 11GB and is increasing.

IMG_1878

alextran1502 commented 1 month ago

@sevvlor Thanks, this is indeed an issue when the size inflated that much. Just to confirm that you are only seeing the size increase when running backup, correct?

sevvlor commented 1 month ago

It happened again. 118G of storage that's listed as System Data. To answer your question:

on my phone: 15772 photos and 520 videos: 16292 total on Immich: 20152 photos and 687 videos : 20839 total

Edit: I only see it when its backing up or when the feature is enabled. The data does not get removed after the backup has finished or gets turned off.

This graph also shows something interesting. Around 12 the storage gets filled rapidly. Bear in mind that the phone checks in every few hours so it might've taken an hour to fill.

1284575607484710928

alextran1502 commented 1 month ago

Are you using iCloud to store your media there? All 16292 items

sevvlor commented 1 month ago

@alextran1502 yes, I am.

hugoduraes commented 1 month ago

Came here to open an issue related to iPhone storage and found this one. I'm having the same issue but in my case the space is allocated to the Immich app, not the System Data: IMG_6813

A bit of context... I have around 6.5k images/videos on my phone, all backed up on iCloud. On the Immich app, I've selected the Recents (All) as the backup album but haven't started the backup. The issue seems to happen while indexing the assets from the selected album, which btw, takes a lot of time! If I don't select any backup album, the storage remains low. I've reinstalled the app several times and the issue remains.

This is happening on an iPhone 14 Pro, running iOS 18.0. As for Immich, I'm using server v1.116.2 and app v1.116.1 build 177.

Not sure if it helps but here are the app logs (the album names were redacted): Immich_log_2024-09-30T22:53:46.822743.log

dennisoderwald commented 1 month ago

I have exactly the same problems... (50k iPhone video/pictures mixed) and also use iCloud in parallel.

For me it is 19.06 GB that Immich uses...

hugoduraes commented 4 days ago

@alextran1502 any hint on what may be causing the issue? I've just tried this again with Immich v1.119 and the problem remains.

alextran1502 commented 4 days ago

@hugoduraes I think when the app upload the iCloud asset, the file is first download to the device before it can get upload to the server, however that downloaded content is now staying on the device

hugoduraes commented 3 days ago

@alextran1502 If that was the case, the occupied space would be assigned to Photos and not Immich, no?