[BUG] Deleting android motion photos cause storage usage to go negative

Ynng commented 7 months ago

The bug

When a user uploads a motion photo in the MP or MVIMG format, and then permanently deletes the photo, the amount of storage space "used" by the user decreases beyond the size of the jpg. The size of the video portion of the motion photo seems to be double counted.

Reproduction steps:

Create a user account with a storage quota.
Login as the user.
Upload a motion photo from the web GUI, here's an example https://files.catbox.moe/b59ftp.jpg
Permanently delete the motion photo.
Observe that your "used storage" value becomes negative

Here's a video demonstrating the reproduction steps:

https://github.com/immich-app/immich/assets/44907675/f1665c6a-7a02-4b6e-9655-571aaa762b36

The "used storage" value goes back to 0 correctly if I change the user account's storage quota.

The OS that Immich Server is running on

Fedora Linux 37

Version of Immich Server

v1.100.0

Version of Immich Mobile App

v1.100.0

Platform with the issue

[X] Server
[X] Web
[ ] Mobile

Your docker-compose.yml content

version: '3.8'

#
# WARNING: Make sure to use the docker-compose.yml of the current release:
#
# https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
#
# The compose file on main may not be compatible with the latest release.
#

name: immich

services:
  immich-prometheus:
    container_name: immich_prometheus
    ports:
      # this exposes the default port for Prometheus so you can interact with it
      - 9090:9090
    image: prom/prometheus
    volumes:
      # the Prometheus configuration file - a barebones one is provided to get started
      - ./prometheus.yml:/etc/prometheus/prometheus.yml
      # a named volume defined in the bottom of the Compose file; it can also be a mounted folder
      - prometheus-data:/prometheus

  immich-grafana:
    container_name: immich_grafana
    command: ['./run.sh', '-disable-reporting'] # this is to disable Grafana's telemetry
    ports:
      - 3000:3000
    image: grafana/grafana
    volumes:
      # stores your pretty dashboards and panels
      - grafana-data:/var/lib/grafana

  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    command: ['start.sh', 'immich']
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
      - ${EXTERNAL_PATH}:/usr/src/app/external
    env_file:
      - .env
    ports:
      - 2283:3001
    depends_on:
      - redis
      - database
    restart: always

  immich-microservices:
    container_name: immich_microservices
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/hardware-transcoding
      file: hwaccel.transcoding.yml
      service: nvenc # set to one of [nvenc, quicksync, rkmpp, vaapi, vaapi-wsl] for accelerated transcoding
    command: ['start.sh', 'microservices']
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
      - ${EXTERNAL_PATH}:/usr/src/app/external
    env_file:
      - .env
    depends_on:
      - redis
      - database
    restart: always

  immich-machine-learning:
    container_name: immich_machine_learning
    # For hardware acceleration, add one of -[armnn, cuda, openvino] to the image tag.
    # Example tag: ${IMMICH_VERSION:-release}-cuda
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}-cuda
    extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/ml-hardware-acceleration
      file: hwaccel.ml.yml
      service: cuda # set to one of [armnn, cuda, openvino, openvino-wsl] for accelerated inference - use the `-wsl` version for WSL2 where applicable
    volumes:
      - model-cache:/cache
    env_file:
      - .env
    restart: always

  redis:
    container_name: immich_redis
    image: registry.hub.docker.com/library/redis:6.2-alpine@sha256:51d6c56749a4243096327e3fb964a48ed92254357108449cb6e23999c37773c5
    restart: always

  database:
    container_name: immich_postgres
    image: registry.hub.docker.com/tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
    volumes:
      - pgdata:/var/lib/postgresql/data
    restart: always

volumes:
  pgdata:
  model-cache:
  prometheus-data:
  grafana-data:

Your .env content

# You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables

# The location where your uploaded files are stored
UPLOAD_LOCATION=<redacted>

# The Immich version to use. You can pin this to a specific version like "v1.71.0"
IMMICH_VERSION=release

# Connection secret for postgres. You should change it to a random password
DB_PASSWORD=<redacted>

IMMICH_METRICS=true
EXTERNAL_PATH=<redacted>

# The values below this line do not need to be changed
###################################################################################
DB_HOSTNAME=immich_postgres
DB_USERNAME=postgres
DB_DATABASE_NAME=immich

REDIS_HOSTNAME=immich_redis
LOG_LEVEL=debug

Reproduction steps

See above

Additional information

No response

Ynng commented 7 months ago

Looks like it might be related to #8202

Ynng commented 7 months ago

I narrowed the issue down with some debugging: Android motion photos have videos embedded in the jpg. When a motion photo is uploaded, the video is extracted and added as a separate asset, taking up more space. This extra usage is not added to quotaUsageInBytes

The erroneously usage is corrected by syncUsage when the user's quota is updated

immich-app / immich