immich-app / immich

High performance self-hosted photo and video management solution.
https://immich.app
GNU Affero General Public License v3.0
49.94k stars 2.65k forks source link

Duplicate files, remainder is 0, but files are still being uploaded #9510

Closed navennn closed 5 months ago

navennn commented 5 months ago

The bug

Hi I'm having a difficult time understanding the 'duplicates', and how backup upload works. I've some 8200 photos and videos on my android phone that I want to backup up with Immich. Initially, "Total" (assets in backup tab), said 8190 (approx. around this much).

Today, finally I've reached 0 on the remainder (quite poor connection to server, usually 100-2000kbps, upload took like a week). But files are still being uploaded. I assume these are the 'duplicates'? AFAIK, I do not have any duplicates, probably not even one.

I've read https://github.com/immich-app/immich/issues/9365 and https://github.com/immich-app/immich/issues/7738 which did not actually explain anything. I'm quite a bit confused about this. 8197−7224 = 973, not 1063 (count of duplicates) - see screenshots at the bottom of the post

Could I view the duplicates somehow? Are duplicates being uploaded as well?

This was this morning (the first time the counter hit 0 remaining) Screenshot_2024-05-15-14-27-16-26_d8aea95ef633eb6b7eb1cae13c34b40e

This is now Screenshot_2024-05-15-16-36-25-48_d8aea95ef633eb6b7eb1cae13c34b40e

Duplicates in settings Screenshot_2024-05-15-16-36-18-63_d8aea95ef633eb6b7eb1cae13c34b40e

Thank you!

The OS that Immich Server is running on

Debian 11.9

Version of Immich Server

v1.105.1

Version of Immich Mobile App

v1.105.0

Platform with the issue

Your docker-compose.yml content

#
# WARNING: Make sure to use the docker-compose.yml of the current release:
#
# https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
#
# The compose file on main may not be compatible with the latest release.
#

name: immich

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    command: ['start.sh', 'immich']
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env
    ports:
      - 2283:3001
    depends_on:
      - redis
      - database
    restart: always

  immich-microservices:
    container_name: immich_microservices
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    # extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/hardware-transcoding
    #   file: hwaccel.transcoding.yml
    #   service: cpu # set to one of [nvenc, quicksync, rkmpp, vaapi, vaapi-wsl] for accelerated transcoding
    command: ['start.sh', 'microservices']
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env
    depends_on:
      - redis
      - database
    restart: always

  immich-machine-learning:
    container_name: immich_machine_learning
    # For hardware acceleration, add one of -[armnn, cuda, openvino] to the image tag.
    # Example tag: ${IMMICH_VERSION:-release}-cuda
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
    # extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/ml-hardware-acceleration
    #   file: hwaccel.ml.yml
    #   service: cpu # set to one of [armnn, cuda, openvino, openvino-wsl] for accelerated inference - use the `-wsl` version for WSL2 where applicable
    volumes:
      - model-cache:/cache
    env_file:
      - .env
    restart: always

  redis:
    container_name: immich_redis
    image: registry.hub.docker.com/library/redis:6.2-alpine@sha256:84882e87b54734154586e5f8abd4dce69fe7311315e2fc6d67c29614c8de2672
    restart: always

  database:
    container_name: immich_postgres
    image: registry.hub.docker.com/tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
    volumes:
      - ${DB_DATA_LOCATION}:/var/lib/postgresql/data
    restart: always

volumes:
  model-cache:

Your .env content

# You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables

# The location where your uploaded files are stored
UPLOAD_LOCATION=/media/immich/
# The location where your database files are stored
DB_DATA_LOCATION=./postgres

# The Immich version to use. You can pin this to a specific version like "v1.71.0"
IMMICH_VERSION=release

# Connection secret for postgres. You should change it to a random password
DB_PASSWORD=xxxx

# The values below this line do not need to be changed
###################################################################################
DB_USERNAME=postgres
DB_DATABASE_NAME=immich

Reproduction steps

1. Do fresh install
2. Wait for all files to upload, wait for it to finish
3. Open backup tab
4. Open settings  -> advanced
...

Relevant log output

No response

Additional information

Phone is OnePlus 9, Android 14. App installed from google play store.

danieldietzler commented 5 months ago

This seems to be the typical backup state issue the app currently has

frejaya commented 5 months ago

This seems to be the typical backup state issue the app currently has

@danieldietzler

Is there another issue number where we can track the resolution?

danieldietzler commented 5 months ago

@frejaya #2567