BackUp Again all items after reinstalling Android App

cristian98149 commented 5 months ago

The bug

After reinstalling Android App and selected the same albums like before, backup starts for all items again so duplicated are generated and the app suggest me to clear them but after I delete duplicated, backup starts again for all items.

The OS that Immich Server is running on

Ubuntu 24.04

Version of Immich Server

v1.105.0

Version of Immich Mobile App

v1.105.0

Platform with the issue

[ ] Server
[ ] Web
[X] Mobile

Your docker-compose.yml content

#
# WARNING: Make sure to use the docker-compose.yml of the current release:
#
# https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
#
# The compose file on main may not be compatible with the latest release.
#

name: immich

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    command: ['start.sh', 'immich']
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env
    ports:
      - 2283:3001
    depends_on:
      - redis
      - database
    restart: always

  immich-microservices:
    container_name: immich_microservices
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    # extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/hardware-transcoding
    #   file: hwaccel.transcoding.yml
    #   service: cpu # set to one of [nvenc, quicksync, rkmpp, vaapi, vaapi-wsl] for accelerated transcoding
    command: ['start.sh', 'microservices']
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env
    depends_on:
      - redis
      - database
    restart: always

  immich-machine-learning:
    container_name: immich_machine_learning
    # For hardware acceleration, add one of -[armnn, cuda, openvino] to the image tag.
    # Example tag: ${IMMICH_VERSION:-release}-cuda
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
    # extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/ml-hardware-acceleration
    #   file: hwaccel.ml.yml
    #   service: cpu # set to one of [armnn, cuda, openvino, openvino-wsl] for accelerated inference - use the `-wsl` version for WSL2 where applicable
    volumes:
      - model-cache:/cache
    env_file:
      - .env
    restart: always

  redis:
    container_name: immich_redis
    image: registry.hub.docker.com/library/redis:6.2-alpine@sha256:84882e87b54734154586e5f8abd4dce69fe7311315e2fc6d67c29614c8de2672
    restart: always

  database:
    container_name: immich_postgres
    image: registry.hub.docker.com/tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
    volumes:
      - ${DB_DATA_LOCATION}:/var/lib/postgresql/data
    restart: always

volumes:
  model-cache:

Your .env content

# You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables

# The location where your uploaded files are stored
UPLOAD_LOCATION=./library
# The location where your database files are stored
DB_DATA_LOCATION=./postgres

# The Immich version to use. You can pin this to a specific version like "v1.71.0"
IMMICH_VERSION=release

# Connection secret for postgres. You should change it to a random password
DB_PASSWORD=postgres

# The values below this line do not need to be changed
###################################################################################
DB_USERNAME=postgres
DB_DATABASE_NAME=immich

Reproduction steps

1. Install android app
2. Select albums for backup
3. Backup all items
4. Delete the app
5. Reinstall the app
6. Select the same albums for backup

Relevant log output

No response

Additional information

No response

alextran1502 commented 5 months ago

Did you change your installation method i.e moving from Google Playstore to APK or Fdroid?

cristian98149 commented 5 months ago

Hi @alextran1502,

I've a Huawei smartphone without Google play services so I use Aurora Store to manage some apps that is not on Huawei app Gallery.

When I tried to update the immich app, aurora store displayed and error saying the apk was not compatibile with an already installed one so I deleted immich app and reinstalled it with the new apk.

Deathproof76 commented 5 months ago

I experience the same on v1.105.1. From the Google Playstore on a Samsung S23U. When I tried to use the android app after updating (most likely from v1.104) I just got a spinning wheel "building the timeline". Deleting cache didn't help and I couldn't make out anything significant in the logs. Web was working perfectly. So I reinstalled the mobile app completely. After login and setting up the backup albums, just like before spinning Wheel, but it started re-uploading all my already backed up files. Will try to produce and add logs tomorrow (currently midnight here).

postgres doesn't seem happy: ERROR: duplicate key value violates unique constraint "UQ_assets_owner_library_checksum" I'm using https://github.com/imagegenius/docker-immich btw

postgres16  | 2024-05-16 00:40:17.008 CEST [28919] ERROR:  duplicate key value violates unique constraint "UQ_assets_owner_library_checksum"
postgres16  | 2024-05-16 00:40:17.008 CEST [28919] DETAIL:  Key ("ownerId", "libraryId", checksum)=(4d2fd6f2-0050-466d-ba32-c0b77244ac65, 784fd177-6244-4870-81f0-07754d305290, \xeabd6dc86aa4630d64cd8b361fb7508b0f9a6e41) already exists.
postgres16  | 2024-05-16 00:40:17.008 CEST [28919] STATEMENT:  INSERT INTO "assets"("id", "deviceAssetId", "ownerId", "libraryId", "deviceId", "type", "originalPath", "previewPath", "thumbnailPath", "thumbhash", "encodedVideoPath", "createdAt", "updatedAt", "deletedAt", "fileCreatedAt", "localDateTime", "fileModifiedAt", "isFavorite", "isArchived", "isExternal", "isOffline", "checksum", "duration", "isVisible", "livePhotoVideoId", "originalFileName", "sidecarPath", "stackId") VALUES (DEFAULT, $1, $2, $3, $4, $5, $6, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, $7, $8, $9, $10, $11, DEFAULT, $12, $13, $14, $15, DEFAULT, $16, $17, DEFAULT) RETURNING "id", "thumbnailPath", "encodedVideoPath", "createdAt", "updatedAt", "deletedAt", "isFavorite", "isArchived", "isExternal", "isOffline", "isVisible"
postgres16  | 2024-05-16 00:40:25.839 CEST [28919] ERROR:  duplicate key value violates unique constraint "UQ_assets_owner_library_checksum"
postgres16  | 2024-05-16 00:40:25.839 CEST [28919] DETAIL:  Key ("ownerId", "libraryId", checksum)=(4d2fd6f2-0050-466d-ba32-c0b77244ac65, 784fd177-6244-4870-81f0-07754d305290, \xdd23750605816294af6221111cf856de84beb5eb) already exists.
postgres16  | 2024-05-16 00:40:25.839 CEST [28919] STATEMENT:  INSERT INTO "assets"("id", "deviceAssetId", "ownerId", "libraryId", "deviceId", "type", "originalPath", "previewPath", "thumbnailPath", "thumbhash", "encodedVideoPath", "createdAt", "updatedAt", "deletedAt", "fileCreatedAt", "localDateTime", "fileModifiedAt", "isFavorite", "isArchived", "isExternal", "isOffline", "checksum", "duration", "isVisible", "livePhotoVideoId", "originalFileName", "sidecarPath", "stackId") VALUES (DEFAULT, $1, $2, $3, $4, $5, $6, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, $7, $8, $9, $10, $11, DEFAULT, $12, $13, $14, $15, DEFAULT, $16, $17, DEFAULT) RETURNING "id", "thumbnailPath", "encodedVideoPath", "createdAt", "updatedAt", "deletedAt", "isFavorite", "isArchived", "isExternal", "isOffline", "isVisible"
postgres16  | 2024-05-16 00:40:37.931 CEST [28919] ERROR:  duplicate key value violates unique constraint "UQ_assets_owner_library_checksum"
postgres16  | 2024-05-16 00:40:37.931 CEST [28919] DETAIL:  Key ("ownerId", "libraryId", checksum)=(4d2fd6f2-0050-466d-ba32-c0b77244ac65, 784fd177-6244-4870-81f0-07754d305290, \x88defea5a5f5409df6e40659a27af0b36cf2a422) already exists.
postgres16  | 2024-05-16 00:40:37.931 CEST [28919] STATEMENT:  INSERT INTO "assets"("id", "deviceAssetId", "ownerId", "libraryId", "deviceId", "type", "originalPath", "previewPath", "thumbnailPath", "thumbhash", "encodedVideoPath", "createdAt", "updatedAt", "deletedAt", "fileCreatedAt", "localDateTime", "fileModifiedAt", "isFavorite", "isArchived", "isExternal", "isOffline", "checksum", "duration", "isVisible", "livePhotoVideoId", "originalFileName", "sidecarPath", "stackId") VALUES (DEFAULT, $1, $2, $3, $4, $5, $6, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, $7, $8, $9, $10, $11, DEFAULT, $12, $13, $14, $15, DEFAULT, $16, $17, DEFAULT) RETURNING "id", "thumbnailPath", "encodedVideoPath", "createdAt", "updatedAt", "deletedAt", "isFavorite", "isArchived", "isExternal", "isOffline", "isVisible"
postgres16  | 2024-05-16 00:40:38.808 CEST [28919] ERROR:  duplicate key value violates unique constraint "UQ_assets_owner_library_checksum"
postgres16  | 2024-05-16 00:40:38.808 CEST [28919] DETAIL:  Key ("ownerId", "libraryId", checksum)=(4d2fd6f2-0050-466d-ba32-c0b77244ac65, 784fd177-6244-4870-81f0-07754d305290, \x44c096c2ce83417cc668b4beff3eeeac4113fec9) already exists.
postgres16  | 2024-05-16 00:40:38.808 CEST [28919] STATEMENT:  INSERT INTO "assets"("id", "deviceAssetId", "ownerId", "libraryId", "deviceId", "type", "originalPath", "previewPath", "thumbnailPath", "thumbhash", "encodedVideoPath", "createdAt", "updatedAt", "deletedAt", "fileCreatedAt", "localDateTime", "fileModifiedAt", "isFavorite", "isArchived", "isExternal", "isOffline", "checksum", "duration", "isVisible", "livePhotoVideoId", "originalFileName", "sidecarPath", "stackId") VALUES (DEFAULT, $1, $2, $3, $4, $5, $6, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, $7, $8, $9, $10, $11, DEFAULT, $12, $13, $14, $15, DEFAULT, $16, $17, DEFAULT) RETURNING "id", "thumbnailPath", "encodedVideoPath", "createdAt", "updatedAt", "deletedAt", "isFavorite", "isArchived", "isExternal", "isOffline", "isVisible"
postgres16  | 2024-05-16 00:40:39.278 CEST [28933] ERROR:  duplicate key value violates unique constraint "UQ_assets_owner_library_checksum"
postgres16  | 2024-05-16 00:40:39.278 CEST [28933] DETAIL:  Key ("ownerId", "libraryId", checksum)=(4d2fd6f2-0050-466d-ba32-c0b77244ac65, 784fd177-6244-4870-81f0-07754d305290, \xf76465a672a2ef851c99d9c8f08375bf1bffb01d) already exists.
postgres16  | 2024-05-16 00:40:39.278 CEST [28933] STATEMENT:  INSERT INTO "assets"("id", "deviceAssetId", "ownerId", "libraryId", "deviceId", "type", "originalPath", "previewPath", "thumbnailPath", "thumbhash", "encodedVideoPath", "createdAt", "updatedAt", "deletedAt", "fileCreatedAt", "localDateTime", "fileModifiedAt", "isFavorite", "isArchived", "isExternal", "isOffline", "checksum", "duration", "isVisible", "livePhotoVideoId", "originalFileName", "sidecarPath", "stackId") VALUES (DEFAULT, $1, $2, $3, $4, $5, $6, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, $7, $8, $9, $10, $11, DEFAULT, $12, $13, $14, $15, DEFAULT, $16, $17, DEFAULT) RETURNING "id", "thumbnailPath", "encodedVideoPath", "createdAt", "updatedAt", "deletedAt", "isFavorite", "isArchived", "isExternal", "isOffline", "isVisible"
postgres16  | 2024-05-16 00:40:47.025 CEST [28919] ERROR:  duplicate key value violates unique constraint "UQ_assets_owner_library_checksum"
postgres16  | 2024-05-16 00:40:47.025 CEST [28919] DETAIL:  Key ("ownerId", "libraryId", checksum)=(4d2fd6f2-0050-466d-ba32-c0b77244ac65, 784fd177-6244-4870-81f0-07754d305290, \x142a8c1e7ed4a4b92b0fc571408d87003eb1fbd3) already exists.
postgres16  | 2024-05-16 00:40:47.025 CEST [28919] STATEMENT:  INSERT INTO "assets"("id", "deviceAssetId", "ownerId", "libraryId", "deviceId", "type", "originalPath", "previewPath", "thumbnailPath", "thumbhash", "encodedVideoPath", "createdAt", "updatedAt", "deletedAt", "fileCreatedAt", "localDateTime", "fileModifiedAt", "isFavorite", "isArchived", "isExternal", "isOffline", "checksum", "duration", "isVisible", "livePhotoVideoId", "originalFileName", "sidecarPath", "stackId") VALUES (DEFAULT, $1, $2, $3, $4, $5, $6, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, $7, $8, $9, $10, $11, DEFAULT, $12, $13, $14, $15, DEFAULT, $16, $17, DEFAULT) RETURNING "id", "thumbnailPath", "encodedVideoPath", "createdAt", "updatedAt", "deletedAt", "isFavorite", "isArchived", "isExternal", "isOffline", "isVisible"
postgres16  | 2024-05-16 00:40:50.418 CEST [28933] ERROR:  duplicate key value violates unique constraint "UQ_assets_owner_library_checksum"
postgres16  | 2024-05-16 00:40:50.418 CEST [28933] DETAIL:  Key ("ownerId", "libraryId", checksum)=(4d2fd6f2-0050-466d-ba32-c0b77244ac65, 784fd177-6244-4870-81f0-07754d305290, \xf0fabef59854f24eb5cf56900e7c786ae40e8119) already exists.
postgres16  | 2024-05-16 00:40:50.418 CEST [28933] STATEMENT:  INSERT INTO "assets"("id", "deviceAssetId", "ownerId", "libraryId", "deviceId", "type", "originalPath", "previewPath", "thumbnailPath", "thumbhash", "encodedVideoPath", "createdAt", "updatedAt", "deletedAt", "fileCreatedAt", "localDateTime", "fileModifiedAt", "isFavorite", "isArchived", "isExternal", "isOffline", "checksum", "duration", "isVisible", "livePhotoVideoId", "originalFileName", "sidecarPath", "stackId") VALUES (DEFAULT, $1, $2, $3, $4, $5, $6, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, $7, $8, $9, $10, $11, DEFAULT, $12, $13, $14, $15, DEFAULT, $16, $17, DEFAULT) RETURNING "id", "thumbnailPath", "encodedVideoPath", "createdAt", "updatedAt", "deletedAt", "isFavorite", "isArchived", "isExternal", "isOffline", "isVisible"
postgres16  | 2024-05-16 00:40:54.423 CEST [28919] ERROR:  duplicate key value violates unique constraint "UQ_assets_owner_library_checksum"
postgres16  | 2024-05-16 00:40:54.423 CEST [28919] DETAIL:  Key ("ownerId", "libraryId", checksum)=(4d2fd6f2-0050-466d-ba32-c0b77244ac65, 784fd177-6244-4870-81f0-07754d305290, \xaa75742d0f2d065b8d9009b16043b12868e86b22) already exists.
postgres16  | 2024-05-16 00:40:54.423 CEST [28919] STATEMENT:  INSERT INTO "assets"("id", "deviceAssetId", "ownerId", "libraryId", "deviceId", "type", "originalPath", "previewPath", "thumbnailPath", "thumbhash", "encodedVideoPath", "createdAt", "updatedAt", "deletedAt", "fileCreatedAt", "localDateTime", "fileModifiedAt", "isFavorite", "isArchived", "isExternal", "isOffline", "checksum", "duration", "isVisible", "livePhotoVideoId", "originalFileName", "sidecarPath", "stackId") VALUES (DEFAULT, $1, $2, $3, $4, $5, $6, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, $7, $8, $9, $10, $11, DEFAULT, $12, $13, $14, $15, DEFAULT, $16, $17, DEFAULT) RETURNING "id", "thumbnailPath", "encodedVideoPath", "createdAt", "updatedAt", "deletedAt", "isFavorite", "isArchived", "isExternal", "isOffline", "isVisible"
postgres16  | 2024-05-16 00:40:56.750 CEST [28933] ERROR:  duplicate key value violates unique constraint "UQ_assets_owner_library_checksum"
postgres16  | 2024-05-16 00:40:56.750 CEST [28933] DETAIL:  Key ("ownerId", "libraryId", checksum)=(4d2fd6f2-0050-466d-ba32-c0b77244ac65, 784fd177-6244-4870-81f0-07754d305290, \xaa75742d0f2d065b8d9009b16043b12868e86b22) already exists.
postgres16  | 2024-05-16 00:40:56.750 CEST [28933] STATEMENT:  INSERT INTO "assets"("id", "deviceAssetId", "ownerId", "libraryId", "deviceId", "type", "originalPath", "previewPath", "thumbnailPath", "thumbhash", "encodedVideoPath", "createdAt", "updatedAt", "deletedAt", "fileCreatedAt", "localDateTime", "fileModifiedAt", "isFavorite", "isArchived", "isExternal", "isOffline", "checksum", "duration", "isVisible", "livePhotoVideoId", "originalFileName", "sidecarPath", "stackId") VALUES (DEFAULT, $1, $2, $3, $4, $5, $6, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, $7, $8, $9, $10, $11, DEFAULT, $12, $13, $14, $15, DEFAULT, $16, $17, DEFAULT) RETURNING "id", "thumbnailPath", "encodedVideoPath", "createdAt", "updatedAt", "deletedAt", "isFavorite", "isArchived", "isExternal", "isOffline", "isVisible"
postgres16  | 2024-05-16 00:41:07.302 CEST [28933] ERROR:  duplicate key value violates unique constraint "UQ_assets_owner_library_checksum"
postgres16  | 2024-05-16 00:41:07.302 CEST [28933] DETAIL:  Key ("ownerId", "libraryId", checksum)=(4d2fd6f2-0050-466d-ba32-c0b77244ac65, 784fd177-6244-4870-81f0-07754d305290, \xf0fabef59854f24eb5cf56900e7c786ae40e8119) already exists.
postgres16  | 2024-05-16 00:41:07.302 CEST [28933] STATEMENT:  INSERT INTO "assets"("id", "deviceAssetId", "ownerId", "libraryId", "deviceId", "type", "originalPath", "previewPath", "thumbnailPath", "thumbhash", "encodedVideoPath", "createdAt", "updatedAt", "deletedAt", "fileCreatedAt", "localDateTime", "fileModifiedAt", "isFavorite", "isArchived", "isExternal", "isOffline", "checksum", "duration", "isVisible", "livePhotoVideoId", "originalFileName", "sidecarPath", "stackId") VALUES (DEFAULT, $1, $2, $3, $4, $5, $6, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, $7, $8, $9, $10, $11, DEFAULT, $12, $13, $14, $15, DEFAULT, $16, $17, DEFAULT) RETURNING "id", "thumbnailPath", "encodedVideoPath", "createdAt", "updatedAt", "deletedAt", "isFavorite", "isArchived", "isExternal", "isOffline", "isVisible"
postgres16  | 2024-05-16 00:42:13.538 CEST [28940] ERROR:  duplicate key value violates unique constraint "UQ_assets_owner_library_checksum"
postgres16  | 2024-05-16 00:42:13.538 CEST [28940] DETAIL:  Key ("ownerId", "libraryId", checksum)=(4d2fd6f2-0050-466d-ba32-c0b77244ac65, 784fd177-6244-4870-81f0-07754d305290, \xdabc0505c1aed8ce3cb8dd32b61c9980a87e0feb) already exists.
postgres16  | 2024-05-16 00:42:13.538 CEST [28940] STATEMENT:  INSERT INTO "assets"("id", "deviceAssetId", "ownerId", "libraryId", "deviceId", "type", "originalPath", "previewPath", "thumbnailPath", "thumbhash", "encodedVideoPath", "createdAt", "updatedAt", "deletedAt", "fileCreatedAt", "localDateTime", "fileModifiedAt", "isFavorite", "isArchived", "isExternal", "isOffline", "checksum", "duration", "isVisible", "livePhotoVideoId", "originalFileName", "sidecarPath", "stackId") VALUES (DEFAULT, $1, $2, $3, $4, $5, $6, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, $7, $8, $9, $10, $11, DEFAULT, $12, $13, $14, $15, DEFAULT, $16, $17, DEFAULT) RETURNING "id", "thumbnailPath", "encodedVideoPath", "createdAt", "updatedAt", "deletedAt", "isFavorite", "isArchived", "isExternal", "isOffline", "isVisible"
redis2      | 1:M 15 May 2024 22:42:36.040 * 100 changes in 300 seconds. Saving...
redis2      | 1:M 15 May 2024 22:42:36.040 * Background saving started by pid 308
redis2      | 308:C 15 May 2024 22:42:36.058 * DB saved on disk
redis2      | 308:C 15 May 2024 22:42:36.059 * Fork CoW for RDB: current 1 MB, peak 1 MB, average 0 MB
redis2      | 1:M 15 May 2024 22:42:36.141 * Background saving terminated with success
postgres16  | 2024-05-16 00:42:37.152 CEST [28] LOG:  checkpoint complete: wrote 8391 buffers (51.2%); 0 WAL file(s) added, 7 removed, 0 recycled; write=269.998 s, sync=0.031 s, total=270.061 s; sync files=32, longest=0.008 s, average=0.001 s; distance=117455 kB, estimate=117455 kB; lsn=2/2DAC0C28, redo lsn=2/2D8DAF40

Immich_log_2024-05-16T00_51_10.057660.log

(I changed my domain name, hope that's sufficient)

alextran1502 commented 5 months ago

@cristian98149 this is expected behavior, changing the installation method will cause the device Id to be different which is used to distinguish assets from each device at the moment. So having new device id will cause the app to reupload.

alextran1502 commented 5 months ago

@Deathproof76 your issue is different from this one. And it seems like your instance have some problem. Have you managed to manually do something with the database recently?

Deathproof76 commented 5 months ago

@Deathproof76 your issue is different from this one. And it seems like your instance have some problem. Have you managed to manually do something with the database recently?

@alextran1502 https://github.com/vale46n1/immich_duplicate_finder yes, actually. I used this to find and delete some duplicate photos 🤦 I should have a backup of the db, will try to restore later today

Edit: It currently looks like my backup policy for the db was flawed. I don't seem to have any backups from before I used duplicate finder. Do you have any guidance? Sorry to bother you for this

immich-app / immich