immich-app / immich

High performance self-hosted photo and video management solution.
https://immich.app
GNU Affero General Public License v3.0
51.09k stars 2.7k forks source link

[BUG] After importing everything through the CLI, the app still tries to upload every file #3316

Closed gcarrarom closed 1 year ago

gcarrarom commented 1 year ago

The bug

I've uploaded all my photos by using the CLI, which worked perfectly fine. Now the issue is that whenever I open the immich app on my phone, it still tries to upload all the images. This results in poor performance of the app and constant usage trying to upload thousands of photos that are already there, only to respond with this error on the server side:

immich/immich-typesense-6bf6bfb8b9-nf8wq[immich-typesense]: I20230717 16:17:25.044348   181 raft_server.h:60] Peer refresh succeeded!
immich/immich-microservices-7b866f9584-lzkv6[immich-microservices]: [Nest] 7  - 07/17/2023, 4:17:25 PM   ERROR [TypesenseRepository] Unable to index documents
immich/immich-postgresql-0[postgresql]: 2023-07-17 16:17:28.242 GMT [13981] ERROR:  duplicate key value violates unique constraint "UQ_userid_checksum"
immich/immich-postgresql-0[postgresql]: 2023-07-17 16:17:28.242 GMT [13981] DETAIL:  Key ("ownerId", checksum)=(cd4afb29-fb65-4511-9827-947542f03fc8, \xe79ba9bbe3568621486219b06b1f5ec9807fb6e9) already exists.
immich/immich-postgresql-0[postgresql]: 2023-07-17 16:17:28.242 GMT [13981] STATEMENT:  INSERT INTO "assets"("id", "deviceAssetId", "ownerId", "deviceId", "type", "originalPath", "resizePath", "webpPath", "thumbhash", "encodedVideoPath", "createdAt", "updatedAt", "fileCreatedAt", "fileModifiedAt", "isFavorite", "isArchived", "isReadOnly", "checksum", "duration", "isVisible", "livePhotoVideoId", "originalFileName", "sidecarPath") VALUES (DEFAULT, $1, $2, $3, $4, $5, $6, $7, $8, $9, DEFAULT, DEFAULT, $10, $11, $12, $13, $14, $15, $16, $17, DEFAULT, $18, $19) RETURNING "id", "webpPath", "encodedVideoPath", "createdAt", "updatedAt", "isFavorite", "isArchived", "isReadOnly", "isVisible"
immich/immich-typesense-6bf6bfb8b9-nf8wq[immich-typesense]: E20230717 16:17:29.044747   165 raft_server.cpp:635] 512 queued writes > healthy write lag of 500
immich/immich-server-7c58fcff6-p9bsc[immich-server]: [Nest] 8  - 07/17/2023, 4:17:30 PM     LOG [CommunicationGateway] Client FpRgXQt-j_9ndPKTAAAB disconnected from Websocket
immich/immich-microservices-7b866f9584-lzkv6[immich-microservices]: [Nest] 7  - 07/17/2023, 4:17:30 PM   ERROR [TypesenseRepository] Unable to index documents
immich/immich-typesense-6bf6bfb8b9-nf8wq[immich-typesense]: I20230717 16:17:35.045292   165 raft_server.cpp:545] Term: 1069, last_index index: 12281, committed_index: 12281, known_applied_index: 12281, applying_index: 0, queued_writes: 512, pending_queue_size: 0, local_sequence: 237094

The OS that Immich Server is running on

Kubernetes - k3s - MicroOS

Version of Immich Server

v1.67.2

Version of Immich Mobile App

1.67.0 build.107

Platform with the issue

Your docker-compose.yml content

apiVersion: helm.toolkit.fluxcd.io/v2beta1
kind: HelmRelease
metadata:
  name: immich
  namespace: immich
spec:
  interval: 5m
  chart:
    spec:
      chart: immich
      sourceRef:
        kind: HelmRepository
        name: immich
        namespace: fancywhale
      interval: 1m
  values:

    postgresql:
      enabled: true
    redis:
      enabled: true

    typesense:
      enabled: true
      persistence:
        tsdata:
          enabled: true
          existingClaim: typesense-data

    machine-learning:
      enabled: true
      persistence:
        cache:
          enabled: true
          existingClaim: machinelearning-data
    proxy:
      ingress:
        main:
          enabled: true
          ingressClassName: nginx
          annotations:
            # proxy-body-size is set to 0 to remove the body limit on file uploads
            nginx.ingress.kubernetes.io/proxy-body-size: "0"
            cert-manager.io/cluster-issuer: letsencrypt
          hosts:
            - host: $MY_HOSTNAME
              paths:
                - path: "/"
          tls:
            - hosts:
                - $MY_HOSTNAME
              secretName: $MY_HOSTNAME-tls
    image:
      tag: v1.67.2
    immich:
      persistence:
        # Main data store for all photos shared between different components.
        library:
          existingClaim: photos

Your .env content

N/A

Reproduction steps

1. Upload images from the phone using the CLI on a computer.
2. Open the app on the phone and wait for the sync
3. Check that the images are still pending upload.

Additional information

No response

alextran1502 commented 1 year ago

We haven't implemented the use of client-side hashing yet on the mobile app, so it will still upload those assets and have the server check for the hash and then reject it. Then those files will be blacklisted and won't be uploaded again. Please follow this for more info

https://github.com/immich-app/immich/issues/2567