search

immich-app / immich

High performance self-hosted photo and video management solution.
https://immich.app
GNU Affero General Public License v3.0
51.55k stars 2.73k forks source link

[BUG] Import a photo with a duplicate prompt, but cannot be found in the library folder and assets table #5254

Closed malanxi2022 closed 2 months ago

malanxi2022 commented 11 months ago

The bug

Import some photo with a duplicate prompt, but cannot be found in the library folder and assets table How do I determine whether it is a duplicate? Why are some photos in the local table, but not in the library? I tried to import the photos in the local table again, but it also prompted a duplicate.

The OS that Immich Server is running on

docker desktop

Version of Immich Server

v1.88.1

Version of Immich Mobile App

v1.88.1

Platform with the issue

Your docker-compose.yml content

no changes

Your .env content

no changes

Reproduction steps

1. Import the photo 
2. duplicate

Additional information

No response

alextran1502 commented 11 months ago

how are you importing the photos?

malanxi2022 commented 11 months ago

how are you importing the photos?你是如何导入照片的？

Upload photos through the UploadButton on the web page

tedstriker commented 11 months ago

got the same behaviour with version 1.90.2

It might be, that this photo has been in the library previously. If this was the case, it has been deleted before I tried to upload it again. The way I used to upload it, was by dropping it on an opened album.

Is there a way to search for orphan database entries?

mgreystone commented 10 months ago

I've got the same issue, i think. I uploaded a small number of photos through the web app. They are no where to be found. When i re-upload them, they are marked as duplicates. When i upload them to an album, it says they have been added to the album, but i cannot find them there either.

Curiously, though... one of the photos does appear as the album's thumbnail. So they are in the db somewhere. Anyway to force-upload and ignore the duplicate?

tedstriker commented 8 months ago

In Version 1.94.1 this error is still existing. I removed a broken tile from the photo library, because it pointed to a folder that did not exist. After that, I tried to re-upload it via the website and it gets denied with of a duplicate error message. I found, there's a "repair" section in the administration section of immich now. Also the problematic image is listed there, but unfortunately the documentation doesn't mention this new section yet and it's not intuitive to use. But I think we can have hope...

stephen304 commented 8 months ago

I'm also experiencing some "duplicates" when uploading through the web uploader, but I dragged and dropped from a single folder with ~2000 uniquely named files and for me it's a first time import of these files.

It's made harder to diagnose since the duplicate entries in the upload list go away as soon as it's detected as a duplicate, and there's no way to review which files were detected as duplicates unless you were screen recording the upload or remembering the filename as soon as you see the duplicate counter increment.

It would be great if after a batch upload finishes with errors or duplicates, if there was a persistent notification with a button to open a modal with a scrollable list of all the filenames that had issues. Then I would be able to inspect the exact files, check to see if they already exist somehow, or retry just those files.

jee-r commented 8 months ago

Is there any update on this ? I'm also stuck on this issue, have you found any workaround other than starting from scratch ?

MrColumbo commented 7 months ago

I am not sure if it is same on my side. It started actually with a problem with the reverse proxy where i got trouble to backup a large - 4.5GB video. I might have fixed the reverse proxy issue but when I try to upload the video via the browser I get a skipped dupliacate message.

On postgres i see this

06.04.2024
13:55:44
2024-04-06 11:55:44.346 UTC [2167] ERROR:  duplicate key value violates unique constraint "UQ_assets_owner_library_checksum"
06.04.2024
13:55:44
2024-04-06 11:55:44.346 UTC [2167] DETAIL:  Key ("ownerId", "libraryId", checksum)=(0ebe999a-0c1c-4331-92bd-4d6690b62629, 52db6a1c-261a-4eed-81c1-1af5c4bace01, \x8ead89ed7867fb940b99ee55816e26e69f7731cd) already exists.
06.04.2024
13:55:44
2024-04-06 11:55:44.346 UTC [2167] STATEMENT:  INSERT INTO "assets"("id", "deviceAssetId", "ownerId", "libraryId", "deviceId", "type", "originalPath", "previewPath", "thumbnailPath", "thumbhash", "encodedVideoPath", "createdAt", "updatedAt", "deletedAt", "fileCreatedAt", "localDateTime", "fileModifiedAt", "isFavorite", "isArchived", "isExternal", "isReadOnly", "isOffline", "checksum", "duration", "isVisible", "livePhotoVideoId", "originalFileName", "sidecarPath", "stackId") VALUES (DEFAULT, $1, $2, $3, $4, $5, $6, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, DEFAULT, $7, $8, $9, $10, $11, DEFAULT, $12, $13, $14, $15, $16, DEFAULT, $17, $18, DEFAULT) RETURNING "id", "thumbnailPath", "encodedVideoPath", "createdAt", "updatedAt", "deletedAt", "isFavorite", "isArchived", "isExternal", "isReadOnly", "isOffline", "isVisible"

Since I know the filename i did a search but i am not able to find the asset / video.

Oh - i should mention that i started to try uploading via the webintrface since the client does not even show that video anymore.

Basically when I look in the library and look into the local folder the app pretends that this particular video doet not exist anymore and there is nothing left to backup.

The repair button does not show anything ....

i am on 101 - mobile app and server

after a while i got this message on the fix page

504 Gateway Time-out

504 Gateway Time-out

openresty
alextran1502 commented 7 months ago

Please try double check in the trash/archive

MrColumbo commented 7 months ago

nothing in neither archive nor deleted ...

MrColumbo commented 7 months ago

i just found the file in the library ....so the file actually exists and one or the other way was successfully uploaded

MrColumbo commented 7 months ago

i fooled myself - the video is shown but with a duration of 0 seconds. All the other videos are showing with the right duration and the one i missed is about 35 minutes but since it showed 0 seconds i thought it is not the one i am looking for

ap-andersson commented 4 months ago

Please try double check in the trash/archive

This was my problem. I had "deleted" the images. When I created a new album and tried to re-upload them in that album they already existed (in trash) and was not visible anywhere. I emptied trash and tried again and it worked.