immich-app / immich

High performance self-hosted photo and video management solution.
https://immich.app
GNU Affero General Public License v3.0
45.06k stars 2.18k forks source link

Deduplication of video files, false positives, #9794

Closed GrumpyMeow closed 3 months ago

GrumpyMeow commented 3 months ago

The bug

I had 17000 duplicates in my library. I was already antipating the deduplication functionality. Many video-files (which i copied from VHS-tape) start with a few seconds of all-black screen. The deduplication-algorithm sees them all as duplicates, which i don't think it should. As this makes it very easy to delete files of which only the first frames match with each other. I don't think the algorithm should only take the first frame into account. The algorithm is accurate for photos, but very inaccurate for video-files. It's too easy for people to delete their video files by mistake.

I also noticed that the algorithm marks images (black.gif) and movie files (with first black frame) as duplicates. I don't think this should be the case.

The OS that Immich Server is running on

Debian 12

Version of Immich Server

v1.105.1

Version of Immich Mobile App

NA

Platform with the issue

Your docker-compose.yml content

NA

Your .env content

NA

Reproduction steps

NA

Relevant log output

No response

Additional information

No response

bo0tzz commented 3 months ago

9784

mertalev commented 3 months ago

Fixed via #9747 and #9784