Improve duplicate image detection

[teolemon]

What

• Improve duplicate image detection to detect duplicates with byte size difference that are not perceivable.
• We should have a tolerance threshold to be defined (1%, 5%)
• We should also backprocess the 2 last years of edits by yuka

  Why

• We lack disk space, and since Yuka (mostly) has had this behaviour for at least 2 years, there are sizeable savings to be made.

https://world.openfoodfacts.org/product/3173289105708/quadro https://world.openfoodfacts.org/images/products/317/328/910/5708/2.jpg https://world.openfoodfacts.org/images/products/317/328/910/5708/4.jpg

Who for

Part of

• 1627

[teolemon]

https://openfoodfacts.slack.com/archives/CDC9K6RFT/p1684863284900529

[github-actions[bot]]

This issue has been open 90 days with no activity. Can you give it a little love by linking it to a parent issue, adding relevant labels and projets, creating a mockup if applicable, adding code pointers from https://github.com/openfoodfacts/openfoodfacts-server/blob/main/.github/labeler.yml, giving it a priority, editing the original issue to have a more comprehensive description… Thank you very much for your contribution to 🍊 Open Food Facts

[github-actions[bot]]

This issue has been open 90 days with no activity. Can you give it a little love by linking it to a parent issue, adding relevant labels and projets, creating a mockup if applicable, adding code pointers from https://github.com/openfoodfacts/openfoodfacts-server/blob/main/.github/labeler.yml, giving it a priority, editing the original issue to have a more comprehensive description… Thank you very much for your contribution to 🍊 Open Food Facts

[teolemon]

Potentially

• Better detect image reuploads
• https://pypi.org/project/ImageHash/