search

LAION-AI / project-menu

Projects at LAION
MIT License
10 stars 4 forks source link

Dedup laion5b by image content #28

Open rom1504 opened 2 years ago

rom1504 commented 2 years ago

https://docs.google.com/document/d/1AryWpV0dD_r9x82I_quUzBuRyzDotL_HHnKuNB9H3Zc/edit?usp=drivesdk

Also important: evaluate dedup method on 1M samples and check if it works. (We want to remove points too near and not remove other ones)