search

rhsimplex / image-match

🎇 Quickly search over billions of images
2.94k stars 405 forks source link

Skip duplicates on indexing? #69

Closed ecdeveloper closed 7 years ago

ecdeveloper commented 7 years ago

Is there a way to skip indexing an image, if it already has been indexed? I have some images in elasticsearch, and I need to index some others. However, some of those images were already indexed. Is there way to make those images, which have 100% match in DB, to not be indexed again?

rhsimplex commented 7 years ago

Currently, no. You could do a lookup first and only accept new images above a certain distance, but this would add a lot of overhead to the image insertion and therefore could not be a default configuration.

Alternatively, if the filenames (or something) are unique, you could index on that with Elasticsearch.