facebookresearch / faiss

A library for efficient similarity search and clustering of dense vectors.
https://faiss.ai
MIT License
31.63k stars 3.65k forks source link

Use custom centroids in IVFFlat #4004

Closed JingyuanHe1222 closed 3 weeks ago

JingyuanHe1222 commented 3 weeks ago

Hi, I wonder if there's any interface we can use to construct IVF clusters with custom centroids vectors?

So instead of initialize the IVF index and add corpus vectors for train (which construct corpus centroids), can we manually designate some vectors as the centroids and update the clustering accordingly?

Seemingly this could be achieved by constructing a IndexFlat clusters on the custom centroids as quantizer, and manually construct nlist IndexFlat for each of these clusters after calculating the vector assignments using KNN.

But I wonder if there's a prebuild interface, which might be faster?

Thanks in advance!