search

facebookresearch / faiss

A library for efficient similarity search and clustering of dense vectors.
https://faiss.ai
MIT License
31.23k stars 3.63k forks source link

Does faiss support online spherical clustering with GPU? #1658

Closed RealNewNoob closed 2 years ago

RealNewNoob commented 3 years ago

We are going to perform a spherical clustering on a large dataset which is over our gpu memory, can and how I do it with faiis?

mdouze commented 3 years ago

NB that only the centroids need to fit in GPU memory, not the dataset.

RealNewNoob commented 3 years ago

NB that only the centroids need to fit in GPU memory, not the dataset.

I read the wiki and learned how to set up the index on disk. However, I still don't know how to load huge data properly in faiss.Clustering() or faiss.Kmeans(). Is there any tutorial for this?

gasparyanartur commented 1 year ago

Seeing as this was completed, could you mind pointing to where you found the answer to the question?