search

stanford-futuredata / ColBERT

ColBERT: state-of-the-art neural search (SIGIR'20, TACL'21, NeurIPS'21, NAACL'22, CIKM'22, ACL'23, EMNLP'23)
MIT License
2.95k stars 377 forks source link

What is faiss actually used for? #237

Closed lambdaofgod closed 5 months ago

lambdaofgod commented 1 year ago

First of all, let me thank you for providing this code. I don't often see such research-related code of such quality.

I see that faiss is only used in collection_indexer.

Because of my experience of running into all kinds of nasty bugs with faiss I'd like to consider removing it from dependencies.

Is my understanding right that removing faiss would only require replacing compute_faiss_kmeans?

Of course if I get to it I'd leave faiss kmeans but I'll add alternative kmeans implementation that will be used when it's specified in config.

santhnm2 commented 1 year ago

This is correct, FAISS is only used for the k-means computation.