search

perone / euclidesdb

A multi-model machine learning feature embedding database
http://euclidesdb.readthedocs.io
Other
631 stars 30 forks source link

Replace Annoy with Faiss as search engine ? #7

Closed yupbank closed 5 years ago

yupbank commented 5 years ago

So that we can use GPU to speed up the indexing

perone commented 5 years ago

That's a good idea, I'll evaluate that with more time after implementing the GPU support (need to evaluate how to distribute/link to CUDA libraries and libtorch-gpu). After this integration, I'll take a look at Faiss because it might have some integration issues to work together with libtorch-gpu. Thanks for posting this. Another comment, I'm trying to keep a design where I have "search engines", annoy is just one search engine, so we don't need to remove it, we just need to add Faiss as another search engine (together with linear search for example). This can be configurable in the EuclidesDB configuration files.

perone commented 5 years ago

CPU-only Faiss is now integrated with the search engine name faiss. Documentation is here.