I am building a personal system to provide the similarity result among large scale (1 to 10 millions) vectors. I searched and found there are lots of libraries to provide/service this functionality.
My hope is to save the expenses on the cloud infrasture and spend less maintainence on the code/cloud because it is only a personal project. But I am not sure because I didn't used these tools/services myself. So I could not compare them and get which library is better and I should choose: amazon-elasticsearch-knn, faiss or milvus ?
Hi,
I am building a personal system to provide the similarity result among large scale (1 to 10 millions) vectors. I searched and found there are lots of libraries to provide/service this functionality.
Faiss - https://github.com/facebookresearch/faiss
Amazon-elasticsearch-knn - https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/knn.html
Milvus - https://github.com/milvus-io/milvus
It seems that Amazon-elasticsearch-knn will occupy more memory compared with the other 2 libraries from :
https://medium.com/@kumon/how-to-realize-similarity-search-with-elasticsearch-3dd5641b9adb
https://medium.com/@kumon/similarity-search-and-similar-image-search-in-elasticsearch-14552a8a8dea
My hope is to save the expenses on the cloud infrasture and spend less maintainence on the code/cloud because it is only a personal project. But I am not sure because I didn't used these tools/services myself. So I could not compare them and get which library is better and I should choose: amazon-elasticsearch-knn, faiss or milvus ?
Put the same on the https://stackoverflow.com/questions/64493655/the-choice-of-vector-similarity-search-engine-amazon-elasticsearch-knn-faiss