index with scalar quantization not supported

criteo / autofaiss

Automatically create Faiss knn indices with the most optimal similarity search parameters.

https://criteo.github.io/autofaiss/

Apache License 2.0

817 stars 74 forks source link

index with scalar quantization not supported #122

Open OmniscienceAcademy opened 2 years ago

OmniscienceAcademy commented 2 years ago

I wanted to build IVF262144_HNSW32,SQ8, but the estimation of the required memory was way too big.

OmniscienceAcademy commented 2 years ago

I think it's important to have those kinds of index in the library since it does not consume a lot of ram when memory mapped and do not compress too much the vectors.

So it's an intermediary between HNSW32 and OPQ_PQ