search

facebookresearch / faiss

A library for efficient similarity search and clustering of dense vectors.
https://faiss.ai
MIT License
31.52k stars 3.65k forks source link

How to reproduce the benchmark result more easily #2556

Open chasingegg opened 2 years ago

chasingegg commented 2 years ago

Summary

I think the benchmark result is not very easy to reproduce. Like scann benchmark, https://github.com/facebookresearch/faiss/wiki/Indexing-1M-vectors#4-bit-pq-comparison-with-scann, it seems that the benchmark code is in bench_all_ivf/cmp_with_scann.py, but we could not directly generate the figs by running this script since there are no codes related to fig drawing, in addition there are lots of hyper parameter combination so that it is not trivial to select the 'dots' in the benchmark fig, I wonder if there are some code I was missing or it is not included at all, could anyone elaborate this ?

mdouze commented 2 years ago

Right, the code is a bit scattered around. I am looking into it...