search

erikbern / ann-benchmarks

Benchmarks of approximate nearest neighbor libraries in Python
http://ann-benchmarks.com
MIT License
4.74k stars 718 forks source link

Standardize parameters across hnsw implementations #430

Closed jbellis closed 11 months ago

jbellis commented 1 year ago

It looks like some of the hnsw implementations are run with very different parameters which makes it hard to compare apples to apples in the results.

I count seven hnsw implementations that should probably be run with similar parameters (the ones labeled hnsw, plus lucene and weaviate). Without looking at the code I'm not sure if there's more.

e.g. weaviate has M=8..72 and efConstruction=64..512, lucene has M=4..96 and efConstruction=500

maumueller commented 11 months ago

@jbellis Our philosophy is that it's up to the authors contributing the implementation to fix the hyper-parameter settings.