search

erikbern / ann-benchmarks

Benchmarks of approximate nearest neighbor libraries in Python
http://ann-benchmarks.com
MIT License
4.84k stars 725 forks source link

Are numerical results available as well? #62

Open ilyaraz opened 6 years ago

ilyaraz commented 6 years ago

Right now, I can see only plots in the repo, which are pretty useless for the "external" comparison as such.

In particular, is HNSW still using hours for preprocessing? :) There does not seem to be a way to check now.

maumueller commented 6 years ago

Information about build time and other metrics will be included on the website. (And the website should be available once #57 is done.) I think we should also make it possible to access the data that the plots are based on. What do you think, Erik? (I could host it.)

ilyaraz commented 6 years ago

I think without the data, 99% of the benchmarks' value is lost.

erikbern commented 6 years ago

You mean the data in results/ right? Or do you mean the granular metrics (i.e. the stuff that's plotted just in numerical form)?

Can make that available although I'm a bit confused why the benchmarks are useless until then. Everything is containerized so it's very easy to re-run in case you want to run your own experiments

ilyaraz commented 6 years ago

The granular metrics.

erikbern commented 6 years ago

Ok, sounds good...as @maumueller pointed out they should be published with #57