search

erikbern / ann-benchmarks

Benchmarks of approximate nearest neighbor libraries in Python
http://ann-benchmarks.com
MIT License
4.88k stars 735 forks source link

Run ONNG for nytimes #99

Closed masajiro closed 5 years ago

masajiro commented 5 years ago

@maumueller I resolved the problem (#93) of the results due to invalid zero vectors in the queries. Sorry for troubling you, but could you rebuild NGT and recreate results for the nytimes dataset? Thank you in advance.

erikbern commented 5 years ago

@masajiro what's preventing you from doing it yourself?

masajiro commented 5 years ago

@erikbern Sorry for multiple similar comments, because Github had big system problems yesterday.

Since I have already confirmed that the updated ONNG clearly improved its results for the nytimes dataset on my local environment, I would like you to update only ONNG results in the figure. Do you mean that you can replace the figures with figures that I made on my local environment?

erikbern commented 5 years ago

@masajiro we generally re-run the benchmarks every 3-4 months or so. maybe it's time to do it again. i think it would be a lot of work to do it ad-hoc any time there's library updates.

you can also go ahead and do it yourself if you want to update the repo – can run everything on a c5.4xlarge in AWS

masajiro commented 5 years ago

@erikbern Thank you for your reply. I will consider updating the repo as well.

yurymalkov commented 5 years ago

@masajiro @erikbern If there are zero vectors in the test then it is a test bed issue which affects all algorithms. So, I think, removing the zero vectors from the datasets is the right solution.

masajiro commented 5 years ago

@yurymalkov I totally agree with you.