search

harsha-simhadri / big-ann-benchmarks

Framework for evaluating ANNS algorithms on billion scale datasets.
https://big-ann-benchmarks.com
MIT License
313 stars 103 forks source link

pinecone's streaming algorithm #265

Closed ingberam closed 5 months ago

ingberam commented 6 months ago

pinecone's streaming algorithm. Expected results on the standard Azure VM:

pinecone,"pinecone(('R32_L100', {'Ls': 500, 'k_1': 30, 'T': 8}))",msturing-30M-clustered(final_runbook.yaml),10,,0.0,0.4697451591491699,2010972.0,0.0,0.0,streaming,0.997470000000012

streaming

harsha-simhadri commented 5 months ago

@ingberam All four pinecone submissions have a different naming convention. Do you want to make it unform before I merge?

harsha-simhadri commented 5 months ago

Is this the commandline? python run.py --algorithm pinecone --neurips23track streaming --dataset msturing-30M-clustered

ingberam commented 5 months ago

@ingberam All four pinecone submissions have a different naming convention. Do you want to make it unform before I merge?

That's fine. all names are wither pinecone or pinecone-xxx. Let's just merge.

ingberam commented 5 months ago

Is this the commandline? python run.py --algorithm pinecone --neurips23track streaming --dataset msturing-30M-clustered

@harsha-simhadri actually this is not the right commandline (it fails for all algorithms).

This is the correct command (note the explicit runbook path):

python run.py --dataset msturing-30M-clustered --algorithm pinecone --neurips23track streaming --runbook_path neurips23/streaming/final_runbook.yaml
harsha-simhadri commented 5 months ago

Only one entry ran to completion. The other two did not complete.

pinecone,"pinecone(('R32_L100', {'Ls': 300, 'k_1': 30, 'T': 8}))",msturing-30M-clustered(final_runbook.yaml),10,,0.0,0.4844532012939453,2012204.0,0.0,0.0,streaming,0.9908199999999938