search

harsha-simhadri / big-ann-benchmarks

Framework for evaluating ANNS algorithms on billion scale datasets.
https://big-ann-benchmarks.com
MIT License
341 stars 114 forks source link

pinecone's OOD algorithm #263

Closed ingberam closed 8 months ago

ingberam commented 9 months ago

Pinecone's OOD algorithm Expected results on the standard Azure VM: 26k-27k QPS for recall=0.9 (for text2image-10M). @harsha-simhadri can you run and update the leaderboard plots?

running time < 10min.

harsha-simhadri commented 8 months ago

pinecone-ood,"pinecone-ood(('OODIndex32768[HNSW32]_spillage=1', {'index_str': 'OODIndex32768[HNSW32]_spillage=1'}, {'nprobe': '45', 'kfactor': '1', 'Ls': '45'}))",text2image-10M,10,29752.692462523653,0.0,-1.0,12562160.0,0.0,0.0,ood,0.8672700000000001 pinecone-ood,"pinecone-ood(('OODIndex32768[HNSW32]_spillage=1', {'index_str': 'OODIndex32768[HNSW32]_spillage=1'}, {'nprobe': '50', 'kfactor': '2', 'Ls': '50'}))",text2image-10M,10,27167.53739200569,0.0,-1.0,12562160.0,0.0,0.0,ood,0.895101 pinecone-ood,"pinecone-ood(('OODIndex32768[HNSW32]_spillage=1', {'index_str': 'OODIndex32768[HNSW32]_spillage=1'}, {'nprobe': '51', 'kfactor': '2', 'Ls': '51'}))",text2image-10M,10,26326.969183923735,0.0,-1.0,12562160.0,0.0,0.0,ood,0.896649 pinecone-ood,"pinecone-ood(('OODIndex32768[HNSW32]_spillage=1', {'index_str': 'OODIndex32768[HNSW32]_spillage=1'}, {'nprobe': '49', 'kfactor': '3', 'Ls': '49'}))",text2image-10M,10,25574.431811945822,0.0,-1.0,12562160.0,0.0,0.0,ood,0.897937 pinecone-ood,"pinecone-ood(('OODIndex32768[HNSW32]_spillage=1', {'index_str': 'OODIndex32768[HNSW32]_spillage=1'}, {'nprobe': '52', 'kfactor': '2', 'Ls': '52'}))",text2image-10M,10,25936.241910584577,0.0,-1.0,12562160.0,0.0,0.0,ood,0.898148 pinecone-ood,"pinecone-ood(('OODIndex32768[HNSW32]_spillage=1', {'index_str': 'OODIndex32768[HNSW32]_spillage=1'}, {'nprobe': '50', 'kfactor': '3', 'Ls': '50'}))",text2image-10M,10,25567.748494986212,0.0,-1.0,12562160.0,0.0,0.0,ood,0.8994009999999999 pinecone-ood,"pinecone-ood(('OODIndex32768[HNSW32]_spillage=1', {'index_str': 'OODIndex32768[HNSW32]_spillage=1'}, {'nprobe': '53', 'kfactor': '2', 'Ls': '53'}))",text2image-10M,10,25440.496904114716,0.0,-1.0,12562160.0,0.0,0.0,ood,0.899405 pinecone-ood,"pinecone-ood(('OODIndex32768[HNSW32]_spillage=1', {'index_str': 'OODIndex32768[HNSW32]_spillage=1'}, {'nprobe': '51', 'kfactor': '3', 'Ls': '51'}))",text2image-10M,10,25297.367274141463,0.0,-1.0,12562160.0,0.0,0.0,ood,0.9007799999999999 pinecone-ood,"pinecone-ood(('OODIndex32768[HNSW32]_spillage=1', {'index_str': 'OODIndex32768[HNSW32]_spillage=1'}, {'nprobe': '54', 'kfactor': '2', 'Ls': '54'}))",text2image-10M,10,25439.472333576712,0.0,-1.0,12562160.0,0.0,0.0,ood,0.90084 pinecone-ood,"pinecone-ood(('OODIndex32768[HNSW32]_spillage=1', {'index_str': 'OODIndex32768[HNSW32]_spillage=1'}, {'nprobe': '60', 'kfactor': '2', 'Ls': '60'}))",text2image-10M,10,23613.462684143793,0.0,-1.0,12562160.0,0.0,0.0,ood,0.9085000000000001