search

harsha-simhadri / big-ann-benchmarks

Framework for evaluating ANNS algorithms on billion scale datasets.
https://big-ann-benchmarks.com
MIT License
313 stars 103 forks source link

EC2 results filter/ood/sparse #250

Closed maumueller closed 7 months ago

maumueller commented 7 months ago
dataset algorithm qps
yfcc-10M parlayivf 30744.178014
puck 17160.390356
hwtl_sdu_anns_filter 15433.837498
wm_filter 13723.065895
fdufilterdiskann 6085.293763
pyanns 5260.477613
faissplus 3851.283822
rubignn 3289.234566
faiss 3254.200190
text2image-10M pyanns 22476.070400
mysteryann-dif 17764.966620
mysteryann 17665.716757
sustech-ood 11594.134313
puck-fizz 8460.214238
puck 8167.845988
vamana 6322.589569
cufe 3414.617622
sparse-full pyanns 6280.871386
shnsw 4359.145718
nle 1297.986119
sustech-whu [1] 670.864748
cufe 64.665603
linscan 63.026394

[1] Build time exceeded 12 hours (13 hours, 34 minutes)

ingberam commented 7 months ago

Thanks @maumueller. Are these results on the private or public query sets?

maumueller commented 7 months ago

Thanks @maumueller. Are these results on the private or public query sets?

This is for the private workload (sparse/filter) and the public workload (ood), replicating the Azure evaluation.