Run default experiment Failed.

replay method

Clone the repo.
Run pip install -r requirements.txt.
Run python install.py to build all the libraries inside Docker containers (this can take a while, like 10-30 minutes).
Run python run.py (this can take an extremely long time, potentially days)

log

2021-02-20 13:04:00,189 - annb - INFO - not all docker images available, only: {'ann-benchmarks-milvus', 'ann-benchmarks-mrpt', 'ann-benchmarks-annoy', 'ann-benchmarks-flann', 'ann-benchmarks-ngt', 'ann-benchmarks-mih', 'ann-benchmarks-elasticsearch', 'ubuntu', 'ann-benchmarks-hnswlib', 'ann-benchmarks-scann', 'ann-benchmarks-dolphinn', 'ann-benchmarks-nearpy', 'ann-benchmarks-faiss', 'milvusdb/milvus', 'ann-benchmarks-rpforest', 'ann-benchmarks-pynndescent', 'ann-benchmarks-puffinn', 'ann-benchmarks-n2', 'ann-benchmarks-sklearn', 'ann-benchmarks', 'ann-benchmarks-datasketch', 'ann-benchmarks-kgraph', 'ann-benchmarks-nmslib', 'hello-world', 'ann-benchmarks-elastiknn'} 2021-02-20 13:04:00,189 - annb - INFO - missing docker images: {'ann-benchmarks-opendistroknn', 'ann-benchmarks-sptag'} 2021-02-20 13:04:00,189 - annb - INFO - Not running disabled algorithms [Definition(algorithm='faiss-gpu', constructor='FaissGPU', module='ann_benchmarks.algorithms.faiss_gpu', docker_tag='ann-benchmarks-faiss', arguments=[4096, 10], query_argument_groups=[], disabled=True), Definition(algorithm='nearpy', constructor='NearPy', module='ann_benchmarks.algorithms.nearpy', docker_tag='ann-benchmarks-nearpy', arguments=['angular', 16, 5], query_argument_groups=[], disabled=True), Definition(algorithm='bruteforce', constructor='BruteForce', module='ann_benchmarks.algorithms.bruteforce', docker_tag='ann-benchmarks-sklearn', arguments=['angular'], query_argument_groups=[], disabled=True), Definition(algorithm='nearpy', constructor='NearPy', module='ann_benchmarks.algorithms.nearpy', docker_tag='ann-benchmarks-nearpy', arguments=['angular', 12, 20], query_argument_groups=[], disabled=True), Definition(algorithm='nearpy', constructor='NearPy', module='ann_benchmarks.algorithms.nearpy', docker_tag='ann-benchmarks-nearpy', arguments=['angular', 16, 40], query_argument_groups=[], disabled=True), Definition(algorithm='nearpy', constructor='NearPy', module='ann_benchmarks.algorithms.nearpy', docker_tag='ann-benchmarks-nearpy', arguments=['angular', 16, 10], query_argument_groups=[], disabled=True), Definition(algorithm='nearpy', constructor='NearPy', module='ann_benchmarks.algorithms.nearpy', docker_tag='ann-benchmarks-nearpy', arguments=['angular', 14, 20], query_argument_groups=[], disabled=True), Definition(algorithm='faiss-gpu', constructor='FaissGPU', module='ann_benchmarks.algorithms.faiss_gpu', docker_tag='ann-benchmarks-faiss', arguments=[8192, 100], query_argument_groups=[], disabled=True), Definition(algorithm='faiss-gpu', constructor='FaissGPU', module='ann_benchmarks.algorithms.faiss_gpu', docker_tag='ann-benchmarks-faiss', arguments=[400, 10], query_argument_groups=[], disabled=True), Definition(algorithm='dummy-algo-st', constructor='DummyAlgoSt', module='ann_benchmarks.algorithms.dummy_algo', docker_tag='ann-benchmarks-sklearn', arguments=['angular'], query_argument_groups=[], disabled=True), Definition(algorithm='faiss-gpu', constructor='FaissGPU', module='ann_benchmarks.algorithms.faiss_gpu', docker_tag='ann-benchmarks-faiss', arguments=[8192, 1], query_argument_groups=[], disabled=True), Definition(algorithm='faiss-gpu', constructor='FaissGPU', module='ann_benchmarks.algorithms.faiss_gpu', docker_tag='ann-benchmarks-faiss', arguments=[400, 200], query_argument_groups=[], disabled=True), Definition(algorithm='faiss-gpu', constructor='FaissGPU', module='ann_benchmarks.algorithms.faiss_gpu', docker_tag='ann-benchmarks-faiss', arguments=[4096, 100], query_argument_groups=[], disabled=True), Definition(algorithm='dummy-algo-mt', constructor='DummyAlgoMt', module='ann_benchmarks.algorithms.dummy_algo', docker_tag='ann-benchmarks-sklearn', arguments=['angular'], query_argument_groups=[], disabled=True), Definition(algorithm='faiss-gpu', constructor='FaissGPU', module='ann_benchmarks.algorithms.faiss_gpu', docker_tag='ann-benchmarks-faiss', arguments=[8192, 40], query_argument_groups=[], disabled=True), Definition(algorithm='nearpy', constructor='NearPy', module='ann_benchmarks.algorithms.nearpy', docker_tag='ann-benchmarks-nearpy', arguments=['angular', 16, 20], query_argument_groups=[], disabled=True), Definition(algorithm='DolphinnPy', constructor='DolphinnPy', module='ann_benchmarks.algorithms.dolphinnpy', docker_tag='ann-benchmarks-dolphinn', arguments=[10], query_argument_groups=[], disabled=True), Definition(algorithm='ball', constructor='BallTree', module='ann_benchmarks.algorithms.balltree', docker_tag='ann-benchmarks-sklearn', arguments=['angular', 40], query_argument_groups=[], disabled=True), Definition(algorithm='faiss-gpu', constructor='FaissGPU', module='ann_benchmarks.algorithms.faiss_gpu', docker_tag='ann-benchmarks-faiss', arguments=[16384, 100], query_argument_groups=[], disabled=True), Definition(algorithm='faiss-gpu', constructor='FaissGPU', module='ann_benchmarks.algorithms.faiss_gpu', docker_tag='ann-benchmarks-faiss', arguments=[4096, 200], query_argument_groups=[], disabled=True), Definition(algorithm='faiss-gpu', constructor='FaissGPU', module='ann_benchmarks.algorithms.faiss_gpu', docker_tag='ann-benchmarks-faiss', arguments=[1024, 200], query_argument_groups=[], disabled=True), Definition(algorithm='nearpy', constructor='NearPy', module='ann_benchmarks.algorithms.nearpy', docker_tag='ann-benchmarks-nearpy', arguments=['angular', 14, 10], query_argument_groups=[], disabled=True), Definition(algorithm='faiss-gpu', constructor='FaissGPU', module='ann_benchmarks.algorithms.faiss_gpu', docker_tag='ann-benchmarks-faiss', arguments=[400, 40], query_argument_groups=[], disabled=True), Definition(algorithm='faiss-gpu', constructor='FaissGPU', module='ann_benchmarks.algorithms.faiss_gpu', docker_tag='ann-benchmarks-faiss', arguments=[16384, 1], query_argument_groups=[], disabled=True), Definition(algorithm='faiss-gpu', constructor='FaissGPU', module='ann_benchmarks.algorithms.faiss_gpu', docker_tag='ann-benchmarks-faiss', arguments=[1024, 10], query_argument_groups=[], disabled=True), Definition(algorithm='nearpy', constructor='NearPy', module='ann_benchmarks.algorithms.nearpy', docker_tag='ann-benchmarks-nearpy', arguments=['angular', 16, 20], query_argument_groups=[], disabled=True), Definition(algorithm='faiss-lsh', constructor='FaissLSH', module='ann_benchmarks.algorithms.faiss', docker_tag='ann-benchmarks-faiss', arguments=['angular', 1024], query_argument_groups=[], disabled=True), Definition(algorithm='nearpy', constructor='NearPy', module='ann_benchmarks.algorithms.nearpy', docker_tag='ann-benchmarks-nearpy', arguments=['angular', 12, 5], query_argument_groups=[], disabled=True), Definition(algorithm='nearpy', constructor='NearPy', module='ann_benchmarks.algorithms.nearpy', docker_tag='ann-benchmarks-nearpy', arguments=['angular', 12, 40], query_argument_groups=[], disabled=True), Definition(algorithm='faiss-gpu', constructor='FaissGPU', module='ann_benchmarks.algorithms.faiss_gpu', docker_tag='ann-benchmarks-faiss', arguments=[8192, 200], query_argument_groups=[], disabled=True), Definition(algorithm='faiss-gpu', constructor='FaissGPU', module='ann_benchmarks.algorithms.faiss_gpu', docker_tag='ann-benchmarks-faiss', arguments=[4096, 40], query_argument_groups=[], disabled=True), Definition(algorithm='faiss-lsh', constructor='FaissLSH', module='ann_benchmarks.algorithms.faiss', docker_tag='ann-benchmarks-faiss', arguments=['angular', 4096], query_argument_groups=[], disabled=True), Definition(algorithm='DolphinnPy', constructor='DolphinnPy', module='ann_benchmarks.algorithms.dolphinnpy', docker_tag='ann-benchmarks-dolphinn', arguments=[2000], query_argument_groups=[], disabled=True), Definition(algorithm='nearpy', constructor='NearPy', module='ann_benchmarks.algorithms.nearpy', docker_tag='ann-benchmarks-nearpy', arguments=['angular', 10, 40], query_argument_groups=[], disabled=True), Definition(algorithm='nearpy', constructor='NearPy', module='ann_benchmarks.algorithms.nearpy', docker_tag='ann-benchmarks-nearpy', arguments=['angular', 14, 5], query_argument_groups=[], disabled=True), Definition(algorithm='ball', constructor='BallTree', module='ann_benchmarks.algorithms.balltree', docker_tag='ann-benchmarks-sklearn', arguments=['angular', 200], query_argument_groups=[], disabled=True), Definition(algorithm='nearpy', constructor='NearPy', module='ann_benchmarks.algorithms.nearpy', docker_tag='ann-benchmarks-nearpy', arguments=['angular', 10, 10], query_argument_groups=[], disabled=True), Definition(algorithm='DolphinnPy', constructor='DolphinnPy', module='ann_benchmarks.algorithms.dolphinnpy', docker_tag='ann-benchmarks-dolphinn', arguments=[100], query_argument_groups=[], disabled=True), Definition(algorithm='nearpy', constructor='NearPy', module='ann_benchmarks.algorithms.nearpy', docker_tag='ann-benchmarks-nearpy', arguments=['angular', 16, 5], query_argument_groups=[], disabled=True), Definition(algorithm='faiss-gpu', constructor='FaissGPU', module='ann_benchmarks.algorithms.faiss_gpu', docker_tag='ann-benchmarks-faiss', arguments=[8192, 10], query_argument_groups=[], disabled=True), Definition(algorithm='nearpy', constructor='NearPy', module='ann_benchmarks.algorithms.nearpy', docker_tag='ann-benchmarks-nearpy', arguments=['angular', 16, 15], query_argument_groups=[], disabled=True), Definition(algorithm='faiss-lsh', constructor='FaissLSH', module='ann_benchmarks.algorithms.faiss', docker_tag='ann-benchmarks-faiss', arguments=['angular', 2048], query_argument_groups=[], disabled=True), Definition(algorithm='faiss-gpu', constructor='FaissGPU', module='ann_benchmarks.algorithms.faiss_gpu', docker_tag='ann-benchmarks-faiss', arguments=[1024, 100], query_argument_groups=[], disabled=True), Definition(algorithm='ball', constructor='BallTree', module='ann_benchmarks.algorithms.balltree', docker_tag='ann-benchmarks-sklearn', arguments=['angular', 100], query_argument_groups=[], disabled=True), Definition(algorithm='faiss-gpu', constructor='FaissGPU', module='ann_benchmarks.algorithms.faiss_gpu', docker_tag='ann-benchmarks-faiss', arguments=[400, 100], query_argument_groups=[], disabled=True), Definition(algorithm='faiss-gpu', constructor='FaissGPU', module='ann_benchmarks.algorithms.faiss_gpu', docker_tag='ann-benchmarks-faiss', arguments=[16384, 200], query_argument_groups=[], disabled=True), Definition(algorithm='faiss-gpu', constructor='FaissGPU', module='ann_benchmarks.algorithms.faiss_gpu', docker_tag='ann-benchmarks-faiss', arguments=[16384, 40], query_argument_groups=[], disabled=True), Definition(algorithm='nearpy', constructor='NearPy', module='ann_benchmarks.algorithms.nearpy', docker_tag='ann-benchmarks-nearpy', arguments=['angular', 16, 40], query_argument_groups=[], disabled=True), Definition(algorithm='nearpy', constructor='NearPy', module='ann_benchmarks.algorithms.nearpy', docker_tag='ann-benchmarks-nearpy', arguments=['angular', 16, 10], query_argument_groups=[], disabled=True), Definition(algorithm='faiss-gpu', constructor='FaissGPU', module='ann_benchmarks.algorithms.faiss_gpu', docker_tag='ann-benchmarks-faiss', arguments=[1024, 40], query_argument_groups=[], disabled=True), Definition(algorithm='faiss-lsh', constructor='FaissLSH', module='ann_benchmarks.algorithms.faiss', docker_tag='ann-benchmarks-faiss', arguments=['angular', 256], query_argument_groups=[], disabled=True), Definition(algorithm='faiss-gpu', constructor='FaissGPU', module='ann_benchmarks.algorithms.faiss_gpu', docker_tag='ann-benchmarks-faiss', arguments=[1024, 1], query_argument_groups=[], disabled=True), Definition(algorithm='ball', constructor='BallTree', module='ann_benchmarks.algorithms.balltree', docker_tag='ann-benchmarks-sklearn', arguments=['angular', 20], query_argument_groups=[], disabled=True), Definition(algorithm='nearpy', constructor='NearPy', module='ann_benchmarks.algorithms.nearpy', docker_tag='ann-benchmarks-nearpy', arguments=['angular', 14, 40], query_argument_groups=[], disabled=True), Definition(algorithm='faiss-lsh', constructor='FaissLSH', module='ann_benchmarks.algorithms.faiss', docker_tag='ann-benchmarks-faiss', arguments=['angular', 512], query_argument_groups=[], disabled=True), Definition(algorithm='nearpy', constructor='NearPy', module='ann_benchmarks.algorithms.nearpy', docker_tag='ann-benchmarks-nearpy', arguments=['angular', 12, 10], query_argument_groups=[], disabled=True), Definition(algorithm='DolphinnPy', constructor='DolphinnPy', module='ann_benchmarks.algorithms.dolphinnpy', docker_tag='ann-benchmarks-dolphinn', arguments=[50], query_argument_groups=[], disabled=True), Definition(algorithm='faiss-gpu', constructor='FaissGPU', module='ann_benchmarks.algorithms.faiss_gpu', docker_tag='ann-benchmarks-faiss', arguments=[16384, 10], query_argument_groups=[], disabled=True), Definition(algorithm='ball', constructor='BallTree', module='ann_benchmarks.algorithms.balltree', docker_tag='ann-benchmarks-sklearn', arguments=['angular', 10], query_argument_groups=[], disabled=True), Definition(algorithm='nearpy', constructor='NearPy', module='ann_benchmarks.algorithms.nearpy', docker_tag='ann-benchmarks-nearpy', arguments=['angular', 10, 5], query_argument_groups=[], disabled=True), Definition(algorithm='ball', constructor='BallTree', module='ann_benchmarks.algorithms.balltree', docker_tag='ann-benchmarks-sklearn', arguments=['angular', 1000], query_argument_groups=[], disabled=True), Definition(algorithm='faiss-lsh', constructor='FaissLSH', module='ann_benchmarks.algorithms.faiss', docker_tag='ann-benchmarks-faiss', arguments=['angular', 32], query_argument_groups=[], disabled=True), Definition(algorithm='DolphinnPy', constructor='DolphinnPy', module='ann_benchmarks.algorithms.dolphinnpy', docker_tag='ann-benchmarks-dolphinn', arguments=[200], query_argument_groups=[], disabled=True), Definition(algorithm='nearpy', constructor='NearPy', module='ann_benchmarks.algorithms.nearpy', docker_tag='ann-benchmarks-nearpy', arguments=['angular', 16, 25], query_argument_groups=[], disabled=True), Definition(algorithm='faiss-gpu', constructor='FaissGPU', module='ann_benchmarks.algorithms.faiss_gpu', docker_tag='ann-benchmarks-faiss', arguments=[400, 1], query_argument_groups=[], disabled=True), Definition(algorithm='ball', constructor='BallTree', module='ann_benchmarks.algorithms.balltree', docker_tag='ann-benchmarks-sklearn', arguments=['angular', 400], query_argument_groups=[], disabled=True), Definition(algorithm='elastiknn-exact', constructor='Exact', module='ann_benchmarks.algorithms.elastiknn', docker_tag='ann-benchmarks-elastiknn', arguments=['angular', 100], query_argument_groups=[], disabled=True), Definition(algorithm='nearpy', constructor='NearPy', module='ann_benchmarks.algorithms.nearpy', docker_tag='ann-benchmarks-nearpy', arguments=['angular', 16, 30], query_argument_groups=[], disabled=True), Definition(algorithm='nearpy', constructor='NearPy', module='ann_benchmarks.algorithms.nearpy', docker_tag='ann-benchmarks-nearpy', arguments=['angular', 10, 20], query_argument_groups=[], disabled=True), Definition(algorithm='faiss-lsh', constructor='FaissLSH', module='ann_benchmarks.algorithms.faiss', docker_tag='ann-benchmarks-faiss', arguments=['angular', 64], query_argument_groups=[], disabled=True), Definition(algorithm='faiss-gpu', constructor='FaissGPU', module='ann_benchmarks.algorithms.faiss_gpu', docker_tag='ann-benchmarks-faiss', arguments=[4096, 1], query_argument_groups=[], disabled=True), Definition(algorithm='DolphinnPy', constructor='DolphinnPy', module='ann_benchmarks.algorithms.dolphinnpy', docker_tag='ann-benchmarks-dolphinn', arguments=[1000], query_argument_groups=[], disabled=True), Definition(algorithm='faiss-lsh', constructor='FaissLSH', module='ann_benchmarks.algorithms.faiss', docker_tag='ann-benchmarks-faiss', arguments=['angular', 128], query_argument_groups=[], disabled=True)] 2021-02-20 13:04:00,190 - annb - INFO - Order: [Definition(algorithm='NGT-panng', constructor='PANNG', module='ann_benchmarks.algorithms.panng_ngt', docker_tag='ann-benchmarks-ngt', arguments=['angular', 'Float', {'edge': 20, 'pathadj': 40, 'searchedge': 60}], query_argument_groups=[[0.6], [0.8], [0.9], [1.0], [1.02], [1.05], [1.1], [1.2]], disabled=False), Definition(algorithm='pynndescent', constructor='PyNNDescent', module='ann_benchmarks.algorithms.pynndescent', docker_tag='ann-benchmarks-pynndescent', arguments=['angular', {'n_neighbors': 120, 'diversify_prob': 1.0, 'pruning_degree_multiplier': 2.5, 'leaf_size': 20}], query_argument_groups=[[0.0], [0.04], [0.08], [0.16], [0.2], [0.24], [0.28], [0.32]], disabled=False), Definition(algorithm='n2', constructor='N2', module='ann_benchmarks.algorithms.n2', docker_tag='ann-benchmarks-n2', arguments=['angular', {'M': 48, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='milvus', constructor='Milvus', module='ann_benchmarks.algorithms.milvus', docker_tag='ann-benchmarks-milvus', arguments=['angular', 'IVF_FLAT', 3000], query_argument_groups=[[1], [3], [10], [30], [100], [300]], disabled=False), Definition(algorithm='kd', constructor='KDTree', module='ann_benchmarks.algorithms.kdtree', docker_tag='ann-benchmarks-sklearn', arguments=['angular', 100], query_argument_groups=[], disabled=False), Definition(algorithm='annoy', constructor='Annoy', module='ann_benchmarks.algorithms.annoy', docker_tag='ann-benchmarks-annoy', arguments=['angular', 100], query_argument_groups=[[100], [200], [400], [1000], [2000], [4000], [10000], [20000], [40000], [100000], [200000], [400000]], disabled=False), Definition(algorithm='BallTree(nmslib)', constructor='NmslibReuseIndex', module='ann_benchmarks.algorithms.nmslib', docker_tag='ann-benchmarks-nmslib', arguments=['angular', 'vptree', {'tuneK': 10, 'desiredRecall': 0.3}, False], query_argument_groups=[], disabled=False), Definition(algorithm='puffinn', constructor='Puffinn', module='ann_benchmarks.algorithms.puffinn', docker_tag='ann-benchmarks-puffinn', arguments=['angular', 268435456, 'fht_crosspolytope'], query_argument_groups=[[0.1], [0.2], [0.5], [0.7], [0.9], [0.95], [0.99]], disabled=False), Definition(algorithm='BallTree(nmslib)', constructor='NmslibReuseIndex', module='ann_benchmarks.algorithms.nmslib', docker_tag='ann-benchmarks-nmslib', arguments=['angular', 'vptree', {'tuneK': 10, 'desiredRecall': 0.5}, False], query_argument_groups=[], disabled=False), Definition(algorithm='puffinn', constructor='Puffinn', module='ann_benchmarks.algorithms.puffinn', docker_tag='ann-benchmarks-puffinn', arguments=['angular', 1073741824, 'fht_crosspolytope'], query_argument_groups=[[0.1], [0.2], [0.5], [0.7], [0.9], [0.95], [0.99]], disabled=False), Definition(algorithm='hnsw(faiss)', constructor='FaissHNSW', module='ann_benchmarks.algorithms.faiss_hnsw', docker_tag='ann-benchmarks-faiss', arguments=['angular', {'M': 8, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='n2', constructor='N2', module='ann_benchmarks.algorithms.n2', docker_tag='ann-benchmarks-n2', arguments=['angular', {'M': 64, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='SW-graph(nmslib)', constructor='NmslibReuseIndex', module='ann_benchmarks.algorithms.nmslib', docker_tag='ann-benchmarks-nmslib', arguments=['angular', 'sw-graph', {'NN': 30}, False], query_argument_groups=[[700], [650], [550], [450], [350], [275], [200], [150], [120], [80], [50], [30]], disabled=False), Definition(algorithm='n2', constructor='N2', module='ann_benchmarks.algorithms.n2', docker_tag='ann-benchmarks-n2', arguments=['angular', {'M': 12, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='mrpt', constructor='MRPT', module='ann_benchmarks.algorithms.mrpt', docker_tag='ann-benchmarks-mrpt', arguments=['angular', 10], query_argument_groups=[[0.1], [0.2], [0.3], [0.4], [0.5], [0.6], [0.7], [0.8], [0.85], [0.9], [0.925], [0.95], [0.97], [0.98], [0.99], [0.995]], disabled=False), Definition(algorithm='hnsw(faiss)', constructor='FaissHNSW', module='ann_benchmarks.algorithms.faiss_hnsw', docker_tag='ann-benchmarks-faiss', arguments=['angular', {'M': 36, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='scann', constructor='Scann', module='ann_benchmarks.algorithms.scann', docker_tag='ann-benchmarks-scann', arguments=[1500, 0.55, 1, 'dot_product'], query_argument_groups=[[[1, 30]], [[2, 30]], [[4, 30]], [[8, 30]], [[8, 25]], [[10, 25]], [[12, 25]], [[13, 25]], [[14, 27]], [[15, 30]], [[17, 30]], [[18, 40]], [[20, 40]], [[22, 40]], [[25, 50]], [[30, 50]], [[35, 55]], [[50, 60]], [[60, 60]], [[80, 80]], [[100, 100]]], disabled=False), Definition(algorithm='hnsw(faiss)', constructor='FaissHNSW', module='ann_benchmarks.algorithms.faiss_hnsw', docker_tag='ann-benchmarks-faiss', arguments=['angular', {'M': 24, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='flann', constructor='FLANN', module='ann_benchmarks.algorithms.flann', docker_tag='ann-benchmarks-flann', arguments=['angular', 0.9], query_argument_groups=[], disabled=False), Definition(algorithm='faiss-ivf', constructor='FaissIVF', module='ann_benchmarks.algorithms.faiss', docker_tag='ann-benchmarks-faiss', arguments=['angular', 256], query_argument_groups=[[1], [5], [10], [50], [100], [200]], disabled=False), Definition(algorithm='rpforest', constructor='RPForest', module='ann_benchmarks.algorithms.rpforest', docker_tag='ann-benchmarks-rpforest', arguments=[10, 400], query_argument_groups=[], disabled=False), Definition(algorithm='scann', constructor='Scann', module='ann_benchmarks.algorithms.scann', docker_tag='ann-benchmarks-scann', arguments=[1000, 0.2, 1, 'dot_product'], query_argument_groups=[[[1, 30]], [[2, 30]], [[4, 30]], [[8, 30]], [[9, 25]], [[11, 35]], [[12, 35]], [[13, 35]], [[14, 40]], [[15, 40]], [[16, 40]], [[17, 45]], [[20, 45]], [[20, 55]], [[25, 55]], [[25, 70]], [[30, 70]], [[40, 90]], [[50, 100]], [[60, 120]], [[70, 140]]], disabled=False), Definition(algorithm='SW-graph(nmslib)', constructor='NmslibReuseIndex', module='ann_benchmarks.algorithms.nmslib', docker_tag='ann-benchmarks-nmslib', arguments=['angular', 'sw-graph', {'NN': 15}, False], query_argument_groups=[[80], [50], [30], [20]], disabled=False), Definition(algorithm='rpforest', constructor='RPForest', module='ann_benchmarks.algorithms.rpforest', docker_tag='ann-benchmarks-rpforest', arguments=[3, 10], query_argument_groups=[], disabled=False), Definition(algorithm='hnsw(nmslib)', constructor='NmslibReuseIndex', module='ann_benchmarks.algorithms.nmslib', docker_tag='ann-benchmarks-nmslib', arguments=['angular', 'hnsw', {'M': 20, 'post': 0, 'efConstruction': 800}, False], query_argument_groups=[[2], [5], [10], [15], [20], [30], [40], [50], [70], [80]], disabled=False), Definition(algorithm='BallTree(nmslib)', constructor='NmslibReuseIndex', module='ann_benchmarks.algorithms.nmslib', docker_tag='ann-benchmarks-nmslib', arguments=['angular', 'vptree', {'tuneK': 10, 'desiredRecall': 0.2}, False], query_argument_groups=[], disabled=False), Definition(algorithm='bruteforce-blas', constructor='BruteForceBLAS', module='ann_benchmarks.algorithms.bruteforce', docker_tag='ann-benchmarks-sklearn', arguments=['angular'], query_argument_groups=[], disabled=False), Definition(algorithm='n2', constructor='N2', module='ann_benchmarks.algorithms.n2', docker_tag='ann-benchmarks-n2', arguments=['angular', {'M': 36, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='flann', constructor='FLANN', module='ann_benchmarks.algorithms.flann', docker_tag='ann-benchmarks-flann', arguments=['angular', 0.8], query_argument_groups=[], disabled=False), Definition(algorithm='n2', constructor='N2', module='ann_benchmarks.algorithms.n2', docker_tag='ann-benchmarks-n2', arguments=['angular', {'M': 96, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='milvus', constructor='Milvus', module='ann_benchmarks.algorithms.milvus', docker_tag='ann-benchmarks-milvus', arguments=['angular', 'IVF_FLAT', 10000], query_argument_groups=[[1], [3], [10], [30], [100], [300]], disabled=False), Definition(algorithm='NGT-qg', constructor='QG', module='ann_benchmarks.algorithms.qg_ngt', docker_tag='ann-benchmarks-ngt', arguments=['angular', 'Float', 0.1, {'edge': 100, 'outdegree': 64, 'indegree': 120, 'max_edge': 96}], query_argument_groups=[[1.5, 0.9], [1.5, 0.95], [1.5, 1.0], [1.5, 1.01], [1.5, 1.02], [1.5, 1.03], [1.5, 1.04], [1.5, 1.05], [1.5, 1.06], [1.7, 0.9], [1.7, 0.95], [1.7, 1.0], [1.7, 1.01], [1.7, 1.02], [1.7, 1.03], [1.7, 1.04], [1.7, 1.05], [1.7, 1.06], [2.0, 0.9], [2.0, 0.95], [2.0, 1.0], [2.0, 1.01], [2.0, 1.02], [2.0, 1.03], [2.0, 1.04], [2.0, 1.05], [2.0, 1.06], [2.5, 0.9], [2.5, 0.95], [2.5, 1.0], [2.5, 1.01], [2.5, 1.02], [2.5, 1.03], [2.5, 1.04], [2.5, 1.05], [2.5, 1.06], [3.0, 0.9], [3.0, 0.95], [3.0, 1.0], [3.0, 1.01], [3.0, 1.02], [3.0, 1.03], [3.0, 1.04], [3.0, 1.05], [3.0, 1.06], [5.0, 0.9], [5.0, 0.95], [5.0, 1.0], [5.0, 1.01], [5.0, 1.02], [5.0, 1.03], [5.0, 1.04], [5.0, 1.05], [5.0, 1.06], [8.0, 0.9], [8.0, 0.95], [8.0, 1.0], [8.0, 1.01], [8.0, 1.02], [8.0, 1.03], [8.0, 1.04], [8.0, 1.05], [8.0, 1.06]], disabled=False), Definition(algorithm='hnsw(nmslib)', constructor='NmslibReuseIndex', module='ann_benchmarks.algorithms.nmslib', docker_tag='ann-benchmarks-nmslib', arguments=['angular', 'hnsw', {'M': 48, 'post': 2, 'efConstruction': 800}, False], query_argument_groups=[[50], [70], [90], [120], [160], [200], [400], [600], [700], [800], [1000], [1400], [1600], [2000]], disabled=False), Definition(algorithm='kd', constructor='KDTree', module='ann_benchmarks.algorithms.kdtree', docker_tag='ann-benchmarks-sklearn', arguments=['angular', 200], query_argument_groups=[], disabled=False), Definition(algorithm='puffinn', constructor='Puffinn', module='ann_benchmarks.algorithms.puffinn', docker_tag='ann-benchmarks-puffinn', arguments=['angular', 2147483648, 'fht_crosspolytope'], query_argument_groups=[[0.1], [0.2], [0.5], [0.7], [0.9], [0.95], [0.99]], disabled=False), Definition(algorithm='puffinn', constructor='Puffinn', module='ann_benchmarks.algorithms.puffinn', docker_tag='ann-benchmarks-puffinn', arguments=['angular', 4294967296, 'fht_crosspolytope'], query_argument_groups=[[0.1], [0.2], [0.5], [0.7], [0.9], [0.95], [0.99]], disabled=False), Definition(algorithm='kd', constructor='KDTree', module='ann_benchmarks.algorithms.kdtree', docker_tag='ann-benchmarks-sklearn', arguments=['angular', 1000], query_argument_groups=[], disabled=False), Definition(algorithm='elasticsearch', constructor='ElasticsearchScriptScoreQuery', module='ann_benchmarks.algorithms.elasticsearch', docker_tag='ann-benchmarks-elasticsearch', arguments=['angular', 100], query_argument_groups=[], disabled=False), Definition(algorithm='BallTree(nmslib)', constructor='NmslibReuseIndex', module='ann_benchmarks.algorithms.nmslib', docker_tag='ann-benchmarks-nmslib', arguments=['angular', 'vptree', {'tuneK': 10, 'desiredRecall': 0.4}, False], query_argument_groups=[], disabled=False), Definition(algorithm='faiss-ivf', constructor='FaissIVF', module='ann_benchmarks.algorithms.faiss', docker_tag='ann-benchmarks-faiss', arguments=['angular', 1024], query_argument_groups=[[1], [5], [10], [50], [100], [200]], disabled=False), Definition(algorithm='rpforest', constructor='RPForest', module='ann_benchmarks.algorithms.rpforest', docker_tag='ann-benchmarks-rpforest', arguments=[400, 3], query_argument_groups=[], disabled=False), Definition(algorithm='rpforest', constructor='RPForest', module='ann_benchmarks.algorithms.rpforest', docker_tag='ann-benchmarks-rpforest', arguments=[400, 40], query_argument_groups=[], disabled=False), Definition(algorithm='hnsw(faiss)', constructor='FaissHNSW', module='ann_benchmarks.algorithms.faiss_hnsw', docker_tag='ann-benchmarks-faiss', arguments=['angular', {'M': 48, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='hnswlib', constructor='HnswLib', module='ann_benchmarks.algorithms.hnswlib', docker_tag='ann-benchmarks-hnswlib', arguments=['angular', {'M': 16, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='n2', constructor='N2', module='ann_benchmarks.algorithms.n2', docker_tag='ann-benchmarks-n2', arguments=['angular', {'M': 8, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='hnswlib', constructor='HnswLib', module='ann_benchmarks.algorithms.hnswlib', docker_tag='ann-benchmarks-hnswlib', arguments=['angular', {'M': 8, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='scann', constructor='Scann', module='ann_benchmarks.algorithms.scann', docker_tag='ann-benchmarks-scann', arguments=[2000, 0.2, 2, 'dot_product'], query_argument_groups=[[[1, 30]], [[2, 30]], [[4, 30]], [[8, 30]], [[30, 120]], [[35, 100]], [[40, 80]], [[45, 80]], [[50, 80]], [[55, 95]], [[60, 110]], [[65, 110]], [[75, 110]], [[90, 110]], [[110, 120]], [[130, 150]], [[150, 200]], [[170, 200]], [[200, 300]], [[220, 500]], [[250, 500]], [[310, 300]], [[400, 300]], [[500, 500]], [[800, 1000]]], disabled=False), Definition(algorithm='faiss-ivf', constructor='FaissIVF', module='ann_benchmarks.algorithms.faiss', docker_tag='ann-benchmarks-faiss', arguments=['angular', 512], query_argument_groups=[[1], [5], [10], [50], [100], [200]], disabled=False), Definition(algorithm='kd', constructor='KDTree', module='ann_benchmarks.algorithms.kdtree', docker_tag='ann-benchmarks-sklearn', arguments=['angular', 40], query_argument_groups=[], disabled=False), Definition(algorithm='rpforest', constructor='RPForest', module='ann_benchmarks.algorithms.rpforest', docker_tag='ann-benchmarks-rpforest', arguments=[10, 100], query_argument_groups=[], disabled=False), Definition(algorithm='BallTree(nmslib)', constructor='NmslibReuseIndex', module='ann_benchmarks.algorithms.nmslib', docker_tag='ann-benchmarks-nmslib', arguments=['angular', 'vptree', {'tuneK': 10, 'desiredRecall': 0.99}, False], query_argument_groups=[], disabled=False), Definition(algorithm='BallTree(nmslib)', constructor='NmslibReuseIndex', module='ann_benchmarks.algorithms.nmslib', docker_tag='ann-benchmarks-nmslib', arguments=['angular', 'vptree', {'tuneK': 10, 'desiredRecall': 0.8}, False], query_argument_groups=[], disabled=False), Definition(algorithm='rpforest', constructor='RPForest', module='ann_benchmarks.algorithms.rpforest', docker_tag='ann-benchmarks-rpforest', arguments=[10, 40], query_argument_groups=[], disabled=False), Definition(algorithm='rpforest', constructor='RPForest', module='ann_benchmarks.algorithms.rpforest', docker_tag='ann-benchmarks-rpforest', arguments=[40, 400], query_argument_groups=[], disabled=False), Definition(algorithm='rpforest', constructor='RPForest', module='ann_benchmarks.algorithms.rpforest', docker_tag='ann-benchmarks-rpforest', arguments=[100, 10], query_argument_groups=[], disabled=False), Definition(algorithm='hnsw(nmslib)', constructor='NmslibReuseIndex', module='ann_benchmarks.algorithms.nmslib', docker_tag='ann-benchmarks-nmslib', arguments=['angular', 'hnsw', {'M': 32, 'post': 2, 'efConstruction': 800}, False], query_argument_groups=[[10], [20], [30], [40], [50], [60], [70], [80], [90], [100], [120], [140], [160], [200], [300], [400], [600], [700], [800], [1000], [1200], [1400], [1600], [2000]], disabled=False), Definition(algorithm='rpforest', constructor='RPForest', module='ann_benchmarks.algorithms.rpforest', docker_tag='ann-benchmarks-rpforest', arguments=[40, 40], query_argument_groups=[], disabled=False), Definition(algorithm='hnswlib', constructor='HnswLib', module='ann_benchmarks.algorithms.hnswlib', docker_tag='ann-benchmarks-hnswlib', arguments=['angular', {'M': 96, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='scann', constructor='Scann', module='ann_benchmarks.algorithms.scann', docker_tag='ann-benchmarks-scann', arguments=[1400, 0.15, 3, 'dot_product'], query_argument_groups=[[[1, 30]], [[4, 30]], [[9, 30]], [[16, 32]], [[25, 50]], [[36, 72]], [[49, 98]], [[70, 150]], [[90, 200]], [[120, 210]], [[180, 270]], [[210, 330]], [[260, 400]], [[320, 500]], [[400, 600]], [[500, 700]], [[800, 900]]], disabled=False), Definition(algorithm='NGT-onng', constructor='ONNG', module='ann_benchmarks.algorithms.onng_ngt', docker_tag='ann-benchmarks-ngt', arguments=['angular', 'Float', 0.1, {'edge': 100, 'outdegree': 10, 'indegree': 80, 'search_edge': 60, 'tree': False}], query_argument_groups=[[0.6], [0.9], [0.995], [0.998], [1.0], [1.01], [1.02], [1.05], [1.07]], disabled=False), Definition(algorithm='flann', constructor='FLANN', module='ann_benchmarks.algorithms.flann', docker_tag='ann-benchmarks-flann', arguments=['angular', 0.7], query_argument_groups=[], disabled=False), Definition(algorithm='hnsw(nmslib)', constructor='NmslibReuseIndex', module='ann_benchmarks.algorithms.nmslib', docker_tag='ann-benchmarks-nmslib', arguments=['angular', 'hnsw', {'M': 12, 'post': 0, 'efConstruction': 800}, False], query_argument_groups=[[1], [2], [5], [10], [15], [20], [30], [40], [50], [70], [80]], disabled=False), Definition(algorithm='milvus', constructor='Milvus', module='ann_benchmarks.algorithms.milvus', docker_tag='ann-benchmarks-milvus', arguments=['angular', 'IVF_FLAT', 30000], query_argument_groups=[[1], [3], [10], [30], [100], [300]], disabled=False), Definition(algorithm='faiss-ivf', constructor='FaissIVF', module='ann_benchmarks.algorithms.faiss', docker_tag='ann-benchmarks-faiss', arguments=['angular', 2048], query_argument_groups=[[1], [5], [10], [50], [100], [200]], disabled=False), Definition(algorithm='n2', constructor='N2', module='ann_benchmarks.algorithms.n2', docker_tag='ann-benchmarks-n2', arguments=['angular', {'M': 16, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='BallTree(nmslib)', constructor='NmslibReuseIndex', module='ann_benchmarks.algorithms.nmslib', docker_tag='ann-benchmarks-nmslib', arguments=['angular', 'vptree', {'tuneK': 10, 'desiredRecall': 0.85}, False], query_argument_groups=[], disabled=False), Definition(algorithm='milvus', constructor='Milvus', module='ann_benchmarks.algorithms.milvus', docker_tag='ann-benchmarks-milvus', arguments=['angular', 'IVF_SQ8', 1000], query_argument_groups=[[1], [3], [10], [30], [100], [300]], disabled=False), Definition(algorithm='NGT-onng', constructor='ONNG', module='ann_benchmarks.algorithms.onng_ngt', docker_tag='ann-benchmarks-ngt', arguments=['angular', 'Float', 0.1, {'edge': 100, 'outdegree': 10, 'indegree': 120}], query_argument_groups=[[0.6], [0.9], [1.0], [1.02], [1.03], [1.05], [1.07], [1.1], [1.2]], disabled=False), Definition(algorithm='hnsw(faiss)', constructor='FaissHNSW', module='ann_benchmarks.algorithms.faiss_hnsw', docker_tag='ann-benchmarks-faiss', arguments=['angular', {'M': 12, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='rpforest', constructor='RPForest', module='ann_benchmarks.algorithms.rpforest', docker_tag='ann-benchmarks-rpforest', arguments=[100, 100], query_argument_groups=[], disabled=False), Definition(algorithm='rpforest', constructor='RPForest', module='ann_benchmarks.algorithms.rpforest', docker_tag='ann-benchmarks-rpforest', arguments=[10, 10], query_argument_groups=[], disabled=False), Definition(algorithm='BallTree(nmslib)', constructor='NmslibReuseIndex', module='ann_benchmarks.algorithms.nmslib', docker_tag='ann-benchmarks-nmslib', arguments=['angular', 'vptree', {'tuneK': 10, 'desiredRecall': 0.9}, False], query_argument_groups=[], disabled=False), Definition(algorithm='hnswlib', constructor='HnswLib', module='ann_benchmarks.algorithms.hnswlib', docker_tag='ann-benchmarks-hnswlib', arguments=['angular', {'M': 12, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='rpforest', constructor='RPForest', module='ann_benchmarks.algorithms.rpforest', docker_tag='ann-benchmarks-rpforest', arguments=[10, 3], query_argument_groups=[], disabled=False), Definition(algorithm='hnswlib', constructor='HnswLib', module='ann_benchmarks.algorithms.hnswlib', docker_tag='ann-benchmarks-hnswlib', arguments=['angular', {'M': 48, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='kd', constructor='KDTree', module='ann_benchmarks.algorithms.kdtree', docker_tag='ann-benchmarks-sklearn', arguments=['angular', 20], query_argument_groups=[], disabled=False), Definition(algorithm='milvus', constructor='Milvus', module='ann_benchmarks.algorithms.milvus', docker_tag='ann-benchmarks-milvus', arguments=['angular', 'IVF_FLAT', 1000], query_argument_groups=[[1], [3], [10], [30], [100], [300]], disabled=False), Definition(algorithm='flann', constructor='FLANN', module='ann_benchmarks.algorithms.flann', docker_tag='ann-benchmarks-flann', arguments=['angular', 0.5], query_argument_groups=[], disabled=False), Definition(algorithm='puffinn', constructor='Puffinn', module='ann_benchmarks.algorithms.puffinn', docker_tag='ann-benchmarks-puffinn', arguments=['angular', 536870912, 'fht_crosspolytope'], query_argument_groups=[[0.1], [0.2], [0.5], [0.7], [0.9], [0.95], [0.99]], disabled=False), Definition(algorithm='flann', constructor='FLANN', module='ann_benchmarks.algorithms.flann', docker_tag='ann-benchmarks-flann', arguments=['angular', 0.2], query_argument_groups=[], disabled=False), Definition(algorithm='kd', constructor='KDTree', module='ann_benchmarks.algorithms.kdtree', docker_tag='ann-benchmarks-sklearn', arguments=['angular', 400], query_argument_groups=[], disabled=False), Definition(algorithm='rpforest', constructor='RPForest', module='ann_benchmarks.algorithms.rpforest', docker_tag='ann-benchmarks-rpforest', arguments=[3, 100], query_argument_groups=[], disabled=False), Definition(algorithm='rpforest', constructor='RPForest', module='ann_benchmarks.algorithms.rpforest', docker_tag='ann-benchmarks-rpforest', arguments=[400, 100], query_argument_groups=[], disabled=False), Definition(algorithm='pynndescent', constructor='PyNNDescent', module='ann_benchmarks.algorithms.pynndescent', docker_tag='ann-benchmarks-pynndescent', arguments=['angular', {'n_neighbors': 40, 'diversify_prob': 0.5, 'pruning_degree_multiplier': 1.5, 'leaf_size': 25}], query_argument_groups=[[0.0], [0.02], [0.04], [0.06], [0.08], [0.1], [0.12], [0.14], [0.16]], disabled=False), Definition(algorithm='BallTree(nmslib)', constructor='NmslibReuseIndex', module='ann_benchmarks.algorithms.nmslib', docker_tag='ann-benchmarks-nmslib', arguments=['angular', 'vptree', {'tuneK': 10, 'desiredRecall': 0.1}, False], query_argument_groups=[], disabled=False), Definition(algorithm='rpforest', constructor='RPForest', module='ann_benchmarks.algorithms.rpforest', docker_tag='ann-benchmarks-rpforest', arguments=[100, 40], query_argument_groups=[], disabled=False), Definition(algorithm='n2', constructor='N2', module='ann_benchmarks.algorithms.n2', docker_tag='ann-benchmarks-n2', arguments=['angular', {'M': 4, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='milvus', constructor='Milvus', module='ann_benchmarks.algorithms.milvus', docker_tag='ann-benchmarks-milvus', arguments=['angular', 'IVF_SQ8', 30000], query_argument_groups=[[1], [3], [10], [30], [100], [300]], disabled=False), Definition(algorithm='rpforest', constructor='RPForest', module='ann_benchmarks.algorithms.rpforest', docker_tag='ann-benchmarks-rpforest', arguments=[400, 10], query_argument_groups=[], disabled=False), Definition(algorithm='milvus', constructor='Milvus', module='ann_benchmarks.algorithms.milvus', docker_tag='ann-benchmarks-milvus', arguments=['angular', 'IVF_FLAT', 100], query_argument_groups=[[1], [3], [10], [30], [100], [300]], disabled=False), Definition(algorithm='rpforest', constructor='RPForest', module='ann_benchmarks.algorithms.rpforest', docker_tag='ann-benchmarks-rpforest', arguments=[3, 400], query_argument_groups=[], disabled=False), Definition(algorithm='milvus', constructor='Milvus', module='ann_benchmarks.algorithms.milvus', docker_tag='ann-benchmarks-milvus', arguments=['angular', 'IVF_SQ8', 100], query_argument_groups=[[1], [3], [10], [30], [100], [300]], disabled=False), Definition(algorithm='pynndescent', constructor='PyNNDescent', module='ann_benchmarks.algorithms.pynndescent', docker_tag='ann-benchmarks-pynndescent', arguments=['angular', {'n_neighbors': 80, 'diversify_prob': 0.25, 'pruning_degree_multiplier': 2.0, 'leaf_size': 30}], query_argument_groups=[[0.08], [0.12], [0.16], [0.2], [0.24], [0.28], [0.32], [0.36]], disabled=False), Definition(algorithm='faiss-ivf', constructor='FaissIVF', module='ann_benchmarks.algorithms.faiss', docker_tag='ann-benchmarks-faiss', arguments=['angular', 128], query_argument_groups=[[1], [5], [10], [50], [100], [200]], disabled=False), Definition(algorithm='BallTree(nmslib)', constructor='NmslibReuseIndex', module='ann_benchmarks.algorithms.nmslib', docker_tag='ann-benchmarks-nmslib', arguments=['angular', 'vptree', {'tuneK': 10, 'desiredRecall': 0.7}, False], query_argument_groups=[], disabled=False), Definition(algorithm='rpforest', constructor='RPForest', module='ann_benchmarks.algorithms.rpforest', docker_tag='ann-benchmarks-rpforest', arguments=[400, 400], query_argument_groups=[], disabled=False), Definition(algorithm='hnsw(faiss)', constructor='FaissHNSW', module='ann_benchmarks.algorithms.faiss_hnsw', docker_tag='ann-benchmarks-faiss', arguments=['angular', {'M': 16, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='hnsw(faiss)', constructor='FaissHNSW', module='ann_benchmarks.algorithms.faiss_hnsw', docker_tag='ann-benchmarks-faiss', arguments=['angular', {'M': 64, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='pynndescent', constructor='PyNNDescent', module='ann_benchmarks.algorithms.pynndescent', docker_tag='ann-benchmarks-pynndescent', arguments=['angular', {'n_neighbors': 120, 'diversify_prob': 0.125, 'pruning_degree_multiplier': 2.5, 'leaf_size': 35}], query_argument_groups=[[0.16], [0.2], [0.24], [0.28], [0.32], [0.36]], disabled=False), Definition(algorithm='hnswlib', constructor='HnswLib', module='ann_benchmarks.algorithms.hnswlib', docker_tag='ann-benchmarks-hnswlib', arguments=['angular', {'M': 4, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='kd', constructor='KDTree', module='ann_benchmarks.algorithms.kdtree', docker_tag='ann-benchmarks-sklearn', arguments=['angular', 10], query_argument_groups=[], disabled=False), Definition(algorithm='hnswlib', constructor='HnswLib', module='ann_benchmarks.algorithms.hnswlib', docker_tag='ann-benchmarks-hnswlib', arguments=['angular', {'M': 24, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='annoy', constructor='Annoy', module='ann_benchmarks.algorithms.annoy', docker_tag='ann-benchmarks-annoy', arguments=['angular', 400], query_argument_groups=[[100], [200], [400], [1000], [2000], [4000], [10000], [20000], [40000], [100000], [200000], [400000]], disabled=False), Definition(algorithm='SW-graph(nmslib)', constructor='NmslibReuseIndex', module='ann_benchmarks.algorithms.nmslib', docker_tag='ann-benchmarks-nmslib', arguments=['angular', 'sw-graph', {'NN': 3}, False], query_argument_groups=[[120], [80], [60], [40], [20], [10], [8], [4], [2]], disabled=False), Definition(algorithm='rpforest', constructor='RPForest', module='ann_benchmarks.algorithms.rpforest', docker_tag='ann-benchmarks-rpforest', arguments=[100, 3], query_argument_groups=[], disabled=False), Definition(algorithm='rpforest', constructor='RPForest', module='ann_benchmarks.algorithms.rpforest', docker_tag='ann-benchmarks-rpforest', arguments=[40, 10], query_argument_groups=[], disabled=False), Definition(algorithm='hnswlib', constructor='HnswLib', module='ann_benchmarks.algorithms.hnswlib', docker_tag='ann-benchmarks-hnswlib', arguments=['angular', {'M': 36, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='rpforest', constructor='RPForest', module='ann_benchmarks.algorithms.rpforest', docker_tag='ann-benchmarks-rpforest', arguments=[40, 100], query_argument_groups=[], disabled=False), Definition(algorithm='flann', constructor='FLANN', module='ann_benchmarks.algorithms.flann', docker_tag='ann-benchmarks-flann', arguments=['angular', 0.95],query_argument_groups=[], disabled=False), Definition(algorithm='milvus', constructor='Milvus', module='ann_benchmarks.algorithms.milvus', docker_tag='ann-benchmarks-milvus', arguments=['angular', 'IVF_SQ8', 10000], query_argument_groups=[[1], [3], [10], [30], [100], [300]], disabled=False), Definition(algorithm='BallTree(nmslib)', constructor='NmslibReuseIndex', module='ann_benchmarks.algorithms.nmslib', docker_tag='ann-benchmarks-nmslib', arguments=['angular', 'vptree', {'tuneK': 10, 'desiredRecall': 0.6}, False], query_argument_groups=[], disabled=False), Definition(algorithm='milvus', constructor='Milvus', module='ann_benchmarks.algorithms.milvus', docker_tag='ann-benchmarks-milvus', arguments=['angular', 'IVF_SQ8', 3000], query_argument_groups=[[1], [3], [10], [30], [100], [300]], disabled=False), Definition(algorithm='hnswlib', constructor='HnswLib', module='ann_benchmarks.algorithms.hnswlib', docker_tag='ann-benchmarks-hnswlib', arguments=['angular', {'M': 64, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='pynndescent', constructor='PyNNDescent', module='ann_benchmarks.algorithms.pynndescent', docker_tag='ann-benchmarks-pynndescent', arguments=['angular', {'n_neighbors': 40, 'diversify_prob': 1.0, 'pruning_degree_multiplier': 1.5, 'leaf_size': 25}], query_argument_groups=[[0.0], [0.02], [0.04], [0.06], [0.08], [0.1], [0.12], [0.14], [0.16]], disabled=False), Definition(algorithm='rpforest', constructor='RPForest', module='ann_benchmarks.algorithms.rpforest', docker_tag='ann-benchmarks-rpforest', arguments=[40, 3], query_argument_groups=[], disabled=False), Definition(algorithm='milvus', constructor='Milvus', module='ann_benchmarks.algorithms.milvus', docker_tag='ann-benchmarks-milvus', arguments=['angular', 'IVF_SQ8', 300], query_argument_groups=[[1], [3], [10], [30], [100], [300]], disabled=False), Definition(algorithm='hnsw(faiss)', constructor='FaissHNSW', module='ann_benchmarks.algorithms.faiss_hnsw', docker_tag='ann-benchmarks-faiss', arguments=['angular', {'M': 4, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='pynndescent', constructor='PyNNDescent', module='ann_benchmarks.algorithms.pynndescent', docker_tag='ann-benchmarks-pynndescent', arguments=['angular', {'n_neighbors': 80, 'diversify_prob': 1.0, 'pruning_degree_multiplier': 2.0, 'leaf_size': 20}], query_argument_groups=[[0.0], [0.02], [0.04], [0.08], [0.12], [0.16], [0.2], [0.24]], disabled=False), Definition(algorithm='pynndescent', constructor='PyNNDescent', module='ann_benchmarks.algorithms.pynndescent', docker_tag='ann-benchmarks-pynndescent', arguments=['angular', {'n_neighbors': 20, 'diversify_prob': 1.0, 'pruning_degree_multiplier': 0.5, 'leaf_size': 20}], query_argument_groups=[[0.0], [0.02], [0.04], [0.06], [0.08], [0.1], [0.12]], disabled=False), Definition(algorithm='flann', constructor='FLANN', module='ann_benchmarks.algorithms.flann', docker_tag='ann-benchmarks-flann', arguments=['angular', 0.97], query_argument_groups=[], disabled=False), Definition(algorithm='rpforest', constructor='RPForest', module='ann_benchmarks.algorithms.rpforest', docker_tag='ann-benchmarks-rpforest', arguments=[3, 3], query_argument_groups=[], disabled=False), Definition(algorithm='milvus', constructor='Milvus', module='ann_benchmarks.algorithms.milvus', docker_tag='ann-benchmarks-milvus', arguments=['angular', 'IVF_FLAT', 300], query_argument_groups=[[1], [3], [10], [30], [100], [300]], disabled=False), Definition(algorithm='faiss-ivf', constructor='FaissIVF', module='ann_benchmarks.algorithms.faiss', docker_tag='ann-benchmarks-faiss', arguments=['angular', 32], query_argument_groups=[[1], [5], [10], [50], [100], [200]], disabled=False), Definition(algorithm='faiss-ivf', constructor='FaissIVF', module='ann_benchmarks.algorithms.faiss', docker_tag='ann-benchmarks-faiss', arguments=['angular', 4096], query_argument_groups=[[1], [5], [10], [50], [100], [200]], disabled=False), Definition(algorithm='pynndescent', constructor='PyNNDescent', module='ann_benchmarks.algorithms.pynndescent', docker_tag='ann-benchmarks-pynndescent', arguments=['angular', {'n_neighbors': 20, 'diversify_prob': 1.0, 'pruning_degree_multiplier': 1.0, 'leaf_size': 20}], query_argument_groups=[[0.0], [0.02], [0.04], [0.06], [0.08], [0.1], [0.12]], disabled=False), Definition(algorithm='faiss-ivf', constructor='FaissIVF', module='ann_benchmarks.algorithms.faiss', docker_tag='ann-benchmarks-faiss', arguments=['angular', 64], query_argument_groups=[[1], [5], [10], [50], [100], [200]], disabled=False), Definition(algorithm='rpforest', constructor='RPForest', module='ann_benchmarks.algorithms.rpforest', docker_tag='ann-benchmarks-rpforest', arguments=[100, 400], query_argument_groups=[], disabled=False), Definition(algorithm='annoy', constructor='Annoy', module='ann_benchmarks.algorithms.annoy', docker_tag='ann-benchmarks-annoy', arguments=['angular', 200], query_argument_groups=[[100], [200], [400], [1000], [2000], [4000], [10000], [20000], [40000], [100000], [200000], [400000]], disabled=False), Definition(algorithm='hnsw(faiss)', constructor='FaissHNSW', module='ann_benchmarks.algorithms.faiss_hnsw', docker_tag='ann-benchmarks-faiss', arguments=['angular', {'M': 96, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='rpforest', constructor='RPForest', module='ann_benchmarks.algorithms.rpforest', docker_tag='ann-benchmarks-rpforest', arguments=[3, 40], query_argument_groups=[], disabled=False), Definition(algorithm='BallTree(nmslib)', constructor='NmslibReuseIndex', module='ann_benchmarks.algorithms.nmslib', docker_tag='ann-benchmarks-nmslib', arguments=['angular', 'vptree', {'tuneK': 10, 'desiredRecall': 0.97}, False], query_argument_groups=[], disabled=False), Definition(algorithm='kgraph', constructor='KGraph', module='ann_benchmarks.algorithms.kgraph', docker_tag='ann-benchmarks-kgraph', arguments=['angular', {'reverse': -1, 'K': 200, 'L': 300, 'S': 20}, False], query_argument_groups=[[1], [2], [3], [4], [5], [10], [20], [30], [40], [50], [60], [70], [80], [90], [100]], disabled=False), Definition(algorithm='n2', constructor='N2', module='ann_benchmarks.algorithms.n2', docker_tag='ann-benchmarks-n2', arguments=['angular', {'M': 24, 'efConstruction': 500}], query_argument_groups=[[10], [20], [40], [80], [120], [200], [400], [600], [800]], disabled=False), Definition(algorithm='BallTree(nmslib)', constructor='NmslibReuseIndex', module='ann_benchmarks.algorithms.nmslib', docker_tag='ann-benchmarks-nmslib', arguments=['angular', 'vptree', {'tuneK': 10, 'desiredRecall': 0.95}, False], query_argument_groups=[], disabled=False), Definition(algorithm='faiss-ivf', constructor='FaissIVF', module='ann_benchmarks.algorithms.faiss', docker_tag='ann-benchmarks-faiss', arguments=['angular', 8192], query_argument_groups=[[1], [5], [10], [50], [100], [200]], disabled=False)] 2021-02-20 13:04:00,646 - annb.99aabfb480 - INFO - Created container 99aabfb480: CPU limit 1, mem limit 207765490432, timeout 7200, command ['--dataset', 'glove-100-angular', '--algorithm', 'NGT-panng', '--module', 'ann_benchmarks.algorithms.panng_ngt', '--constructor', 'PANNG', '--runs', '5', '--count', '10', '["angular", "Float", {"edge": 20, "pathadj": 40, "searchedge": 60}]', '[0.6]', '[0.8]', '[0.9]', '[1.0]', '[1.02]', '[1.05]', '[1.1]', '[1.2]'] 2021-02-20 13:04:00,666 - annb.99aabfb480 - INFO - Traceback (most recent call last): 2021-02-20 13:04:00,666 - annb.99aabfb480 - INFO - File "run_algorithm.py", line 2, in 2021-02-20 13:04:00,666 - annb.99aabfb480 - INFO - from ann_benchmarks.runner import run_from_cmdline 2021-02-20 13:04:00,666 - annb.99aabfb480 - INFO - ModuleNotFoundError: No module named 'ann_benchmarks.runner' 2021-02-20 13:04:00,937 - annb.99aabfb480 - ERROR - Traceback (most recent call last): File "run_algorithm.py", line 2, in from ann_benchmarks.runner import run_from_cmdline ModuleNotFoundError: No module named 'ann_benchmarks.runner'

2021-02-20 13:04:00,937 - annb.99aabfb480 - ERROR - Child process for container 99aabfb480 raised exception 1

ScreenShot

erikbern / ann-benchmarks

ModuleNotFoundError: No module named 'ann_benchmarks.runner' #218

replay method

log