stanford-futuredata / ColBERT

ColBERT: state-of-the-art neural search (SIGIR'20, TACL'21, NeurIPS'21, NAACL'22, CIKM'22, ACL'23, EMNLP'23)
MIT License
2.67k stars 355 forks source link

Execution fails in colbert.index_objs() with assert classname.endswith('Vector') #340

Open MayaBercovitch opened 2 months ago

MayaBercovitch commented 2 months ago

Hi, Can you please explain why I'm getting this error? I have a function using ColBERT to index some text objects and it keeps failing with this error. I'm using the CPU version.

Thanks


The full traceback is: Traceback (most recent call last): File ".pyenv/versions/3.11.5/lib/python3.11/multiprocessing/process.py", line 314, in _bootstrap self.run() File ".pyenv/versions/3.11.5/lib/python3.11/multiprocessing/process.py", line 108, in run self._target(*self._args, *self._kwargs) File ".pyenv/versions/solid3.11/lib/python3.11/site-packages/colbert/infra/launcher.py", line 134, in setup_new_process return_val = callee(config, args) ^^^^^^^^^^^^^^^^^^^^^ File ".pyenv/versions/solid3.11/lib/python3.11/site-packages/colbert/indexing/collection_indexer.py", line 33, in encode encoder.run(shared_lists) File "pyenv/versions/solid3.11/lib/python3.11/site-packages/colbert/indexing/collection_indexer.py", line 68, in run self.train(shared_lists) # Trains centroids from selected passages ^^^^^^^^^^^^^^^^^^^^^^^^ File ".pyenv/versions/solid3.11/lib/python3.11/site-packages/colbert/indexing/collection_indexer.py", line 232, in train centroids = self._train_kmeans(sample, shared_lists) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File ".pyenv/versions/solid3.11/lib/python3.11/site-packages/colbert/indexing/collection_indexer.py", line 304, in _train_kmeans centroids = compute_faisskmeans(*args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File ".pyenv/versions/solid3.11/lib/python3.11/site-packages/colbert/indexing/collection_indexer.py", line 507, in compute_faiss_kmeans kmeans.train(sample) File ".pyenv/versions/solid3.11/lib/python3.11/site-packages/faiss/extra_wrappers.py", line 564, in train centroids = faiss.vector_float_to_array(clus.centroids) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File ".pyenv/versions/solid3.11/lib/python3.11/site-packages/faiss/array_conversions.py", line 118, in vector_float_to_array return vector_to_array(v) ^^^^^^^^^^^^^^^^^^ File "pyenv/versions/solid3.11/lib/python3.11/site-packages/faiss/array_conversions.py", line 109, in vector_to_array assert classname.endswith('Vector') AssertionError