Closed gojira closed 6 months ago
Hey!
No, this very much isn't expected. Tried it out on M1 and it took <30s to index the Miyazaki example, which seems to be what you're doing here!
Are you running the standard notebook example on the latest version? I can't reproduce the issue on weaker hardware than yours via VSCode Jupyter (M1 Air 8GB, Python 3.11, indexing time ~27s)🤔
Yes - 01-basic_indexing_and_search.ipynb.
I am still seeing same after doing the following:
When I interrupt it shows multiprocessing
so is this the same issue as if __name__ == "__main__"
?
{
"name": "KeyboardInterrupt",
"message": "",
"stack": "---------------------------------------------------------------------------
KeyboardInterrupt Traceback (most recent call last)
Cell In[4], line 1
----> 1 RAG.index(collection=[full_document], index_name=\"Miyazaki\", max_document_length=180, split_documents=True)
File ~/mambaforge/envs/raga/lib/python3.9/site-packages/ragatouille/RAGPretrainedModel.py:125, in RAGPretrainedModel.index(self, collection, index_name, overwrite_index, max_document_length, split_documents, document_splitter_fn, preprocessing_fn)
123 if overwrite_index:
124 overwrite = True
--> 125 return self.model.index(
126 collection,
127 index_name,
128 max_document_length=max_document_length,
129 overwrite=overwrite,
130 )
File ~/mambaforge/envs/raga/lib/python3.9/site-packages/ragatouille/models/colbert.py:203, in ColBERT.index(self, collection, index_name, max_document_length, overwrite)
197 self.config.avoid_fork_if_possible = True
198 self.indexer = Indexer(
199 checkpoint=self.checkpoint,
200 config=self.config,
201 verbose=self.verbose,
202 )
--> 203 self.indexer.index(
204 name=self.index_name, collection=collection, overwrite=overwrite
205 )
207 index_path = str(
208 Path(self.run_config.root)
209 / Path(self.run_config.experiment)
210 / \"indexes\"
211 / self.index_name
212 )
213 self.config.root = str(
214 Path(self.run_config.root) / Path(self.run_config.experiment) / \"indexes\"
215 )
File ~/mambaforge/envs/raga/lib/python3.9/site-packages/colbert/indexer.py:78, in Indexer.index(self, name, collection, overwrite)
75 self.erase()
77 if index_does_not_exist or overwrite != 'reuse':
---> 78 self.__launch(collection)
80 return self.index_path
File ~/mambaforge/envs/raga/lib/python3.9/site-packages/colbert/indexer.py:89, in Indexer.__launch(self, collection)
87 # Encodes collection into index using the CollectionIndexer class
88 launcher = Launcher(encode)
---> 89 launcher.launch(self.config, collection, shared_lists, shared_queues, self.verbose)
File ~/mambaforge/envs/raga/lib/python3.9/site-packages/colbert/infra/launcher.py:79, in Launcher.launch(self, custom_config, *args)
75 print_memory_stats('MAIN')
77 # TODO: If the processes crash upon join, raise an exception and don't block on .get() below!
---> 79 return_values = sorted([return_value_queue.get() for _ in all_procs])
80 return_values = [val for rank, val in return_values]
82 if not self.return_all:
File ~/mambaforge/envs/raga/lib/python3.9/site-packages/colbert/infra/launcher.py:79, in <listcomp>(.0)
75 print_memory_stats('MAIN')
77 # TODO: If the processes crash upon join, raise an exception and don't block on .get() below!
---> 79 return_values = sorted([return_value_queue.get() for _ in all_procs])
80 return_values = [val for rank, val in return_values]
82 if not self.return_all:
File ~/mambaforge/envs/raga/lib/python3.9/multiprocessing/queues.py:103, in Queue.get(self, block, timeout)
101 if block and timeout is None:
102 with self._rlock:
--> 103 res = self._recv_bytes()
104 self._sem.release()
105 else:
File ~/mambaforge/envs/raga/lib/python3.9/multiprocessing/connection.py:216, in _ConnectionBase.recv_bytes(self, maxlength)
214 if maxlength is not None and maxlength < 0:
215 raise ValueError(\"negative maxlength\")
--> 216 buf = self._recv_bytes(maxlength)
217 if buf is None:
218 self._bad_message_length()
File ~/mambaforge/envs/raga/lib/python3.9/multiprocessing/connection.py:414, in Connection._recv_bytes(self, maxsize)
413 def _recv_bytes(self, maxsize=None):
--> 414 buf = self._recv(4)
415 size, = struct.unpack(\"!i\", buf.getvalue())
416 if size == -1:
File ~/mambaforge/envs/raga/lib/python3.9/multiprocessing/connection.py:379, in Connection._recv(self, size, read)
377 remaining = size
378 while remaining > 0:
--> 379 chunk = read(handle, remaining)
380 n = len(chunk)
381 if n == 0:
KeyboardInterrupt: "
}
It's very odd that it'd hang on a Mac, I think we've had a few hundred people do it locally already and it never occurred 🤔
Although it does reveal another issue -- @anmol6 I've pinged you, we need a more reliable way of bypassing forking, currently RAGatouille has no easy way of enforcing it in every case.
@gojira I've tried a new env on 3.9 as well, and everything runs smoothly... I think once we fixed the flawed MP-bypass implementation it'll also solve your problem!
It's been a little while since I've had things break in AI this way - I guess it is truly cutting edge ;)!
😄
Multiprocessing should now be properly disabled when fewer than 2 GPUs are detected -- let me know if the update fixes your issue!
I installed the following and it still hangs. Now VS Code has problems interrupting the kernel - so I can't get a trace like I did last time
Downloading ragatouille-0.0.4b2-py3-none-any.whl (30 kB)
Downloading colbert_ai-0.2.17-py3-none-any.whl (113 kB)
This is very strange, especially considering it seems to be dependant on something very specific, but utterly unclear what 🤔 ...
Would you mind posting your pip freeze? I'll mark this issue as Help Wanted and it'd be great if someone managed to figure out exactly what's causing it.
Yes - here you go - I deleted it before
Babel @ file:///home/conda/feedstock_root/build_artifacts/babel_1702422572539/work
Brotli @ file:///Users/runner/miniforge3/conda-bld/brotli-split_1695989934239/work
Deprecated==1.2.14
Flask==3.0.0
GitPython==3.1.41
Jinja2 @ file:///home/conda/feedstock_root/build_artifacts/jinja2_1704966972576/work
MarkupSafe @ file:///Users/runner/miniforge3/conda-bld/markupsafe_1695367493730/work
PySocks @ file:///home/conda/feedstock_root/build_artifacts/pysocks_1661604839144/work
PyYAML @ file:///Users/runner/miniforge3/conda-bld/pyyaml_1695373497981/work
Pygments @ file:///home/conda/feedstock_root/build_artifacts/pygments_1700607939962/work
QtPy @ file:///home/conda/feedstock_root/build_artifacts/qtpy_1698112029416/work
RAGatouille==0.0.4b2
SQLAlchemy==2.0.25
Send2Trash @ file:///Users/runner/miniforge3/conda-bld/send2trash_1682601407921/work
Werkzeug==3.0.1
aiohttp==3.9.1
aiosignal==1.3.1
annotated-types==0.6.0
anyio @ file:///home/conda/feedstock_root/build_artifacts/anyio_1702909220329/work
appnope @ file:///home/conda/feedstock_root/build_artifacts/appnope_1649077682618/work
argon2-cffi @ file:///home/conda/feedstock_root/build_artifacts/argon2-cffi_1692818318753/work
argon2-cffi-bindings @ file:///Users/runner/miniforge3/conda-bld/argon2-cffi-bindings_1695386652227/work
arrow @ file:///home/conda/feedstock_root/build_artifacts/arrow_1696128962909/work
asttokens @ file:///home/conda/feedstock_root/build_artifacts/asttokens_1698341106958/work
async-lru @ file:///home/conda/feedstock_root/build_artifacts/async-lru_1690563019058/work
async-timeout==4.0.3
attrs @ file:///home/conda/feedstock_root/build_artifacts/attrs_1704011227531/work
beautifulsoup4 @ file:///home/conda/feedstock_root/build_artifacts/beautifulsoup4_1680888073205/work
bitarray==2.9.2
bleach @ file:///home/conda/feedstock_root/build_artifacts/bleach_1696630167146/work
blinker==1.7.0
blis==0.7.11
cached-property @ file:///home/conda/feedstock_root/build_artifacts/cached_property_1615209429212/work
catalogue==2.0.10
certifi @ file:///home/conda/feedstock_root/build_artifacts/certifi_1700303426725/work/certifi
cffi @ file:///Users/runner/miniforge3/conda-bld/cffi_1696001750083/work
charset-normalizer @ file:///home/conda/feedstock_root/build_artifacts/charset-normalizer_1698833585322/work
click==8.1.7
cloudpathlib==0.16.0
colbert-ai==0.2.17
comm @ file:///home/conda/feedstock_root/build_artifacts/comm_1704278392174/work
confection==0.1.4
cymem==2.0.8
dataclasses-json==0.6.3
datasets==2.16.1
debugpy @ file:///Users/runner/miniforge3/conda-bld/debugpy_1695534371680/work
decorator @ file:///home/conda/feedstock_root/build_artifacts/decorator_1641555617451/work
defusedxml @ file:///home/conda/feedstock_root/build_artifacts/defusedxml_1615232257335/work
dill==0.3.7
distro==1.9.0
entrypoints @ file:///home/conda/feedstock_root/build_artifacts/entrypoints_1643888246732/work
exceptiongroup @ file:///home/conda/feedstock_root/build_artifacts/exceptiongroup_1704921103267/work
executing @ file:///home/conda/feedstock_root/build_artifacts/executing_1698579936712/work
faiss-cpu==1.7.4
fastjsonschema @ file:///home/conda/feedstock_root/build_artifacts/python-fastjsonschema_1703780968325/work/dist
filelock @ file:///home/conda/feedstock_root/build_artifacts/filelock_1698714947081/work
fqdn @ file:///home/conda/feedstock_root/build_artifacts/fqdn_1638810296540/work/dist
frozenlist==1.4.1
fsspec==2023.10.0
git-python==1.0.3
gitdb==4.0.11
gmpy2 @ file:///Users/runner/miniforge3/conda-bld/gmpy2_1666808750085/work
greenlet==3.0.3
h11==0.14.0
httpcore==1.0.2
httpx==0.26.0
huggingface-hub==0.20.2
idna @ file:///home/conda/feedstock_root/build_artifacts/idna_1701026962277/work
importlib-metadata @ file:///home/conda/feedstock_root/build_artifacts/importlib-metadata_1703269254275/work
importlib-resources @ file:///home/conda/feedstock_root/build_artifacts/importlib_resources_1699364556997/work
ipykernel @ file:///Users/runner/miniforge3/conda-bld/ipykernel_1703631823913/work
ipython @ file:///home/conda/feedstock_root/build_artifacts/ipython_1701831663892/work
ipywidgets @ file:///home/conda/feedstock_root/build_artifacts/ipywidgets_1694607144474/work
isoduration @ file:///home/conda/feedstock_root/build_artifacts/isoduration_1638811571363/work/dist
itsdangerous==2.1.2
jedi @ file:///home/conda/feedstock_root/build_artifacts/jedi_1696326070614/work
joblib==1.3.2
json5 @ file:///home/conda/feedstock_root/build_artifacts/json5_1688248289187/work
jsonpatch==1.33
jsonpointer @ file:///Users/runner/miniforge3/conda-bld/jsonpointer_1695397322962/work
jsonschema @ file:///home/conda/feedstock_root/build_artifacts/jsonschema-meta_1700159890288/work
jsonschema-specifications @ file:///tmp/tmpkv1z7p57/src
jupyter @ file:///home/conda/feedstock_root/build_artifacts/jupyter_1696255489086/work
jupyter-console @ file:///home/conda/feedstock_root/build_artifacts/jupyter_console_1678118109161/work
jupyter-events @ file:///home/conda/feedstock_root/build_artifacts/jupyter_events_1699285872613/work
jupyter-lsp @ file:///home/conda/feedstock_root/build_artifacts/jupyter-lsp-meta_1701091994466/work/jupyter-lsp
jupyter_client @ file:///home/conda/feedstock_root/build_artifacts/jupyter_client_1699283905679/work
jupyter_core @ file:///Users/runner/miniforge3/conda-bld/jupyter_core_1704727402891/work
jupyter_server @ file:///home/conda/feedstock_root/build_artifacts/jupyter_server_1704986851950/work
jupyter_server_terminals @ file:///home/conda/feedstock_root/build_artifacts/jupyter_server_terminals_1703611053195/work
jupyterlab @ file:///home/conda/feedstock_root/build_artifacts/jupyterlab_1703881392925/work
jupyterlab-widgets @ file:///home/conda/feedstock_root/build_artifacts/jupyterlab_widgets_1694598704522/work
jupyterlab_pygments @ file:///home/conda/feedstock_root/build_artifacts/jupyterlab_pygments_1700744013163/work
jupyterlab_server @ file:///home/conda/feedstock_root/build_artifacts/jupyterlab_server-split_1700310846957/work
langchain-community==0.0.12
langchain-core==0.1.10
langchain==0.1.0
langcodes==3.3.0
langsmith==0.0.80
llama-index==0.9.31
marshmallow==3.20.2
matplotlib-inline @ file:///home/conda/feedstock_root/build_artifacts/matplotlib-inline_1660814786464/work
mistune @ file:///home/conda/feedstock_root/build_artifacts/mistune_1698947099619/work
mpmath @ file:///home/conda/feedstock_root/build_artifacts/mpmath_1678228039184/work
multidict==6.0.4
multiprocess==0.70.15
murmurhash==1.0.10
mypy-extensions==1.0.0
nbclient @ file:///home/conda/feedstock_root/build_artifacts/nbclient_1684790896106/work
nbconvert @ file:///home/conda/feedstock_root/build_artifacts/nbconvert-meta_1704986998851/work
nbformat @ file:///home/conda/feedstock_root/build_artifacts/nbformat_1690814868471/work
nest-asyncio @ file:///home/conda/feedstock_root/build_artifacts/nest-asyncio_1697083700168/work
networkx @ file:///home/conda/feedstock_root/build_artifacts/networkx_1698504735452/work
ninja==1.11.1.1
nltk==3.8.1
notebook @ file:///home/conda/feedstock_root/build_artifacts/notebook_1697550696415/work
notebook_shim @ file:///home/conda/feedstock_root/build_artifacts/notebook-shim_1682360583588/work
numpy @ file:///Users/runner/miniforge3/conda-bld/numpy_1704280477691/work/dist/numpy-1.26.3-cp39-cp39-macosx_11_0_arm64.whl#sha256=b4904f3177fc1d83f3a54b101f5f80740cd57b4e131aaed5716bf3488e4de663
onnx==1.15.0
openai==1.7.2
overrides @ file:///home/conda/feedstock_root/build_artifacts/overrides_1691338815398/work
packaging @ file:///home/conda/feedstock_root/build_artifacts/packaging_1696202382185/work
pandas==2.1.4
pandocfilters @ file:///home/conda/feedstock_root/build_artifacts/pandocfilters_1631603243851/work
parso @ file:///home/conda/feedstock_root/build_artifacts/parso_1638334955874/work
pexpect @ file:///home/conda/feedstock_root/build_artifacts/pexpect_1667297516076/work
pickleshare @ file:///home/conda/feedstock_root/build_artifacts/pickleshare_1602536217715/work
pillow @ file:///Users/runner/miniforge3/conda-bld/pillow_1704252271887/work
pkgutil_resolve_name @ file:///home/conda/feedstock_root/build_artifacts/pkgutil-resolve-name_1694617248815/work
platformdirs @ file:///home/conda/feedstock_root/build_artifacts/platformdirs_1701708255999/work
preshed==3.0.9
prometheus-client @ file:///home/conda/feedstock_root/build_artifacts/prometheus_client_1700579315247/work
prompt-toolkit @ file:///home/conda/feedstock_root/build_artifacts/prompt-toolkit_1702399386289/work
protobuf==4.25.2
psutil @ file:///Users/runner/miniforge3/conda-bld/psutil_1702833188708/work
ptyprocess @ file:///home/conda/feedstock_root/build_artifacts/ptyprocess_1609419310487/work/dist/ptyprocess-0.7.0-py2.py3-none-any.whl
pure-eval @ file:///home/conda/feedstock_root/build_artifacts/pure_eval_1642875951954/work
pyarrow-hotfix==0.6
pyarrow==14.0.2
pycparser @ file:///home/conda/feedstock_root/build_artifacts/pycparser_1636257122734/work
pydantic==2.5.3
pydantic_core==2.14.6
pyobjc-core @ file:///Users/runner/miniforge3/conda-bld/pyobjc-core_1702119468109/work
pyobjc-framework-Cocoa @ file:///Users/runner/miniforge3/conda-bld/pyobjc-framework-cocoa_1702125476861/work
python-dateutil @ file:///home/conda/feedstock_root/build_artifacts/python-dateutil_1626286286081/work
python-dotenv==1.0.0
python-json-logger @ file:///home/conda/feedstock_root/build_artifacts/python-json-logger_1677079630776/work
pytz @ file:///home/conda/feedstock_root/build_artifacts/pytz_1693930252784/work
pyzmq @ file:///Users/runner/miniforge3/conda-bld/pyzmq_1701783274137/work
qtconsole @ file:///home/conda/feedstock_root/build_artifacts/qtconsole-base_1700168901209/work
referencing @ file:///home/conda/feedstock_root/build_artifacts/referencing_1704489226496/work
regex==2023.12.25
requests @ file:///home/conda/feedstock_root/build_artifacts/requests_1684774241324/work
rfc3339-validator @ file:///home/conda/feedstock_root/build_artifacts/rfc3339-validator_1638811747357/work
rfc3986-validator @ file:///home/conda/feedstock_root/build_artifacts/rfc3986-validator_1598024191506/work
rpds-py @ file:///Users/runner/miniforge3/conda-bld/rpds-py_1705159905602/work
ruff==0.1.13
safetensors==0.4.1
scikit-learn==1.3.2
scipy==1.11.4
sentence-transformers==2.2.2
sentencepiece==0.1.99
six @ file:///home/conda/feedstock_root/build_artifacts/six_1620240208055/work
smart-open==6.4.0
smmap==5.0.1
sniffio @ file:///home/conda/feedstock_root/build_artifacts/sniffio_1662051266223/work
soupsieve @ file:///home/conda/feedstock_root/build_artifacts/soupsieve_1693929250441/work
spacy-legacy==3.0.12
spacy-loggers==1.0.5
spacy==3.7.2
srsly==2.4.8
stack-data @ file:///home/conda/feedstock_root/build_artifacts/stack_data_1669632077133/work
sympy @ file:///home/conda/feedstock_root/build_artifacts/sympy_1684180540116/work
tenacity==8.2.3
terminado @ file:///Users/runner/miniforge3/conda-bld/terminado_1699810180257/work
thinc==8.2.2
threadpoolctl==3.2.0
tiktoken==0.5.2
tinycss2 @ file:///home/conda/feedstock_root/build_artifacts/tinycss2_1666100256010/work
tokenizers==0.15.0
tomli @ file:///home/conda/feedstock_root/build_artifacts/tomli_1644342247877/work
torch==2.1.2
torchaudio==2.1.2
torchvision==0.16.2
tornado @ file:///Users/runner/miniforge3/conda-bld/tornado_1695373661905/work
tqdm==4.66.1
traitlets @ file:///home/conda/feedstock_root/build_artifacts/traitlets_1704212992681/work
transformers==4.36.2
typer==0.9.0
types-python-dateutil @ file:///home/conda/feedstock_root/build_artifacts/types-python-dateutil_1704512562698/work
typing-inspect==0.9.0
typing-utils @ file:///home/conda/feedstock_root/build_artifacts/typing_utils_1622899189314/work
typing_extensions @ file:///home/conda/feedstock_root/build_artifacts/typing_extensions_1702176139754/work
tzdata==2023.4
ujson==5.9.0
uri-template @ file:///home/conda/feedstock_root/build_artifacts/uri-template_1688655812972/work/dist
urllib3 @ file:///home/conda/feedstock_root/build_artifacts/urllib3_1699933488691/work
voyager==2.0.2
wasabi==1.1.2
wcwidth @ file:///home/conda/feedstock_root/build_artifacts/wcwidth_1704731205417/work
weasel==0.3.4
webcolors @ file:///home/conda/feedstock_root/build_artifacts/webcolors_1679900785843/work
webencodings @ file:///home/conda/feedstock_root/build_artifacts/webencodings_1694681268211/work
websocket-client @ file:///home/conda/feedstock_root/build_artifacts/websocket-client_1701630677416/work
widgetsnbextension @ file:///home/conda/feedstock_root/build_artifacts/widgetsnbextension_1694598693908/work
wrapt==1.16.0
xxhash==3.4.1
yarl==1.9.4
zipp @ file:///home/conda/feedstock_root/build_artifacts/zipp_1695255097490/work
@gojira can you provide the exact code you're running via a script file/gist + how you're creating your environment?
Hi guys, first and foremost, thanks a bunch for this lib !
I think I have the same issue (M3 pro 64gb)
Was running python3.10, then tried with 3.9 with no change in the output (didn't wait for the program to finish but let it run > 15 min a couple of times)
install python3.9 -m venv ragvenv pip install ragatouille (then running the Miyazaki example (01))
pip freeze output: aiohttp==3.9.1 aiosignal==1.3.1 annotated-types==0.6.0 anyio==4.2.0 async-timeout==4.0.3 attrs==23.2.0 bitarray==2.9.2 blinker==1.7.0 catalogue==2.0.10 certifi==2024.2.2 charset-normalizer==3.3.2 click==8.1.7 colbert-ai==0.2.19 dataclasses-json==0.6.4 datasets==2.17.0 Deprecated==1.2.14 dill==0.3.8 dirtyjson==1.0.8 distro==1.9.0 exceptiongroup==1.2.0 faiss-cpu==1.7.4 filelock==3.13.1 Flask==3.0.2 frozenlist==1.4.1 fsspec==2023.10.0 git-python==1.0.3 gitdb==4.0.11 GitPython==3.1.41 greenlet==3.0.3 h11==0.14.0 httpcore==1.0.2 httpx==0.26.0 huggingface-hub==0.20.3 idna==3.6 importlib-metadata==7.0.1 itsdangerous==2.1.2 Jinja2==3.1.3 joblib==1.3.2 jsonpatch==1.33 jsonpointer==2.4 langchain==0.1.7 langchain-community==0.0.20 langchain-core==0.1.23 langsmith==0.0.87 llama-index==0.9.48 MarkupSafe==2.1.5 marshmallow==3.20.2 mpmath==1.3.0 multidict==6.0.5 multiprocess==0.70.16 mypy-extensions==1.0.0 nest-asyncio==1.6.0 networkx==3.2.1 ninja==1.11.1.1 nltk==3.8.1 numpy==1.26.4 onnx==1.15.0 openai==1.12.0 packaging==23.2 pandas==2.2.0 pillow==10.2.0 protobuf==4.25.2 pyarrow==15.0.0 pyarrow-hotfix==0.6 pydantic==2.6.1 pydantic_core==2.16.2 python-dateutil==2.8.2 python-dotenv==1.0.1 pytz==2024.1 PyYAML==6.0.1 RAGatouille==0.0.7.post2 regex==2023.12.25 requests==2.31.0 ruff==0.1.15 safetensors==0.4.2 scikit-learn==1.4.0 scipy==1.12.0 sentence-transformers==2.3.1 sentencepiece==0.1.99 six==1.16.0 smmap==5.0.1 sniffio==1.3.0 SQLAlchemy==2.0.27 srsly==2.4.8 sympy==1.12 tenacity==8.2.3 threadpoolctl==3.2.0 tiktoken==0.6.0 tokenizers==0.15.2 torch==2.2.0 tqdm==4.66.2 transformers==4.37.2 typing-inspect==0.9.0 typing_extensions==4.9.0 tzdata==2024.1 ujson==5.9.0 urllib3==2.2.0 voyager==2.0.2 Werkzeug==3.0.1 wrapt==1.16.0 xxhash==3.4.1 yarl==1.9.4 zipp==3.17.0
Let me know if I can provide more information !
(Copy/pasting this message in a few related issues)
Hey guys!
Thanks a lot for bearing with me as I juggle everything and trying to diagnose this. It’s complicated to fix with relatively little time to dedicate to it, as it seems like the dependencies causing issues aren’t the same for everyone, with no clear platform pattern as of yet. Overall, the issues center around the usual suspects of faiss
and CUDA
.
While because of this I can’t fix the issue with PLAID optimised indices just yet, I’m also noticing that most of the bug reports here are about relatively small collections (100s-to-low-1000s). To lower the barrier to entry as much as possible, https://github.com/bclavie/RAGatouille/pull/137 is introducing a second index format, which doesn’t actually build an index, but performs an exact search over all documents (as a stepping stone towards https://github.com/bclavie/RAGatouille/issues/110, which would use an HNSW index to be an in-between compromise between PLAID optimisation and exact search). This approach doesn’t scale, but offers the best possible search accuracy & is still performed in a few hundred milliseconds at most for small collections. Ideally, it’ll also open up the way to shipping lower-dependency versions (https://github.com/bclavie/RAGatouille/issues/136)
The PR above (https://github.com/bclavie/RAGatouille/pull/137) is still a work in progress, as it needs CRUD support, tests, documentation, better precision routing (fp32/bfloat16) etc… (and potentially searching only subset of document ids).
However, it’s working in a rough state for me locally. If you’d like to give it a try (with the caveat that it might very well break!), please feel free to install the library directly from the feat/full_vectors_indexing
branch and adding the following argument to your index()
call:
index(…
index_type=“FULL_VECTORS”,
)
Any feedback is appreciated, as always, and thanks again!
Hey Benjamin,
Thanks a bunch for your time! Just tried it and it went smoothly.
Installed the feat/full_vectors_indexing branch then tried with and without 'index_type=“FULL_VECTORS”,' both runs where a success!
Hi @gojira. This should FINALLY be fixed by the new experimental default indexing in 0.0.8, which skips using faiss
(does K-means in pure pytorch) as long as you're indexing fewer than ~100k documents!
My code is getting stuck at
Loading segmented_maxsim_cpp extension (set COLBERT_LOAD_TORCH_EXTENSION_VERBOSE=True for more info)...
Even when running the most minimalist example like the one below
from ragatouille import RAGPretrainedModel
if __name__ == "__main__":
RAG = RAGPretrainedModel.from_pretrained("colbert-ir/colbertv2.0")
I'm using Python 3.10.6
This is my env
Package Version
---------------------------------------- -----------
aiohappyeyeballs 2.3.4
aiohttp 3.10.0
aiosignal 1.3.1
annotated-types 0.7.0
anyio 4.4.0
appdirs 1.4.4
asgiref 3.8.1
async-timeout 4.0.3
attrs 23.2.0
backoff 2.2.1
bcrypt 4.2.0
beautifulsoup4 4.12.3
bitarray 2.9.2
blinker 1.8.2
build 1.2.1
cachetools 5.4.0
catalogue 2.0.10
certifi 2024.7.4
cffi 1.16.0
charset-normalizer 3.3.2
chroma-hnswlib 0.7.6
chromadb 0.5.5
click 8.1.7
colbert-ai 0.2.19
coloredlogs 15.0.1
cryptography 43.0.0
dataclasses-json 0.6.7
datasets 2.20.0
db-dtypes 1.2.0
Deprecated 1.2.14
dill 0.3.8
dirtyjson 1.0.8
distro 1.9.0
docstring_parser 0.16
exceptiongroup 1.2.2
faiss-cpu 1.8.0.post1
fast-pytorch-kmeans 0.2.0.1
fastapi 0.112.0
filelock 3.15.4
Flask 3.0.3
flatbuffers 24.3.25
frozenlist 1.4.1
fsspec 2024.5.0
git-python 1.0.3
gitdb 4.0.11
GitPython 3.1.43
google-api-core 2.19.1
google-auth 2.32.0
google-auth-oauthlib 1.2.1
google-cloud-aiplatform 1.60.0
google-cloud-bigquery 3.25.0
google-cloud-core 2.4.1
google-cloud-resource-manager 1.12.5
google-cloud-storage 2.18.0
google-crc32c 1.5.0
google-resumable-media 2.7.1
googleapis-common-protos 1.63.2
greenlet 3.0.3
grpc-google-iam-v1 0.13.1
grpcio 1.65.2
grpcio-status 1.62.2
h11 0.14.0
httpcore 1.0.5
httptools 0.6.1
httpx 0.27.0
huggingface-hub 0.24.5
humanfriendly 10.0
idna 3.7
importlib_metadata 8.0.0
importlib_resources 6.4.0
itsdangerous 2.2.0
Jinja2 3.1.4
joblib 1.4.2
jsonpatch 1.33
jsonpointer 3.0.0
kubernetes 30.1.0
langchain 0.1.20
langchain-community 0.0.38
langchain-core 0.1.52
langchain-experimental 0.0.58
langchain-google-vertexai 1.0.4
langchain-huggingface 0.0.3
langchain-openai 0.1.7
langchain-text-splitters 0.0.2
langsmith 0.1.96
llama-cloud 0.0.11
llama-index 0.10.59
llama-index-agent-openai 0.2.9
llama-index-cli 0.1.13
llama-index-core 0.10.59
llama-index-embeddings-openai 0.1.11
llama-index-indices-managed-llama-cloud 0.2.7
llama-index-legacy 0.9.48
llama-index-llms-openai 0.1.27
llama-index-multi-modal-llms-openai 0.1.8
llama-index-program-openai 0.1.7
llama-index-question-gen-openai 0.1.3
llama-index-readers-file 0.1.32
llama-index-readers-llama-parse 0.1.6
llama-parse 0.4.9
markdown-it-py 3.0.0
MarkupSafe 2.1.5
marshmallow 3.21.3
mdurl 0.1.2
mmh3 4.1.0
monotonic 1.6
mpmath 1.3.0
multidict 6.0.5
multiprocess 0.70.16
munch 4.0.0
mypy-extensions 1.0.0
nest-asyncio 1.6.0
networkx 3.3
ninja 1.11.1.1
nltk 3.8.1
numpy 1.26.4
oauthlib 3.2.2
onnx 1.16.2
onnxruntime 1.18.1
openai 1.38.0
opentelemetry-api 1.26.0
opentelemetry-exporter-otlp-proto-common 1.26.0
opentelemetry-exporter-otlp-proto-grpc 1.26.0
opentelemetry-instrumentation 0.47b0
opentelemetry-instrumentation-asgi 0.47b0
opentelemetry-instrumentation-fastapi 0.47b0
opentelemetry-proto 1.26.0
opentelemetry-sdk 1.26.0
opentelemetry-semantic-conventions 0.47b0
opentelemetry-util-http 0.47b0
orjson 3.10.6
overrides 7.7.0
packaging 23.2
pandas 2.2.2
pandas-gbq 0.23.1
pillow 10.4.0
pip 22.2.1
posthog 3.5.0
proto-plus 1.24.0
protobuf 4.25.4
pyarrow 17.0.0
pyarrow-hotfix 0.6
pyasn1 0.6.0
pyasn1_modules 0.4.0
pycparser 2.22
pydantic 2.8.2
pydantic_core 2.20.1
pydata-google-auth 1.8.2
Pygments 2.18.0
pynvml 11.5.3
pypdf 4.3.1
PyPika 0.48.9
pyproject_hooks 1.1.0
pysbd 0.3.4
python-certifi-win32 1.6.1
python-dateutil 2.9.0.post0
python-dotenv 1.0.1
pytz 2024.1
PyYAML 6.0.1
ragas 0.1.12
RAGatouille 0.0.8.post2
rank-bm25 0.2.2
regex 2024.7.24
requests 2.32.3
requests-oauthlib 2.0.0
rich 13.7.1
rsa 4.9
safetensors 0.4.3
scikit-learn 1.5.1
scipy 1.14.0
sentence-transformers 2.7.0
setuptools 63.2.0
setuptools-scm 8.1.0
shapely 2.0.5
shellingham 1.5.4
simsimd 3.7.7
six 1.16.0
smmap 5.0.1
sniffio 1.3.1
soupsieve 2.5
SQLAlchemy 2.0.31
srsly 2.4.8
starlette 0.37.2
striprtf 0.0.26
sympy 1.13.1
tenacity 8.5.0
threadpoolctl 3.5.0
tiktoken 0.7.0
tokenizers 0.19.1
tomli 2.0.1
torch 2.4.0
tqdm 4.66.4
transformers 4.43.3
typer 0.12.3
typing_extensions 4.12.2
typing-inspect 0.9.0
tzdata 2024.1
ujson 5.10.0
urllib3 2.2.2
uvicorn 0.30.5
uvloop 0.19.0
vertexai 1.60.0
voyager 2.0.9
watchfiles 0.22.0
websocket-client 1.8.0
websockets 12.0
Werkzeug 3.0.3
wrapt 1.16.0
xxhash 3.4.1
yarl 1.9.4
zipp 3.19.2
Hi - really excited to try RAGatouille. On Apple mac with M1Max - it's taken over 12 hours to index. Is this expected?
PyTorch emittd some warnings about CUDA not being available but it's running otherwise without error seemingly.
Below is the output in Jupyter in VS Code - it's running