RAGAtouille cpu only installation

AnswerDotAI / RAGatouille

Easily use and train state of the art late-interaction retrieval methods (ColBERT) in any RAG pipeline. Designed for modularity and ease-of-use, backed by research.

Apache License 2.0

2.82k stars 196 forks source link

RAGAtouille cpu only installation #233

Open tacacs1101-debug opened 2 months ago

tacacs1101-debug commented 2 months ago

The current ragatouille installation quite heavy right now. For those who only wants to use it on cpu have to unnecessarily install nvidia-cuda based modules. For a production application, where I have to only load and infer from index on cpu doesn't need to install cuda based modules. Is there a way currently to install only cpu based libraries in ragatouille.

abdelkareemkobo commented 1 month ago

for the current version, it really depends on the GPU (cuda) for the training and searching building a cpu version for the current code is not that easy, and not wanted for now. You may after train your version of Colbert use with fastembed to get best of them.