Question: can be used as a search server?

AnswerDotAI / RAGatouille

Easily use and train state of the art late-interaction retrieval methods (ColBERT) in any RAG pipeline. Designed for modularity and ease-of-use, backed by research.

Apache License 2.0

3.03k stars 206 forks source link

Question: can be used as a search server? #199

Open agori80 opened 6 months ago

agori80 commented 6 months ago

Given the superb peformance of this Colbert implementation, I am considering integrating this into a search pipeline. There would be a server running and accepting queries from a few users (probably not a very high load, typical enterprise setting). Queries would be executed concurrently.

Does it make sense to use this in such situation or is it just meant for offline usage?

dineshdharme commented 6 months ago

The official ColBERT implementation has a built-in query server (using Flask), which you can easily query via API requests and does support indexes generated with RAGatouille! This should be enough for most small applications, so long as you can persist the index on disk.

https://github.com/stanford-futuredata/ColBERT