search

AnswerDotAI / RAGatouille

Easily use and train state of the art late-interaction retrieval methods (ColBERT) in any RAG pipeline. Designed for modularity and ease-of-use, backed by research.
Apache License 2.0
3.08k stars 210 forks source link

Question -- Symmetric search #230

Closed dmacko232 closed 3 months ago

dmacko232 commented 4 months ago

Hello I am not able to find this information anywhere. But is it possible to use ColBERT model also for symmetric search? Lets say we have passages of ~300tokens and I would wanna find most similar ones (something like recommender/symmetric search).

bclavie commented 3 months ago

Hey, for this you'd want to just up the query maximum length to something pretty long, but it is doable and works pretty well -- in fact, it's done for one of the benchmarks in the original ColBERTv2 paper!