bclavie / RAGatouille

Easily use and train state of the art late-interaction retrieval methods (ColBERT) in any RAG pipeline. Designed for modularity and ease-of-use, backed by research.
Apache License 2.0
2.45k stars 173 forks source link

Question: How to get score of ranked document? #201

Open BernhardSchlegel opened 2 months ago

BernhardSchlegel commented 2 months ago

Hey all, I'm using RAGatouille as a retriever for langchain like so

RAG = RAGPretrainedModel.from_index(".ragatouille/colbert/indexes/clr-docs")
retriever = RAG.as_langchain_retriever(k=10)
retrieval = RunnableParallel({
    "context": retriever,
    "question": RunnablePassthrough(),
})
rag_chain = (
    retrieval
    | RunnableParallel({
        "answer": ANSWER_PROMPT | llm,
        "docs": itemgetter("context")
    })
)

I've traced down as_langchain_retriever but I could not find any helpful arguments.

How do I get the score for each retrieved doc so I can, i.e., calculate the auPRC score?

Thanks, Bernhard.

BernhardSchlegel commented 2 months ago

Answer: Using the ContextualCompressionRetriever