search

explodinggradients / ragas

Evaluation framework for your Retrieval Augmented Generation (RAG) pipelines
https://docs.ragas.io
Apache License 2.0
6.56k stars 643 forks source link

Get relevant docs in Context Relevancy #206

Closed sid8491 closed 3 months ago

sid8491 commented 10 months ago

According to my understanding, Ragas makes LLM calls to get relevant sentences from the chunks that are useful for generating the answer to calculate the context relevancy score. Is there any way we can get these relevant sentences in return along with the score? We want to save these for manual analysis.

shahules786 commented 10 months ago

Hi @sid8491, thanks for raising this issue. Are you using any tool for logging/tracing?

We have integration with Langsmith that allows you to see logs/data points from each metric. Check it out here We will be adding logging support to other frameworks like wand soon.

sid8491 commented 10 months ago

Hi @shahules786, i am using our own loggers to save data and analyze it in ELK stack. the ragas metrics scores and relevant chunks, those i want to save in my DB as per requirement. Is there any way to get relevant chunks apart from the score?

jjmachan commented 9 months ago

is this fixed @sid8491 ? I didn't understand the last part, could you elaborate

Is there any way to get relevant chunks apart from the score?