search

zylon-ai / private-gpt

Interact with your documents using the power of GPT, 100% privately, no data leaks
https://privategpt.dev
Apache License 2.0
54.09k stars 7.27k forks source link

Evaluation by RAGAs #1909

Open psmukhopadhyay opened 6 months ago

psmukhopadhyay commented 6 months ago

Hello

Is it possible to link PrivateGPT with RAGAs metrics (https://docs.ragas.io/en/stable/concepts/metrics/index.html#ragas-metrics) for evaluation? Can you please guide me how to achieve this?

Best

jaluma commented 4 months ago

Glad to heard about evaluation. You can start looking at llamaindex integration: https://docs.ragas.io/en/stable/howtos/integrations/llamaindex.html

If you check the doc, you will see that you can already use llamaindex on datasets to evaluate them. For LLM, you can inject LLMComponent and get the PGPT model. For the testing part of the retriever, you will have to externalise or give access to the retriever along with all the node_postprocessor.