search

pingcap / autoflow

pingcap/autoflow is a Graph RAG based and conversational knowledge base tool built with TiDB Serverless Vector Storage. Demo: https://tidb.ai
https://tidb.ai
Apache License 2.0
1.68k stars 86 forks source link

feat: evaluation via ragas and asktug dataset #388

Closed Icemap closed 1 week ago

Icemap commented 1 week ago
  1. Added an eval_dataset builder, which can download and save the data to a CSV file.
  2. Added a feature that can evaluate the answers from autoflow by comparing RAG's responses and AskTUG's references.
  3. Remained a TODO, which is we can not get retrieved_contexts now, due to the external engine, so we cannot generate the LLMContextRecall and Faithfulness metrics.

The evaluation result: results_2024-11-19-23-b16243.csv

vercel[bot] commented 1 week ago

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
tidb-ai-preview ✅ Ready (Inspect) Visit Preview 💬 Add feedback Nov 20, 2024 1:09pm
tidb-ai-storybook ✅ Ready (Inspect) Visit Preview 💬 Add feedback Nov 20, 2024 1:09pm