feat(wren-ai-service): Implement the process for the evaluation framework

Canner / WrenAI

🚀 An open-source SQL AI (Text-to-SQL) Agent that empowers data, product teams to chat with their data. 🤘

https://getwren.ai/oss

GNU Affero General Public License v3.0

2.04k stars 211 forks source link

feat(wren-ai-service): Implement the process for the evaluation framework #525

Closed paopa closed 4 months ago

paopa commented 4 months ago

This PR aims to implement the main processes for the evaluation framework. Instructions on how to use it are provided in the README file in the eval directory. Currently, we don't fully support all metrics (e.g., accuracy, contextual recall, etc.), but we plan to release these features soon. Additionally, we provide an example evaluation dataset and metric to give members a preview.

Canner / WrenAI

feat(wren-ai-service): Implement the process for the evaluation framework #525

Screen