pingcap / tidb.ai

https://TiDB.AI is a Graph RAG based and conversational knowledge base tool built with TiDB Serverless Vector Storage and LlamaIndex. Open source and free to use.
https://tidb.ai
Apache License 2.0
193 stars 25 forks source link

Improve query analysis #315

Closed IANTHEREAL closed 1 month ago

IANTHEREAL commented 1 month ago

The performance of query decomposition between main branch and this pr

dataset: 8 samples

Main branch

without fewshot examples
Average Metric: 4.5 / 8  (56.2): 100%|██████████| 8/8 [00:06<00:00,  1.17it/s] 
56.25

with fewshot examples
Average Metric: 7.275 / 8  (90.9): 100%|██████████| 8/8 [00:06<00:00,  1.31it/s]
90.94

This PR

without fewshot examples
Average Metric: 6.3 / 8  (78.8): 100%|██████████| 8/8 [00:04<00:00,  1.70it/s] 
78.75

with fewshot examples
Average Metric: 7.000000000000001 / 8  (87.5): 100%|██████████| 8/8 [00:05<00:00,  1.59it/s] 
87.5
vercel[bot] commented 1 month ago

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
tidb-ai-preview ✅ Ready (Inspect) Visit Preview 💬 Add feedback Sep 29, 2024 1:47pm
tidb-ai-storybook ✅ Ready (Inspect) Visit Preview 💬 Add feedback Sep 29, 2024 1:47pm
github-actions[bot] commented 1 month ago

E2E Result Deployment

https://tidb-ai-playwright-exl0blct6-djaggers-projects.vercel.app