issues
search
Canner
/
WrenAI
🚀 Open-source SQL AI Agent for Text-to-SQL. Make Text2SQL Easy! 🙌
https://getwren.ai/oss
GNU Affero General Public License v3.0
1.73k
stars
155
forks
source link
feat(wren-ai-service): evaluation dataset curation app
#398
Closed
cyyeh
closed
2 months ago
cyyeh
commented
3 months ago
add eval dataset curation app v1
users can create new eval dataset
Upload an MDL json file first
Get question-sql-pairs given by LLM or you manually enter question and corresponding sql
Do validation on each group of question, context and SQL, and move it to the candidate dataset if you think it's valid
Save the candidate dataset by clicking the "Save as Evaluation Dataset" button.
users can modify existed eval dataset by uploading the dataset
Upload an MDL json file first
Upload the evaluation dataset(.toml file) you want to modify, and please make sure the model in the dataset is the same as the deployed model
Modify the evaluation dataset the same as you create a new one
Save the candidate dataset by clicking the "Save as Evaluation Dataset" button.
users can choose which OpenAI llm model to use
users can type custom instructions for llm to generate question-sql-pairs
users can preview data if sql is valid
update setup instruction on readme
go to the
eval
folder and run
poetry install
to install the dependencies
execute commands in the
wren-ai-service
folder
to run the evaluation dataset curation app:
make data_curation_app
eval
folder and runpoetry install
to install the dependencieswren-ai-service
foldermake data_curation_app