search

Canner / WrenAI

🚀 Open-source SQL AI Agent for Text-to-SQL. Supporting PostgreSQL, DuckDB, MySQL, MS SQL, ClickHouse, Trino, JSON, CSV, Parquet data sources, and more! 🚀
https://getwren.ai/oss
GNU Affero General Public License v3.0
1.9k stars 194 forks source link

Experimenting DSPy #692

Open cyyeh opened 2 weeks ago

cyyeh commented 2 weeks ago

We wonder how DSPy could help us auto-optimize prompt for us, I've created a branch before: https://github.com/Canner/WrenAI/tree/chore/ai-service/dspy, so please do some experiments on DSPy based on the branch.

DSPy related code is based on wren-ai-service/eval/prompt_optimizer.py, wren-ai-service/eval/dspy_modules and wren-ai-service/eval/optimized.

Please read wren-ai-service/eval/prompt_optimizer.py first so you could better understand the overall workflow

You may use public text-to-sql dataset(for example, spider dataset) to do some experiments, and the final result would be optimized json. We hope we can then retrieve optimized prompt from the json and ingest them into the prompts.

Feel free to discuss with me if you have any other questions, welcome to join our Discord: https://discord.gg/5DvshJqG8Z

tedyyan commented 1 week ago

After talking with Willian, he told me to try this issue first. Thanks, William and Jimmy

cyyeh commented 1 week ago

@tedyyan sure, welcome for contribution! I've assigned you to the issue and let's discuss the issue first in Discord :)