vanna-ai / vanna

🤖 Chat with your SQL database 📊. Accurate Text-to-SQL Generation via LLMs using RAG 🔄.
https://vanna.ai/docs/
MIT License
9.12k stars 673 forks source link

generated SQL should be validated before execution #494

Open wgong opened 3 weeks ago

wgong commented 3 weeks ago

Is your feature request related to a problem? Please describe. I asked the following question on SQLite chinook db:

vn.ask(question="which table stores customer's orders")

The answer is correct, yet Vanna tried to execute an invalid SQL

The "invoices" table stores customers' orders. It contains information about each invoice made by a customer, including the customer ID, invoice date, billing address, total amount, and foreign key references to the customer and invoice items

Here is the error log

Couldn't run sql:  Execution failed on sql 'The "invoices" table stores customers' orders. It contains information about each invoice made by a customer, including the customer ID, invoice date, billing address, total amount, and foreign key references to the customer and invoice items.': near "The": syntax error

Describe the solution you'd like use sqlparse.parse(sql)

Describe alternatives you've considered A clear and concise description of any alternative solutions or features you've considered. not sure

Additional context Add any other context or screenshots about the feature request here. N/A

zainhoda commented 2 weeks ago

Yeah, we already use an is_sql_valid in the web app between vn.generate_sql and vn.run_sql, which we should bring over to vn.ask