What if the retrieved ddl is not related to the user question?

vanna-ai / vanna

🤖 Chat with your SQL database 📊. Accurate Text-to-SQL Generation via LLMs using RAG 🔄.

https://vanna.ai/docs/

MIT License

9.4k stars 691 forks source link

What if the retrieved ddl is not related to the user question? #445

Closed Jackie-shi closed 1 month ago

Jackie-shi commented 1 month ago

When do 'ask', the process here is to use vector database to get the most related DDL. But there is a situation that all of the ddl has nothing to do with the user question, which means that the final SQL Vana generates will definitely WRONG!

So I wonder if there is any mechanism or post-process to check or evaluate the retrieved ddl.🤔

THX!

Jackie-shi commented 1 month ago

What's more, when you do embedding, the raw ddl is not a natural language so the problem between ddl and user question is that they cannot be aligned in semantic space. It may make the result inaccurate!