Database and table scanning to gather context

[imrehg]

Dataherald, a similar tool, has a concept of scanning the database's tables to gather more context (see their docs). This goes beyond the what I've seen in the Vanna docs of getting a schema of the tables, and can be useful as the tool then can work with the kind of values that pop up in the table (e.g. it can discover what range of values or what options are available, such as "all the job titles for employees").

[zainhoda]

Thanks for the suggestion! Until now, we were trying to adhere to the constraint that db contents never go to the LLM because the LLM used to be "low trust."

Now that there are ways to run LLMs in a "high trust" manner (locally or for enterprises via Azure OpenAI etc), we plan to remove that constraint. With that constraint lifted, it should unlock all sorts of possibilities like the one you're talking about.

[bmerkle]

I would be interested in the progress/plans on this as we have a larger set of database tables, also with enumeration values etc.