r78v10a07
commented
3 weeks ago I read the notebook you posted here and we see the same problem. In cell 23 you ask about the list of unique cities. The pandas unique function return much more cities than the ones returned by the LLM in the final answer.
Hi @Valentina-Alto, I read your blog "Building your own DB Copilot for Azure SQL with Azure OpenAI GPT-4" and it is really helpful. We followed your code to inquire our Azure SQL database with Langchain and OpenAI. In general it works fine, however we have found issues when the expected answer is truncated by the LLM. We ask a question that is translated correctly to SQL. Then, the SQL sentence return 100 rows but the LLM answer include only information from the first 15. For instance, we have a table Person with 100 children (age < 18). When asked the LLM to return the name of all children it return the first 10 and add a sentence similar to "... and many more." However, when we ask the same question but adding something like: "Give me a list of all children. I know that there are 100, return all", the LLM return the full list. We have modified the prompt instructions in many ways but always the LLM return a truncated answer. Can you give us and advice or point us to someone that could help us to figure out how to make the LLM to return a full answer without truncation? Is this a problem related to how the Langchain agent preset teh data to the LLM or is a problem of the LLM? Thanks, Roberto