Closed jamesbraza closed 11 months ago
🤖
Hello,
Thank you for reporting this issue. It seems like the OllamaEmbeddings(model="llama2:13b")
model is generating an invalid SQL query from the natural language input, which is causing the psycopg2.errors.SyntaxError
.
The NLSQLTableQueryEngine
uses the llm_predictor
to convert a natural language query into an SQL query. If the generated SQL query is not valid, the run_sql
method will raise a psycopg2.errors.SyntaxError
. This is likely what's happening in your case.
A similar issue was reported in the past (#7278) and was resolved by improving the SQL query parsing (#7283). However, it seems like the issue persists when using the OllamaEmbeddings(model="llama2:13b")
model.
Unfortunately, without more information about the differences between the default LlamaIndex embeddings model and the OllamaEmbeddings(model="llama2:13b")
model, it's hard to say why this is happening. It's possible that the OllamaEmbeddings(model="llama2:13b")
model is not suitable for this task and a different model or approach may be needed.
As a workaround, you could try using a different embeddings model or adjust the way the natural language query is processed before it is passed to the model.
I will open a new issue to track this problem and will update you as soon as we have a fix or workaround. In the meantime, if you have any additional information that could help us understand this issue better, please feel free to share.
Best, Dosu
This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.
Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.
If you want to continue the conversation, start your reply with @dosu-bot.
Update: stepping through the code, the issue is BaseSQLTableQueryEngine._parse_response_to_sql
is not properly extracting the SQL. Seems @dosu-bot is agreeing with me. Here is the response
that it fails to parse:
Sure, I'd be happy to help! Here's the SQL query to retrieve the aliases for
<redacted>
Here's the explanation of the query:
🤖
Hello,
Thank you for providing detailed information about the issue you're experiencing. Based on the traceback you've shared, it seems like the BaseSQLTableQueryEngine._parse_response_to_sql
method is indeed failing to correctly extract the SQL from the model's output.
This issue appears to be similar to a previously solved issue in the LlamaIndex repository: [Bug]: text-to-SQL example fails with the new LlamaCPP fallback. This issue was resolved by a pull request: Improve SQL query parsing on August 16, 2023.
The pull request made changes to the BaseSQLTableQueryEngine._parse_response_to_sql
method in LlamaIndex version 0.8.35. The updated method now handles an additional case where both "SQLResult:" and "SQLQuery:" are present in the response. In the previous version (0.8.34), the method only handled the case where "SQLResult:" was present in the response.
Here is the updated method:
def _parse_response_to_sql(self, response: str) -> str:
"""Parse response to SQL."""
# Find and remove SQLResult part
sql_result_start = response.find("SQLResult:")
sql_query_start = response.find("SQLQuery:")
if sql_result_start != -1 and sql_query_start != -1:
response = response[sql_query_start + 1 : sql_result_start].lstrip(
"SQLQuery:"
)
elif sql_result_start != -1:
response = response[:sql_result_start]
result_response = response.strip()
return result_response
Given this, I recommend updating your LlamaIndex to the latest version (v0.8.35) which includes this fix. This should resolve the issue you're experiencing.
Please let me know if this resolves your issue or if you need further assistance.
This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.
Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.
If you want to continue the conversation, start your reply with @dosu-bot.
One thought is it could be useful to have SQL validation before actually execute
ing the SQL query
Bug Description
When using the default LlamaIndex embeddings model, my
NLSQLTableQueryEngine
query works.Next, when trying to use
OllamaEmbeddings(model="llama2:13b")
, it's not currently working with LlamaIndex'sNLSQLTableQueryEngine
. I get apsycopg2.errors.SyntaxError: syntax error at or near "Sure"
.I believe the issue is the embeddings model outputting something that is not pure SQL, but basically a full-blown response.
I am wondering if https://github.com/jerryjliu/llama_index/blob/v0.8.34/llama_index/prompts/default_prompts.py#L188 is not currently adapted to work with
llama2:13b
? Perhaps we need to expand this prompt?Version
0.8.34
Steps to Reproduce
Also with
SQLAlchemy==2.0.21
andpsycopg2==2.9.7
:Relevant Logs/Tracbacks