[Question]: Can SQLTableRetrieverQueryEngine or NLSQLTableQueryEngine filter specific value ?

[Ninlawat-Puhu]

Question Validation

• [ ] I have searched both the documentation and discord for an answer.

Question

I use LLM= gpt-3.5-turbo and llama_index to connect database SQL server but I don't see the document to filter value while query in database or pass some values to filter data.

I see llama_index will convert prompt from users to statement SQL but cannot put or pass values before that and the get result from database.

Can you suggest how I can filter data with using NLSQLTableQueryEngine or SQLTableRetrieverQueryEngine ?

Ref: https://docs.llamaindex.ai/en/stable/examples/index_structs/struct_indices/SQLIndexDemo/

[dosubot[bot]]

To filter data using SQLTableRetrieverQueryEngine or NLSQLTableQueryEngine in LlamaIndex with GPT-3.5 Turbo for SQL Server database queries, you'll need to ensure that the components involved in translating natural language queries to SQL can understand and generate the appropriate SQL syntax with WHERE clauses. Here's a step-by-step guide to achieve this:

1. Adjust the Table Parser Component: The function get_table_context_and_rows_str retrieves relevant table rows based on the query string. To apply specific value filters in SQL queries, you might need to modify this function to generate SQL queries with WHERE clauses based on the input query.

2. Customize the Query Pipeline: The query pipeline (qp) includes various components like sql_retriever. For executing SQL queries with specific value filters, ensure that the text2sql_prompt component generates SQL queries that include these filters. This might involve customizing the prompt sent to the text2sql_llm component.

3. Run Queries with Clear Specifications: When running queries using qp.run(query="..."), make sure your input query clearly specifies the filters you want to apply. The system's ability to apply these filters depends on the prompts' quality and the configuration of the LLM components.

To directly use SQLTableRetrieverQueryEngine or NLSQLTableQueryEngine for executing SQL queries with specific value filters, focus on how the text2sql_prompt component generates SQL queries and how you might influence this process. This could involve customizing the prompts or further training the model to understand and generate the desired SQL query syntax.

For practical examples and more details on using the query pipeline, you can refer to the query_pipeline_sql.ipynb example. Additionally, the source code in sql_query.py provides insights into how SQLTableRetrieverQueryEngine and NLSQLTableQueryEngine are implemented and can be used to execute SQL queries with specific value filters.

If you need further assistance or clarification, feel free to ask.

Sources

• docs/docs/examples/pipeline/query_pipeline_sql.ipynb
• llama-index-legacy/llama_index/legacy/indices/struct_store/sql_query.py

#### About Dosu

This response is meant to be useful and save you time. It is **not meant to be a precise solution**, but rather a starting point for your own research. Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant. If you want to continue the conversation, start your reply with @dosu-bot.

[Ninlawat-Puhu]

Can you show example that easily I can modify code or modified code in https://docs.llamaindex.ai/en/stable/examples/index_structs/struct_indices/SQLIndexDemo/ ?

[logan-markewich]

You probably want to create a more custom text-2-sql query pipeline https://docs.llamaindex.ai/en/stable/examples/pipeline/query_pipeline_sql/?h=querypipeline