search

defog-ai / sqlcoder

SoTA LLM for converting natural language questions to SQL queries
Apache License 2.0
3.42k stars 218 forks source link

There are very long DDL in real situation may be 10000+ tokens, is it possible to support long input DDL? #9

Closed ct-source closed 10 months ago

ct-source commented 1 year ago

Thank you for your awlsome work!

lihaoyang-ruc commented 1 year ago

Hi, the prevalence of lengthy DDL inputs is a common occurrence in real-world applications. To address this issue, an effective approach involves implementing a preprocessing module that extracts the most relevant tables and columns based on the natural language questions. This technique could significantly reduce the overall length of the input sequence, leading to improved efficiency.

For further insight, you can explore our recent work, CodeS. In this work, we have successfully integrated a schema item filter, which plays a pivotal role in attaining the aforementioned objective. 😉

nuaabuaa07 commented 1 year ago

Hi, the prevalence of lengthy DDL inputs is a common occurrence in real-world applications. To address this issue, an effective approach involves implementing a preprocessing module that extracts the most relevant tables and columns based on the natural language questions. This technique could significantly reduce the overall length of the input sequence, leading to improved efficiency.

For further insight, you can explore our recent work, CodeS. In this work, we have successfully integrated a schema item filter, which plays a pivotal role in attaining the aforementioned objective. 😉

the Link you offer is closed!

nuaabuaa07 commented 1 year ago

Hi, the prevalence of lengthy DDL inputs is a common occurrence in real-world applications. To address this issue, an effective approach involves implementing a preprocessing module that extracts the most relevant tables and columns based on the natural language questions. This technique could significantly reduce the overall length of the input sequence, leading to improved efficiency. For further insight, you can explore our recent work, CodeS. In this work, we have successfully integrated a schema item filter, which plays a pivotal role in attaining the aforementioned objective. 😉

the Link you offer is closed!

I found code here. https://github.com/defog-ai/sql-eval/blob/main/utils/pruning.py

github-actions[bot] commented 10 months ago

This issue is stale because it has been open for 30 days with no activity.

github-actions[bot] commented 10 months ago

This issue was closed because it has been inactive for 14 days since being marked as stale.