search

defog-ai / sqlcoder

SoTA LLM for converting natural language questions to SQL queries
Apache License 2.0
3.27k stars 205 forks source link

Special Instruction while inferencing locally deployed model #61

Closed prayagkb closed 7 months ago

prayagkb commented 7 months ago

Deployed SQLCoder-34B Locally, But couldn't provide special instructions as shown in Demo. Where & How can I write special instructions ?

Kirushikesh commented 7 months ago

If i am not wrong the special instructions appears in the Instructions section, This notebook in the repo contains the inference of the model with the schema and question, check the prompt and Adhere to these rules: is what ig to be the special instructions section.

rishsriv commented 7 months ago

Hi there! You can add instructions anywhere in the prompt you construct – either in the instructions section as @Kirushikesh mentioned, or elsewhere in the prompt. The exact location will vary depending on your specific use-case.

Kirushikesh commented 7 months ago

Also @rishsriv, thanks for the reply. What is the actual prompt format used for finetuning the codellama model, mostly i found the Task/Database Schema/Answer, but in the notebook i found a new format of Instructions/Input/Response.

rishsriv commented 7 months ago

It's a mixture of formats – you should pretty good results with the file in prompt.md