Simplify openai prompt - Githubissues

defog-ai / sql-eval

Evaluate the accuracy of LLM generated outputs

Apache License 2.0

540 stars 56 forks source link

Closed rishsriv closed 3 months ago

rishsriv commented 3 months ago

Changed the OpenAI prompt to a JSON so it's more familiar to users in the "system", "user", "assistant" format
Removed support for text-davinci-003, as it's barely used any more and was causing code bloat
Removed "prefilled" assistant prompt from the default prompt. OpenAI model seem to perform much better after this change

wendy-aw commented 3 months ago

Looks good! Thanks for updating it. Would you also like to update the README with the removal of text-davinci-003 and using the new json prompt file?

rishsriv commented 3 months ago

Thanks! Good point - fixing in a bit and pushing