defog-ai / sql-eval

Evaluate the accuracy of LLM generated outputs
Apache License 2.0
448 stars 47 forks source link

Script #176

Closed wongjingping closed 2 weeks ago

wongjingping commented 2 weeks ago

Added a script for testing a single model with cot + api_runner. This makes it a little easier to test individual models without having to manually fire 2 different python commands and then kill the api server manually.

After updating the script with the directory that your model is in, you can test a model by running ./run_model_cot.sh <my_model>

wendy-aw commented 2 weeks ago

Thank you!