search

defog-ai / sql-eval

Evaluate the accuracy of LLM generated outputs
Apache License 2.0
485 stars 52 forks source link

fix temperature/top_p warning #117

Closed wongjingping closed 2 months ago

wongjingping commented 2 months ago

Fix temperature/top_p warnings by setting it explicitly when generating Add batch_size param to gcs_eval.py

Tested with the following and the warnings don't show up anymore:

python3 main.py \
  -db postgres \
  -q data/instruct_basic_postgres.csv data/instruct_advanced_postgres.csv data/questions_gen_postgres.csv \
  -o "results/${model_name}_beam1_basic_hf.csv" "results/${model_name}_beam1_advanced_hf.csv" "results/${model_name}_beam1_v1_hf.csv" \
  -g hf \
  -b 1 \
  -c 0 \
  -f "prompts/prompt.md" \
  -m "${model_name}"