issues
search
defog-ai
/
sql-eval
Evaluate the accuracy of LLM generated outputs
Apache License 2.0
485
stars
52
forks
source link
small changes to vllm runner
#143
Closed
wongjingping
closed
2 months ago
wongjingping
commented
2 months ago
small changes to vllm runner:
changed tensor parallelism to 1 always (ie disabled) as we generally saw better results
here
remove tqdm from vllm's
.generate
as it was cluttering up the output. also added a simple script for evaluating checkpoints in a directory sequentially.
small changes to vllm runner:
.generateas it was cluttering up the output. also added a simple script for evaluating checkpoints in a directory sequentially.