Made it take in an additional -m parameter that will help us tell the script which model's tokenizer to load
VLLM
Enable batching. When set to -bs 200 it reverts to original vllm generation behaviour/scores (see [scores](https://docs.google.com/spreadsheets/d/1rkrhQmrpTWMNorrVQxM_KnHslKQiJDSs_kzxt6fmKbI/edit#gid=736994119)).
HF
Token(izer) updates
Also fixed a small edge case for the gcs_eval.py script where it was reading/processing empty lines
wendy-aw
commented
3 months ago
Updates to runners: API
-mparameter that will help us tell the script which model's tokenizer to load VLLM-bs 200it reverts to original vllm generation behaviour/scores (see[scores](https://docs.google.com/spreadsheets/d/1rkrhQmrpTWMNorrVQxM_KnHslKQiJDSs_kzxt6fmKbI/edit#gid=736994119)). HFAlso fixed a small edge case for the
gcs_eval.pyscript where it was reading/processing empty lines