Open danmcp opened 1 month ago
Steps to recreate:
Launch mmlu on an instance with multiple gpus. Run:
ilab model evaluate --model models/instructlab/granite-7b-lab --benchmark mmlu
Only 1 gpu is consumed. Adjusting batch-size doesn't seem to have any effect.
This might be best fixed with https://github.com/instructlab/eval/issues/50
Steps to recreate:
Launch mmlu on an instance with multiple gpus. Run:
ilab model evaluate --model models/instructlab/granite-7b-lab --benchmark mmlu
Only 1 gpu is consumed. Adjusting batch-size doesn't seem to have any effect.