[REQUEST] support model-parallel evaluation for big models

bigcode-project / bigcode-evaluation-harness

A framework for the evaluation of autoregressive code generation language models.

Apache License 2.0

825 stars 219 forks source link

Open tarahjjeon opened 1 month ago

tarahjjeon commented 1 month ago

Please support big model evaluation such as Qwen 2.5 32B on A100 40G gpus