Why the result of using multiple evaluation is 0

bigcode-project / bigcode-evaluation-harness

A framework for the evaluation of autoregressive code generation language models.

Apache License 2.0

825 stars 219 forks source link

Closed shuaiwang2022 closed 5 months ago

shuaiwang2022 commented 10 months ago

my settings

loubnabnl commented 9 months ago

Are you running the execution inside the docker image we provide for MultiPL-E? as it requires extra dependencies