search

bigcode-project / bigcode-evaluation-harness

A framework for the evaluation of autoregressive code generation language models.
Apache License 2.0
710 stars 183 forks source link

fix post-processing of mbpp #124

Closed loubnabnl closed 11 months ago

loubnabnl commented 11 months ago

fixes #121

(The pass@1 for SantaCoder on the first 30 problems of MBPP goes up from 24.1% to 25.3%)