Why doesn't mbpp use few-shot?

bigcode-project / bigcode-evaluation-harness

A framework for the evaluation of autoregressive code generation language models.

Apache License 2.0

702 stars 180 forks source link

Closed shuaiwang2022 closed 5 months ago

shuaiwang2022 commented 6 months ago

loubnabnl commented 6 months ago

We use the implementation from InCoder which is zero shot