search

bigcode-project / bigcode-evaluation-harness

A framework for the evaluation of autoregressive code generation language models.
Apache License 2.0
831 stars 219 forks source link

Human_eval .yaml #174

Closed inigopm closed 10 months ago

inigopm commented 11 months ago

Is there any implementation of human_eval dataset in the new version of eval_harness? (The one of EleutherAI)

In that new version, the tasks are configured via .yaml. This file is the one i need.

Thanks.

loubnabnl commented 11 months ago

I don't think HumanEval is implemented in lm-evaluation-harness