search

bigcode-project / bigcode-evaluation-harness

A framework for the evaluation of autoregressive code generation language models.
Apache License 2.0
771 stars 201 forks source link

[Minor] Missing task template #27

Closed JeanKaddour closed 1 year ago

JeanKaddour commented 1 year ago

The guide on adding a new task recommends to

From the bigcode-evaluation-harness project root, copy over the new_task.py template to lm_eval/tasks

but there is no new_task.py template in the project root.

I suspect this is a relic from the lm-evaluation-harness repo?

loubnabnl commented 1 year ago

Indeed thanks for flagging it! I forgot to push that file