Incomplete humaneval evaluation code

bigcode-project / Megatron-LM

Ongoing research training transformer models at scale

Other

371 stars 48 forks source link

Incomplete humaneval evaluation code #57

Open huybery opened 1 year ago

huybery commented 1 year ago

How to reproduce the humaneval performance in this repo ? I tried to evaluate using the evaluation branch, but it seems to be very different from the the default branch, is it possible to add a full evaluation process in default branch ? Thanks a lot.