bigcode-project / bigcode-evaluation-harness

A framework for the evaluation of autoregressive code generation language models.
Apache License 2.0
830 stars 219 forks source link

run the MBPP in the HumanEval data format #218

Closed virt9 closed 6 months ago

virt9 commented 7 months ago

Hello. thk you for your prior help! i want to run the MBPP dataset in the HumanEval data format. But I found that when i do this, the generation repeat until to the max_length ,but in HumanEval dataset it doesnt happen. so I wander if there is something I shoud change . perhaps the postprocess?