bigcode-project / bigcode-evaluation-harness

A framework for the evaluation of autoregressive code generation language models.
Apache License 2.0
698 stars 180 forks source link

run the MBPP in the HumanEval data format #218

Closed virt9 closed 1 month ago

virt9 commented 2 months ago

Hello. thk you for your prior help! i want to run the MBPP dataset in the HumanEval data format. But I found that when i do this, the generation repeat until to the max_length ,but in HumanEval dataset it doesnt happen. so I wander if there is something I shoud change . perhaps the postprocess?