Closed ganler closed 9 months ago
Similar to #187, we have converted MBPP+ into a form that is similar to the original MBPP format and is made compatible with bigcode-evaluation-harness.
bigcode-evaluation-harness
MBPP+ dataset is also available on the hub: https://huggingface.co/datasets/evalplus/mbppplus
Tested on deepseekcoder-1.3b-base and can repro the scores in the EvalPlus leaderboard:
cc: @loubnabnl
Similar to #187, we have converted MBPP+ into a form that is similar to the original MBPP format and is made compatible with
bigcode-evaluation-harness.MBPP+ dataset is also available on the hub: https://huggingface.co/datasets/evalplus/mbppplus
Tested on deepseekcoder-1.3b-base and can repro the scores in the EvalPlus leaderboard:
cc: @loubnabnl