bigcode-project / bigcode-evaluation-harness

A framework for the evaluation of autoregressive code generation language models.
Apache License 2.0
744 stars 193 forks source link

[WIS] Program repair #59

Closed keyboardAnt closed 1 year ago

keyboardAnt commented 1 year ago

Add an Automatic Program Repair (APR) Task evaluation over a dataset of ~2500 repairs in Python, inspired by https://arxiv.org/abs/2105.12787 and https://carper.ai/diff-models-a-new-way-to-edit-code/.