HumanEval-X-Bugs - Githubissues

bigcode-project / octopack

🐙 OctoPack: Instruction Tuning Code Large Language Models

https://arxiv.org/abs/2308.07124

MIT License

420 stars 27 forks source link

Closed Muennighoff closed 1 year ago

Muennighoff commented 1 year ago

HumanEval-X-Bugs

Bug Types:

Missing Logic: 33
Incorrect Logic: 100
- Value Misuse: 45
- Operator Misuse: 25
- Variable Misuse: 23
- Function Misuse: 7
Excess Logic: 31

By looking through diffs in the Files changed tab one can directly see the bugs.