Follow-up "Code repairing task to enable models to fix code with compilation errors" - Githubissues

symflower / eval-dev-quality

DevQualityEval: An evaluation benchmark 📈 and framework to compare and evolve the quality of code generation of LLMs.

https://symflower.com/en/company/blog/2024/dev-quality-eval-v0.4.0-is-llama-3-better-than-gpt-4-for-generating-tests/

MIT License

57 stars 3 forks source link

Follow-up "Code repairing task to enable models to fix code with compilation errors" #200

Open ruiAzevedo19 opened 1 week ago

ruiAzevedo19 commented 1 week ago

Follow-ups

[x] Move the log file creation into a helper https://github.com/symflower/eval-dev-quality/pull/170#discussion_r1642908512
[ ] Do the validation for mistakes testdata only once https://github.com/symflower/eval-dev-quality/pull/170#discussion_r1642925996