search

symflower / eval-dev-quality

DevQualityEval: An evaluation benchmark 📈 and framework to compare and evolve the quality of code generation of LLMs.
https://symflower.com/en/company/blog/2024/dev-quality-eval-v0.4.0-is-llama-3-better-than-gpt-4-for-generating-tests/
MIT License
58 stars 3 forks source link

Add linters where each error is a metric #81

Open zimmski opened 2 months ago

zimmski commented 2 months ago

E.g. assertNotNull(Plain.class) is always true, and should not be in the code. A real programmer would not test that.

bauersimon commented 2 months ago

This is very similar to #44