search

symflower / eval-dev-quality

DevQualityEval: An evaluation benchmark 📈 and framework to compare and evolve the quality of code generation of LLMs.
https://symflower.com/en/company/blog/2024/dev-quality-eval-v0.4.0-is-llama-3-better-than-gpt-4-for-generating-tests/
MIT License
137 stars 5 forks source link

Docker containers may use the same result-path #273

Closed Munsio closed 4 months ago

Munsio commented 4 months ago

According to: https://github.com/symflower/eval-dev-quality/actions/runs/9974654021/job/27562748106?pr=271 2 of the Docker runtimes are writing their data into the same folder symflower_symbolic-execution-0 which leads to problems where they override the files of each other.