Tool/command to combine multiple evaluations into one

symflower / eval-dev-quality

DevQualityEval: An evaluation benchmark 📈 and framework to compare and evolve the quality of code generation of LLMs.

MIT License

57 stars 3 forks source link

Open bauersimon opened 1 week ago

bauersimon commented 1 week ago

If we run them on multiple machines, we want to easily combine them.

bauersimon commented 1 week ago

Current workarounds:

cat docs/reports/<version>/*/models-summed.csv | sort | uniq for overall scores
cat docs/reports/<version>/*/evaluation.csv | sort | uniq for repository-specific scores