search

empirical-run / empirical

Test and evaluate LLMs and model configurations, across all the scenarios that matter for your application
https://docs.empirical.run
MIT License
148 stars 13 forks source link

fix: cli summary shows all run-specific scores #205

Closed mukeshblackhat closed 5 months ago

mukeshblackhat commented 5 months ago

closes https://github.com/empirical-run/empirical/issues/167

@saikatmitra91 two things are done first

  1. all the scorer names are taken out , instead of assuming that first run will have all the jobs as done previously in your code . const scorerNames = [ ...new Set( runs.flatMap( (run) => run?.stats?.scores?.map((score) => score.name) || [], ), ), ];

  2. then if their is no score then is should show "-" instead of "0% " if (scoreStats) { const metric = percentStr(scoreStats.average * 100); return enableColors ? setMetricColor(metric, scoreStats.average) : metric; } else { return "-"; }

changeset-bot[bot] commented 5 months ago

🦋 Changeset detected

Latest commit: 49a4db07d4febf5c24fc9c843e9757683f06cf11

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package | Name | Type | | ----------------- | ----- | | @empiricalrun/cli | Patch |

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

mukeshblackhat commented 5 months ago

@arjunattam comment is removed