empirical-run / empirical

Test and evaluate LLMs and model configurations, across all the scenarios that matter for your application
https://docs.empirical.run
MIT License
141 stars 10 forks source link

feat: py-script scorer has a simpler return type #193

Closed arjunattam closed 2 months ago

changeset-bot[bot] commented 2 months ago

🦋 Changeset detected

Latest commit: e699d8a5e7af9d570f8d1774473a96c541f6c959

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 3 packages | Name | Type | | -------------------- | ----- | | @empiricalrun/scorer | Patch | | @empiricalrun/core | Patch | | @empiricalrun/cli | Patch |

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

github-actions[bot] commented 2 months ago

Empirical Run Summary

Run #1bb5: gpt-3.5-turbo Run #09e8: gpt-4-turbo-preview
Outputs 100% 100%
Scores
json-syntax 50% 100%
Avg latency 834ms 1547ms

Total dataset samples: 2