awslabs / agent-evaluation

A generative AI-powered framework for testing virtual agents.
https://awslabs.github.io/agent-evaluation/
Apache License 2.0
117 stars 20 forks source link

Pass Rate Metric #60

Closed dferguson992 closed 6 months ago

dferguson992 commented 6 months ago

Issue #, if available: 32

Description of changes: Created the metrics dictionary which is passed to the summary template. calculate_pass_rate_metric populates the pass_rate metric which is currently the only metric published in the output summary.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.