awslabs / agent-evaluation

A generative AI-powered framework for testing virtual agents.
https://awslabs.github.io/agent-evaluation/
Apache License 2.0
64 stars 10 forks source link

Pass Rate Metric #60

Closed dferguson992 closed 1 month ago

dferguson992 commented 1 month ago

Issue #, if available: 32

Description of changes: Created the metrics dictionary which is passed to the summary template. calculate_pass_rate_metric populates the pass_rate metric which is currently the only metric published in the output summary.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.