Added TraceHandler to capture evaluation trace, which includes the evaluator steps as well as any data the target might provide along with a response (e.g. trace data from Amazon Bedrock). Traces are dumped to agenteval_traces/
Added TargetResponse to capture the agent's response and additional data.
Added ConversationHandler to capture user-agent interaction.
Rename TaskResult to TestResult and updated attributes for more clarity.
Updated documentation.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
Issue #, if available:
Description of changes:
TraceHandler
to capture evaluation trace, which includes the evaluator steps as well as any data the target might provide along with a response (e.g. trace data from Amazon Bedrock). Traces are dumped toagenteval_traces/
TargetResponse
to capture the agent's response and additional data.ConversationHandler
to capture user-agent interaction.TaskResult
toTestResult
and updated attributes for more clarity.By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.