issues
search
awslabs
/
agent-evaluation
A generative AI-powered framework for testing virtual agents.
https://awslabs.github.io/agent-evaluation/
Apache License 2.0
118
stars
20
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
Major project refactor
#44
tonykchen
closed
7 months ago
0
Set a safe default max num of threads
#43
sharonxiaohanli
closed
7 months ago
0
Revamp user documentation
#42
tonykchen
closed
7 months ago
0
Add a demo UI
#41
sharonxiaohanli
closed
7 months ago
0
Add release workflows
#40
tonykchen
closed
7 months ago
0
Add release workflows
#39
tonykchen
closed
7 months ago
0
add bedrock knowledgebase as target
#38
sharonxiaohanli
closed
7 months ago
0
Implement release pipelines
#37
tonykchen
closed
7 months ago
0
Save run outputs to an artifacts directory
#36
tonykchen
closed
7 months ago
0
Implement backoff strategy
#35
tonykchen
closed
7 months ago
0
Use `.jinja` extension
#34
tonykchen
closed
7 months ago
0
Remove autoescaping for templates
#33
tonykchen
closed
7 months ago
0
Add pass rate metric
#32
tonykchen
closed
6 months ago
2
Global test cases
#31
tonykchen
opened
7 months ago
0
Add exceptions to trace
#30
tonykchen
opened
7 months ago
0
Fix security vulnerabilities
#29
tonykchen
closed
7 months ago
0
Fix inconsistent evaluation results
#28
tonykchen
closed
7 months ago
1
add work-dir as a configurable param
#27
sharonxiaohanli
closed
7 months ago
1
add work-dir as cli parameter
#26
sharonxiaohanli
closed
7 months ago
0
Pre-release tasks
#25
tonykchen
closed
8 months ago
0
Address inconsistent evaluation results
#24
tonykchen
closed
7 months ago
0
add junit/ to gitignore and fix ci.yml
#23
sharonxiaohanli
closed
8 months ago
0
pass in configurable work directory
#22
sharonxiaohanli
closed
7 months ago
0
Formalize test plan schema
#21
sharonxiaohanli
opened
8 months ago
1
Add Bedrock knowledge base as target
#20
tonykchen
closed
7 months ago
0
Provide user interface to analyze and compare runs
#19
tonykchen
closed
6 months ago
0
Add diagram on how Agent Evaluation works
#18
tonykchen
closed
7 months ago
0
Dispatch evaluator as Lambda function
#17
tonykchen
closed
1 month ago
1
Add backoff strategy for evaluators and targets
#16
tonykchen
closed
7 months ago
0
Add Amazon Lex as a target
#15
tonykchen
closed
5 months ago
1
Add a CLI option to select tests to run
#14
tonykchen
closed
7 months ago
0
Add support for Claude 3 Haiku
#13
tonykchen
opened
8 months ago
1
Add hooks
#12
tonykchen
closed
8 months ago
0
Add example for REST API as a target
#11
tonykchen
closed
8 months ago
0
Add documentation regarding traces
#10
tonykchen
closed
8 months ago
0
Rename task docstring reference to test
#9
tonykchen
closed
8 months ago
0
Add Creators section to README
#8
bobbywlindsey
closed
8 months ago
0
Rename task to test
#7
tonykchen
closed
8 months ago
0
Update task to test; some small fixes
#6
bobbywlindsey
closed
8 months ago
0
Add trace handler
#5
tonykchen
closed
8 months ago
0
Update docs
#4
tonykchen
closed
8 months ago
0
Prerelease chores
#3
tonykchen
closed
8 months ago
0
Add CI workflow
#2
tonykchen
closed
8 months ago
0
Update docs
#1
tonykchen
closed
8 months ago
0
Previous