Closed pamelafox closed 1 month ago
/evaluate
Starting evaluation! Check the Actions tab for progress, or wait for a comment with the results.
metric | stat | baseline | pr126 |
---|---|---|---|
gpt_groundedness | pass_rate | 1.0 | 1.0 |
↑ | mean_rating | 5.0 | 5.0 |
gpt_relevance | pass_rate | 1.0 | 1.0 |
↑ | mean_rating | 5.0 | 5.0 |
answer_length | mean | 1017.6 | 1011.4 |
latency | mean | 2.56 | 2.62 |
citations_matched | mean | 0.73 | 0.73 |
Purpose
This pull request includes changes to the GitHub Actions workflow to correctly identify and use the pull request branch name when committing and pushing evaluation results. The most important changes involve adding a step to get the PR branch name and modifying the branch checkout and push commands.
Improvements to workflow:
.github/workflows/evaluate.yaml
: Added a step to get the PR branch name usingactions/github-script@v7
and modified the branch checkout command to create a temporary branch..github/workflows/evaluate.yaml
: Updated the push command to use the PR branch name instead of the default branch.Does this introduce a breaking change?
When developers merge from main and run the server, azd up, or azd deploy, will this produce an error? If you're not sure, try it out on an old environment.
Type of change
Code quality checklist
See CONTRIBUTING.md for more details.
python -m pytest
).python -m pytest --cov
to verify 100% coverage of added linespython -m mypy
to check for type errorsruff
manually on my code.