Previously we were not able to see the model response for Fairness and Accuracy, and we calculate the score in the generated_results , without knowing the model response, which leads to confusion.
The aim of this Pull request is to allow user to see in the model repones for Fairness and Accuracy.
➤ Fixes # (issue)
Type of change
Please delete options that are not relevant.
[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update
Usage
harness.model_response(category="accuracy")
Checklist:
[ ] I've added Google style docstrings to my code.
[ ] I've used pydantic for typing when/where necessary.
Description
Previously we were not able to see the model response for Fairness and Accuracy, and we calculate the score in the generated_results , without knowing the model response, which leads to confusion.
The aim of this Pull request is to allow user to see in the model repones for Fairness and Accuracy.
➤ Fixes # (issue)
Type of change
Please delete options that are not relevant.
Usage
Checklist:
pydanticfor typing when/where necessary.Screenshots (if appropriate):
QA
ner
text-classification
summarization