search

aws / fmeval

Foundation Model Evaluations Library
http://aws.github.io/fmeval
Apache License 2.0
153 stars 40 forks source link

feat: update implementation of SummarizationAccuracySemanticRobustness to use Transform-based approach #233

Closed danielezhu closed 3 months ago

danielezhu commented 3 months ago

Description of changes: See title. This PR also fixes a bug introduced in the previous PR where the invoke_model call in BertScore was not replaced with get_helper_scores.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.