aws / fmeval

Foundation Model Evaluations Library
http://aws.github.io/fmeval
Apache License 2.0
151 stars 40 forks source link

feat: add faithfulness eval algo #291

Closed xiaoyi-cheng closed 1 week ago

xiaoyi-cheng commented 2 weeks ago

Issue #, if available: This PR adds the faithfulness metric under the Faithfulness evaluation algorithm.

Description of changes:

  1. Default dataset currently stored in a private S3 bucket. (haven't tested yet)

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.