search

aws / fmeval

Foundation Model Evaluations Library
http://aws.github.io/fmeval
Apache License 2.0
153 stars 40 forks source link

Toxicity transforms #227

Closed franluca closed 3 months ago

franluca commented 3 months ago

Refactor toxicity evaluation to use transform pipelines.

Added batch_size property for transforms, in order to use map_batch

Minor modifications to helper models

Note: eval-specific unit and integration tests have not been modified!!!!

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

danielezhu commented 3 months ago

Some context for anyone else who sees the huge git diff on this PR. I raised PR #224 which has a huge diff, and Luca started his work on Toxicity based off my branch.