aws / fmeval

Foundation Model Evaluations Library
http://aws.github.io/fmeval
Apache License 2.0
153 stars 40 forks source link

refactor: update Transform API #216

Closed danielezhu closed 3 months ago

danielezhu commented 3 months ago

Description of changes: This PR updates Transform.__init__ so that input_keys and output_keys are no longer required as the first two positional arguments. Instead, to set the values of input_keys and output_keys, we now rely on the register_input_output_keys method. Under this new scheme, the __init__ signature for transforms like the semantic perturbation transforms becomes less awkward, as we no longer have to pass the redundant input_keys argument.

This PR also adds a couple unit tests for validate_call that should've been included in the PR where it was implemented.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.