aws / fmeval

Foundation Model Evaluations Library
http://aws.github.io/fmeval
Apache License 2.0
214 stars 46 forks source link

feat: modify GeneratePrompt transform to take placeholder_dict #288

Closed xiaoyi-cheng closed 5 months ago

xiaoyi-cheng commented 5 months ago

Issue #, if available: Currently GeneratePrompt transform can only substitute default placeholder $model_input with data field. This PR will support placeholder_data_dict (https://github.com/aws/fmeval/pull/273)

Description of changes: placeholder_keys_dict will be placeholder keys to input keys mapping, but in this way it only supports transform 1 prompt with one placeholder dict. See unit test for use case

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.