aws / fmeval

Foundation Model Evaluations Library
http://aws.github.io/fmeval
Apache License 2.0
151 stars 40 forks source link

feat: allow placeholder dict for prompt composer #273

Closed xiaoyi-cheng closed 1 month ago

xiaoyi-cheng commented 1 month ago

Issue #, if available: Existing PromptComposer only accept $model_input to be replaced in template. Need to add flexibility to later accept $context, $statement, $target_output, etc.

Description of changes: Add placeholder_data_dict to PromptComposer.compose(). This is a non-breaking change.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.