aws-samples / fm-leaderboarder

FM-Leaderboard-er allows you to create leaderboard to find the best LLM/prompt for your own business use case based on your data, task, prompts
Apache License 2.0
18 stars 5 forks source link

Support Claude-3 like messages API #14

Open gilinachum opened 5 months ago

gilinachum commented 5 months ago

According to: https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-anthropic-claude-messages.html We'll have a dedicated Runner class that its constructor will accept a function that will generate the messages json, then the predict function will call this function with the specific piece of data, and get back the messages json to use.