search

aws-samples / foundation-model-benchmarking-tool

Foundation model benchmarking tool. Run any model on any AWS platform and benchmark for performance across instance type and serving stack options.
https://aws-samples.github.io/foundation-model-benchmarking-tool/
MIT No Attribution
196 stars 31 forks source link

Add support for Bedrock Custom Import Models #224

Open dickren123 opened 2 weeks ago

dickren123 commented 2 weeks ago

Now the custom import model is GAed, we need to evaluate the cost performance using FMBench with customized model arn.

madhurprash commented 2 weeks ago

Thanks for this idea! We will be working on it and keep you posted. To benchmark a custom model, we would simply have to write a predictor (https://aws-samples.github.io/foundation-model-benchmarking-tool/byo_rest_predictor.html) that supports the prediction format for custom model imports (nothing too complex, the function would be similar to what is given in https://aws.amazon.com/cn/blogs/machine-learning/amazon-bedrock-custom-model-import-now-generally-available/, using the converse API.