Add support for non aws models - openAI + gemini

aws-samples / foundation-model-benchmarking-tool

Foundation model benchmarking tool. Run any model on any AWS platform and benchmark for performance across instance type and serving stack options.

MIT No Attribution

202 stars 31 forks source link

Open madhurprash opened 1 month ago

madhurprash commented 1 month ago

This PR contains the following:

Added external_predictor.py for handling inferences for openAI + gemini
Prompt templates for gemini + openAI
configuration files for openai only + gemini only + openai/gemini/bedrock models