search

aws-samples / foundation-model-benchmarking-tool

Foundation model benchmarking tool. Run any model on any AWS platform and benchmark for performance across instance type and serving stack options.
https://aws-samples.github.io/foundation-model-benchmarking-tool/
MIT No Attribution
182 stars 27 forks source link

adding platform identification for deployment #187

Closed madhurprash closed 3 weeks ago

madhurprash commented 1 month ago

This PR contains the following changes:

  1. Adding support for platform identification during model deployment. For example, if the model is being deployed on EC2, then the 'list_endpoints' will not be called, only if the model is deployed on SageMaker
  2. Change in 2_deploy notebook to handle/track if the ep is a sagemaker ep
  3. Changes in the deploy scripts + constants

This PR is tested with a config file on EC2 (without any sagemaker permissions) and with SageMaker to test for backward compatibility.