Update Sagemaker Predictor to delete EP - Githubissues

aws-samples / foundation-model-benchmarking-tool

Foundation model benchmarking tool. Run any model on any AWS platform and benchmark for performance across instance type and serving stack options.

https://aws-samples.github.io/foundation-model-benchmarking-tool/

MIT No Attribution

197 stars 31 forks source link

Update Sagemaker Predictor to delete EP #233

Closed dheerajoruganty closed 2 weeks ago

dheerajoruganty commented 2 weeks ago

Summary

This PR modifies the sagemaker_predictor.py function to ensure that the SageMaker endpoint is deleted after inference is completed.

Changes

Updated the sagemaker_predictor.py function to automatically delete the SageMaker endpoint once inference is finished.
Tested the changes with multiple endpoint types (ml.g5.xlarge, ml.g4dn.xlarge) and a single endpoint (ml.g5.xlarge).

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.