aws / sagemaker-inference-toolkit

Serve machine learning models within a 🐳 Docker container using 🧠 Amazon SageMaker.
Apache License 2.0
370 stars 82 forks source link

Launch MMS without repackaging model contents #102

Closed fm1ch4 closed 1 year ago

fm1ch4 commented 2 years ago

Describe the feature you'd like Inference toolkit, when starting up MMS, will repackage the model contents by copying the contents from /opt/ml/model to /.sagemaker/mms/models: https://github.com/aws/sagemaker-inference-toolkit/blob/master/src/sagemaker_inference/model_server.py#L76.

This is unnecessary and MMS can simply read the model contents from /opt/ml/model. This will save some startup time due to removing the need to copy files from one location to another, and will also help the container restart successfully as it will not run into the issue where /.sagemaker/mms/models/model is already present (say if the container crashed and is restarted on the same host).

How would this feature be used? Please describe. See above

Describe alternatives you've considered N/A

Additional context N/A