Toolkit for allowing inference and serving with PyTorch on SageMaker. Dockerfiles used for building SageMaker Pytorch Containers are at https://github.com/aws/deep-learning-containers.
the inference-toolkit hasn't yet adopted the change required to use Elastic Inference with PyTorch 1.5.1. Following this guide. It looks like the default_model_fn would have to attach the EIA to the model before returning. Similar to this .
Hello,
the inference-toolkit hasn't yet adopted the change required to use Elastic Inference with PyTorch 1.5.1. Following this guide. It looks like the
default_model_fnwould have to attach theEIAto the model before returning. Similar to this .Any feedback is appreciated.
Thank you, Theiss