Toolkit for allowing inference and serving with PyTorch on SageMaker. Dockerfiles used for building SageMaker Pytorch Containers are at https://github.com/aws/deep-learning-containers.
Issue #, if available:
The torchserve vmargs argument is hard-coded and by default uses a small fraction of the total available memory. This causes issues when loading models into memory.
Description of changes:
The torchserve configuration process now respects the pre-existing environment variable "SAGEMAKER_MODEL_SERVER_VMARGS". When this environment variable is missing, the default value (taken from sagemaker_inference.environment) matches the previously hard-coded value.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
Issue #, if available: The torchserve vmargs argument is hard-coded and by default uses a small fraction of the total available memory. This causes issues when loading models into memory.
Description of changes: The torchserve configuration process now respects the pre-existing environment variable "SAGEMAKER_MODEL_SERVER_VMARGS". When this environment variable is missing, the default value (taken from sagemaker_inference.environment) matches the previously hard-coded value.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.