search

aws-neuron / aws-neuron-sdk

Powering AWS purpose-built machine learning chips. Blazing fast and cost effective, natively integrated into PyTorch and TensorFlow and integrated with your favorite AWS services
https://aws.amazon.com/machine-learning/neuron/
Other
468 stars 154 forks source link

Using neuron sdk with sagemaker #469

Closed Shubham-Sharma-09 closed 2 years ago

Shubham-Sharma-09 commented 2 years ago

I am trying to use neuron sdk on sagemaker and unable to import torch.neuron on sagemaker. Kindly guide as to how to import it on sagemaker. I am trying to build this low cost inferencing solution on sagemaker.

aws-taylor commented 2 years ago

Hello @Shubham-Sharma-09,

Take a look at https://awsdocs-neuron.readthedocs-hosted.com/en/latest/src/examples/pytorch/byoc_sm_bert_tutorial/sagemaker_container_neuron.html and linked documentation for an example on how to deploy a model using Inferentia and SageMaker. You may find this blog post useful as well: https://aws.amazon.com/blogs/machine-learning/achieve-12x-higher-throughput-and-lowest-latency-for-pytorch-natural-language-processing-applications-out-of-the-box-on-aws-inferentia/

-Taylor