search

philschmid / huggingface-sagemaker-workshop-series

Enterprise Scale NLP with Hugging Face & SageMaker Workshop series
MIT License
234 stars 56 forks source link

Conflicting AWS and HF docs for AWS inf instances, confused about which one to follow #9

Closed sidyakinian closed 2 years ago

sidyakinian commented 2 years ago

I'm trying to set up inference on AWS inf for a pretrained HF transformer. I followed this HF tutorial and ran into errors, opened an issue on aws-neuron-sdk repo, was told that env setup is wrong. I'm quite confused now whether I should follow AWS tutorials, which advise to setup env according to this AWS doc, or HF tutorials which setup env with two pip install lines. Would appreciate the help!

philschmid commented 2 years ago

Hey @sidyakinian,

The tutorial uses the exact same setup than the AWS Doc see: https://awsdocs-neuron.readthedocs-hosted.com/en/latest/neuron-intro/pytorch-setup/pytorch-install.html#compile-on-compute-instance.

# Note: There is no DLAMI Conda environment for this framework version
#       Framework will be installed/updated inside a Python environment

# Activate PyTorch
source activate aws_neuron_pytorch_p36

# Set Pip repository  to point to the Neuron repository
pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com

#Install Neuron PyTorch
pip install torch-neuron==1.9.1.* neuron-cc[tensorflow] "protobuf<4" torchvision
sidyakinian commented 2 years ago

@philschmid Makes sense, sorry, got a little confused with torch versions. Thanks!