Error when executing the LLama2 inference notebook

aws-neuron / aws-neuron-samples

Example code for AWS Neuron SDK developers building inference and training applications

Other

101 stars 32 forks source link

Error when executing the LLama2 inference notebook #35

Closed massi-ang closed 9 months ago

massi-ang commented 9 months ago

Trying to run https://github.com/aws-neuron/aws-neuron-samples/blob/master/torch-neuronx/transformers-neuronx/inference/meta-llama-2-13b-sampling.ipynb

but getting error about transformers library missing

Fix: added transformers to the pip install...

shebbur-aws commented 9 months ago

Please make sure you run the " !pip install transformers-neuronx sentencepiece -U " command as in the "Install Dependencies" section of the sample notebook. This will install transformers as one of the dependencies of transformers-neuronx. We will close this issue since it sounds like you have resolved the problem.

massi-ang commented 9 months ago

Running pip install transformers-neuronx does not install transformers. I had to explicitly add it

massi-ang commented 9 months ago

!pip show transformers-neuronx

Name: transformers-neuronx
Version: 0.1
Summary: UNKNOWN
Home-page: UNKNOWN
Author: 
Author-email: 
License: Proprietary
Location: /home/ubuntu/.local/lib/python3.10/site-packages
Requires: 
Required-by:

massi-ang commented 9 months ago

After executing pip install transformers-neuronx --extra-index-url=https://pip.repos.neuron.amazonaws.com then !pip show transformers-neuronx gives:

Name: transformers-neuronx
Version: 0.6.106
Summary: UNKNOWN
Home-page: UNKNOWN
Author: 
Author-email: 
License: UNKNOWN
Location: /home/ubuntu/.local/lib/python3.10/site-packages
Requires: accelerate, torch-neuronx, transformers
Required-by: