Error in importing optimum neuron: cannot import name 'GenerationMode' from 'transformers.generation.utils'

huggingface / optimum-neuron

Easy, fast and very cheap training and inference on AWS Trainium and Inferentia chips.

Apache License 2.0

184 stars 57 forks source link

Error in importing optimum neuron: cannot import name 'GenerationMode' from 'transformers.generation.utils' #210

Open nd7141 opened 11 months ago

nd7141 commented 11 months ago

from transformers import AutoTokenizer
from optimum.neuron import NeuronModelForCausalLM

results in

RuntimeError: Failed to import optimum.neuron.modeling because of the following error (look up to see its traceback):
cannot import name 'GenerationMode' from 'transformers.generation.utils' (/opt/aws_neuron_venv_pytorch/lib/python3.8/site-packages/transformers/generation/utils.py)

nd7141 commented 11 months ago

I just followed this snippet: https://huggingface.co/docs/optimum-neuron/guides/models#text-generation-inference

vishwanath-gowda commented 9 months ago

I had a similar error, I checked the version of optimum package installed on my system.

$ pip list | grep optimum
optimum                       1.13.2
optimum-neuron                0.0.13

I noticed that though I had the latest version of optimum, optimum-neuron was pointing to older github release tag. when I compared the code between the release tags, I noticed the code deviations.

So I installed the latest version of optimum-neuron which fixed my issue.

pip index versions optimum-neuron
pip install optimum-neuron==0.0.13

maekawataiki commented 9 months ago

GenerationMode is added in this commit which is available from transformers>=4.32.0 so running pip install "transformers>=4.32.0" was required.