search

huggingface / optimum-nvidia

Apache License 2.0
867 stars 86 forks source link

ImportError: Using `low_cpu_mem_usage=True` or a `device_map` requires Accelerate: `pip install accelerate` #56

Open taozhang9527 opened 8 months ago

taozhang9527 commented 8 months ago

I am trying to run the example generate code provided with the docker image version 0.1.0b2. It gives me the following error:

  File "/opt/optimum-nvidia/src/optimum/nvidia/runtime.py", line 154, in _from_pretrained
    builder.build(engine_folder, optimization_level)
  File "/opt/optimum-nvidia/src/optimum/nvidia/builder.py", line 378, in build
    hf_model = AutoModelForCausalLM.from_pretrained(
  File "/usr/local/lib/python3.10/dist-packages/transformers/models/auto/auto_factory.py", line 566, in from_pretrained
    return model_class.from_pretrained(
  File "/usr/local/lib/python3.10/dist-packages/transformers/modeling_utils.py", line 2863, in from_pretrained
    raise ImportError(
ImportError: Using `low_cpu_mem_usage=True` or a `device_map` requires Accelerate: `pip install accelerate`

Confirmed that the accelerate package has been installed. 

root@r760xa-l40:/opt/optimum-nvidia/examples# pip list | grep accelerate
accelerate                0.20.3

It seems the + use_fp8=True, is where the issue was generated. I can run it ok with the transformers lib without fp8.