huggingface / transformers

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.
https://huggingface.co/transformers
Apache License 2.0
135.05k stars 27.02k forks source link

Issue with transformers 4.45.0 and torchao 0.1 cannot import name 'quantize_' from 'torchao.quantization' #33727

Closed EnragedAntelope closed 1 month ago

EnragedAntelope commented 1 month ago

System Info

Transformers 4.45.0 torchao 0.1 Python 3.11.9 Windows 11

(venv) D:\ComfyUI>transformers-cli env
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "D:\ComfyUI\venv\Scripts\transformers-cli.exe\__main__.py", line 5, in <module>
  File "D:\ComfyUI\venv\Lib\site-packages\transformers\commands\transformers_cli.py", line 24, in <module>
    from .run import RunCommand
  File "D:\ComfyUI\venv\Lib\site-packages\transformers\commands\run.py", line 17, in <module>
    from ..pipelines import Pipeline, PipelineDataFormat, get_supported_tasks, pipeline
  File "D:\ComfyUI\venv\Lib\site-packages\transformers\pipelines\__init__.py", line 77, in <module>
    from .text_to_audio import TextToAudioPipeline
  File "D:\ComfyUI\venv\Lib\site-packages\transformers\pipelines\text_to_audio.py", line 22, in <module>
    from ..models.speecht5.modeling_speecht5 import SpeechT5HifiGan
  File "D:\ComfyUI\venv\Lib\site-packages\transformers\models\speecht5\modeling_speecht5.py", line 36, in <module>
    from ...modeling_utils import PreTrainedModel
  File "D:\ComfyUI\venv\Lib\site-packages\transformers\modeling_utils.py", line 58, in <module>
    from .quantizers import AutoHfQuantizer, HfQuantizer
  File "D:\ComfyUI\venv\Lib\site-packages\transformers\quantizers\__init__.py", line 14, in <module>
    from .auto import AutoHfQuantizer, AutoQuantizationConfig
  File "D:\ComfyUI\venv\Lib\site-packages\transformers\quantizers\auto.py", line 42, in <module>
    from .quantizer_torchao import TorchAoHfQuantizer
  File "D:\ComfyUI\venv\Lib\site-packages\transformers\quantizers\quantizer_torchao.py", line 35, in <module>
    from torchao.quantization import quantize_
ImportError: cannot import name 'quantize_' from 'torchao.quantization' (D:\ComfyUI\venv\Lib\site-packages\torchao\quantization\__init__.py)

Who can help?

@SunMarc I'm assuming to tag you because it lists a quantize error?

Information

Tasks

Reproduction

I originally found this when starting ComfyUI immediately after updating to Transformers 4.45.0, since a new VLM package indicated it needed latest Transformers. Easy to reproduce:

(venv) D:\ComfyUI>transformers-cli env
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "D:\ComfyUI\venv\Scripts\transformers-cli.exe\__main__.py", line 5, in <module>
  File "D:\ComfyUI\venv\Lib\site-packages\transformers\commands\transformers_cli.py", line 24, in <module>
    from .run import RunCommand
  File "D:\ComfyUI\venv\Lib\site-packages\transformers\commands\run.py", line 17, in <module>
    from ..pipelines import Pipeline, PipelineDataFormat, get_supported_tasks, pipeline
  File "D:\ComfyUI\venv\Lib\site-packages\transformers\pipelines\__init__.py", line 77, in <module>
    from .text_to_audio import TextToAudioPipeline
  File "D:\ComfyUI\venv\Lib\site-packages\transformers\pipelines\text_to_audio.py", line 22, in <module>
    from ..models.speecht5.modeling_speecht5 import SpeechT5HifiGan
  File "D:\ComfyUI\venv\Lib\site-packages\transformers\models\speecht5\modeling_speecht5.py", line 36, in <module>
    from ...modeling_utils import PreTrainedModel
  File "D:\ComfyUI\venv\Lib\site-packages\transformers\modeling_utils.py", line 58, in <module>
    from .quantizers import AutoHfQuantizer, HfQuantizer
  File "D:\ComfyUI\venv\Lib\site-packages\transformers\quantizers\__init__.py", line 14, in <module>
    from .auto import AutoHfQuantizer, AutoQuantizationConfig
  File "D:\ComfyUI\venv\Lib\site-packages\transformers\quantizers\auto.py", line 42, in <module>
    from .quantizer_torchao import TorchAoHfQuantizer
  File "D:\ComfyUI\venv\Lib\site-packages\transformers\quantizers\quantizer_torchao.py", line 35, in <module>
    from torchao.quantization import quantize_
ImportError: cannot import name 'quantize_' from 'torchao.quantization' (D:\ComfyUI\venv\Lib\site-packages\torchao\quantization\__init__.py)

Expected behavior

No errors between Transformers and Torchao. I think something changed in latest Transformers since everything was OK until I updated today.

EnragedAntelope commented 1 month ago

Sorry. Realized my torchao was outdated, had to build it from source but everything is working again. My apologies. Will leave this in case anyone else has a similar error message, they will need to update torchao to latest per their github version (for some reason pip install --upgrade torchao kept me on 0.1, only building from source updated my version).