BUG: 最新的glm4v代码使用transformers `AutoModelForCausalLM`类加载时强制要求安装`flash_attn`

ChengjieLi28 commented 6 days ago

System Info / 系統信息

Python 3.10 Cuda 12.1

Who can help? / 谁可以帮助到您？

@zRzRzRzRzRzRzR

Information / 问题信息

[ ] The official example scripts / 官方的示例脚本
[X] My own modified scripts / 我自己修改的脚本和任务

Reproduction / 复现过程

MODEL_PATH='<path>'
model = AutoModelForCausalLM.from_pretrained(
            MODEL_PATH,
            low_cpu_mem_usage=True,
            trust_remote_code=True,
            torch_dtype=torch.float16,
            device_map="auto"
        )

报错大概是这个错误栈，transformers的错误栈：

File "/home/lichengjie/workspace/inference/xinference/model/llm/pytorch/glm4v.py", line 87, in load
    model = AutoModelForCausalLM.from_pretrained(
  File "/home/lichengjie/miniconda3/envs/xinf/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 550, in from_pretrained
    model_class = get_class_from_dynamic_module(
  File "/home/lichengjie/miniconda3/envs/xinf/lib/python3.10/site-packages/transformers/dynamic_module_utils.py", line 501, in get_class_from_dynamic_module
    final_module = get_cached_module_file(
  File "/home/lichengjie/miniconda3/envs/xinf/lib/python3.10/site-packages/transformers/dynamic_module_utils.py", line 326, in get_cached_module_file
    modules_needed = check_imports(resolved_module_file)
  File "/home/lichengjie/miniconda3/envs/xinf/lib/python3.10/site-packages/transformers/dynamic_module_utils.py", line 181, in check_imports
    raise ImportError(
ImportError: [address=[0.0.0.0:33683](http://0.0.0.0:33683/), pid=1938627] This modeling file requires the following packages that were not found in your environment: flash_attn. Run `pip install flash_attn`