search

InternLM / xtuner

An efficient, flexible and full-featured toolkit for fine-tuning LLM (InternLM2, Llama3, Phi3, Qwen, Mistral, ...)
https://xtuner.readthedocs.io/zh-cn/latest/
Apache License 2.0
4.01k stars 315 forks source link

AttributeError: 'Qwen2FlashAttention2' object has no attribute '_flash_attention_forward' #935

Open zhangyuqi-1 opened 2 months ago

zhangyuqi-1 commented 2 months ago

https://github.com/modelscope/ms-swift/issues/2107#issuecomment-2370042882

xtuner 0.1.23

rank1: File "/data1/zhangyq/miniforge3/envs/py310/lib/python3.10/site-packages/xtuner/model/modules/dispatch/qwen2.py", line 160, in qwen2_attn_forward rank1: attn_output = self._flash_attention_forward( rank1: File "/data1/zhangyq/miniforge3/envs/py310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1729, in getattr rank1: raise AttributeError(f"'{type(self).name}' object has no attribute '{name}'")

jifeng0320 commented 2 months ago

https://github.com/InternLM/xtuner/blob/main/requirements/runtime.txt transformers >= 4.43.0 use _flash_attention_forward but not self._flash_attention_forward

shiningliang commented 1 month ago

Same issue when trying Qwen2.5 sequence parallel and fix it by downgrade transformers to 4.42.4. Will the owner migrate the code to support different versions of transformers?