使用脚本convert_llama13b_to_fs.sh转换模型时报错

[coolboyqu]

Traceback (most recent call last): File "/dssg/home/scs2010812167/qy/Ziya-LLaMA/Fengshenbang-LM-main/fengshen/utils/llama_convert/hf_to_fs.py", line 87, in fs_model = FengshenLlama(fs_config) File "/dssg/home/scs2010812167/qy/Ziya-LLaMA/Fengshenbang-LM-main/fengshen/models/llama/modeling_llama.py", line 244, in init self.llama = LlamaModel(config) File "/dssg/home/scs2010812167/qy/Ziya-LLaMA/Fengshenbang-LM-main/fengshen/models/llama/modeling_llama.py", line 120, in init rotary=True) for i in range(config.num_hidden_layers)]) File "/dssg/home/scs2010812167/qy/Ziya-LLaMA/Fengshenbang-LM-main/fengshen/models/llama/modeling_llama.py", line 120, in rotary=True) for i in range(config.num_hidden_layers)]) File "/dssg/home/scs2010812167/qy/Ziya-LLaMA/Fengshenbang-LM-main/fengshen/models/megatron/layers/transformer.py", line 668, in init parallel_output=self.gpt_j_residual, File "/dssg/home/scs2010812167/qy/Ziya-LLaMA/Fengshenbang-LM-main/fengshen/models/megatron/layers/transformer.py", line 271, in init from .flash_attention import ( File "/dssg/home/scs2010812167/qy/Ziya-LLaMA/Fengshenbang-LM-main/fengshen/models/megatron/layers/flash_attention.py", line 7, in import flash_attn_cuda ModuleNotFoundError: No module named 'flash_attn_cuda'

已在根目录执行pip3 install --editable . 且执行成功

nvcc版本：

nvcc -V nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2021 NVIDIA Corporation Built on Sun_Mar_21_19:15:46_PDT_2021 Cuda compilation tools, release 11.3, V11.3.58 Build cuda_11.3.r11.3/compiler.29745058_0

pytorch所需要的cuda版本：

torch.version.cuda '11.3'

可以帮忙分析一下是什么问题嘛

[tobi0520]

我也是这个问题

[karlshoo]

同一个问题，请问楼主解决了吗 @coolboyqu

[Desein-Yang]

因为 fengshen 其他模块为了训练加速引入了 flash_attention 模块。

只用转换脚本的话不用安装这个，可以注释掉fengshen/models/megatron/layers/transformer.py的from .flash_attention import xx

如需安装安装参考 https://github.com/Dao-AILab/flash-attention/tree/main/flash_attn。