search

ymcui / Chinese-LLaMA-Alpaca-2

中文LLaMA-2 & Alpaca-2大模型二期项目 + 64K超长上下文模型 (Chinese LLaMA-2 & Alpaca-2 LLMs with 64K long context models)
Apache License 2.0
7.04k stars 581 forks source link

fix: 解决llama-2-70B(chat)模型推理阶段报错 #413

Open olifei opened 9 months ago

olifei commented 9 months ago

原因：Llama-2-70B transformer结构num_heads与num_key_value_heads不相等，会导致在forward阶段计算attention时维度出现错误。参考transformers进行修改: https://github.com/huggingface/transformers/blob/main/src/transformers/models/llama/modeling_llama.py#L263