运行model.generate报错IndexError: too many indices for tensor of dimension 2

[1801ZDL]

提交前必须检查以下项目

• [X] 请确保使用的是仓库最新代码（git pull），一些问题已被解决和修复。
• [X] 我已阅读项目文档和FAQ章节并且已在Issue中对问题进行了搜索，没有找到相似问题和解决方案。
• [X] 第三方插件问题：例如llama.cpp、LangChain、text-generation-webui等，同时建议到对应的项目中查找解决方案。

问题类型

模型推理

基础模型

Chinese-Alpaca-2 (7B/13B)

操作系统

None

详细描述问题

# 请在此处粘贴运行代码（请粘贴在本代码块里）

Start inference. Traceback (most recent call last): File "/data/zhangdl/Chinese-LLaMA-Alpaca-2-main/scripts/inference/inference_hf1.py", line 246, in generation_output = model.generate( File "/data/zhangdl/anaconda3/envs/py310/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(*args, kwargs) File "/data/zhangdl/anaconda3/envs/py310/lib/python3.10/site-packages/transformers/generation/utils.py", line 1719, in generate return self.sample( File "/data/zhangdl/anaconda3/envs/py310/lib/python3.10/site-packages/transformers/generation/utils.py", line 2801, in sample outputs = self( File "/data/zhangdl/anaconda3/envs/py310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, *kwargs) File "/data/zhangdl/anaconda3/envs/py310/lib/python3.10/site-packages/accelerate/hooks.py", line 165, in new_forward output = old_forward(args, kwargs) File "/data/zhangdl/anaconda3/envs/py310/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py", line 1034, in forward outputs = self.model( File "/data/zhangdl/anaconda3/envs/py310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, kwargs) File "/data/zhangdl/anaconda3/envs/py310/lib/python3.10/site-packages/accelerate/hooks.py", line 165, in new_forward output = old_forward(*args, *kwargs) File "/data/zhangdl/anaconda3/envs/py310/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py", line 922, in forward layer_outputs = decoder_layer( File "/data/zhangdl/anaconda3/envs/py310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(args, kwargs) File "/data/zhangdl/anaconda3/envs/py310/lib/python3.10/site-packages/accelerate/hooks.py", line 165, in new_forward output = old_forward(*args, kwargs) File "/data/zhangdl/anaconda3/envs/py310/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py", line 672, in forward hidden_states, self_attn_weights, present_key_value = self.self_attn( File "/data/zhangdl/anaconda3/envs/py310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, *kwargs) File "/data/zhangdl/anaconda3/envs/py310/lib/python3.10/site-packages/accelerate/hooks.py", line 165, in new_forward output = old_forward(args, kwargs) File "/data/zhangdl/Chinese-LLaMA-Alpaca-2-main/scripts/attn_and_long_ctx_patches.py", line 53, in xformers_forward cos, sin = self.rotary_emb(value_states, seq_len=kv_seq_len) File "/data/zhangdl/anaconda3/envs/py310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, **kwargs) File "/data/zhangdl/Chinese-LLaMA-Alpaca-2-main/scripts/attn_and_long_ctx_patches.py", line 172, in adaptive_ntk_forward self.cos_cached[:, :, :seq_len, ...].to(dtype=x.dtype), IndexError: too many indices for tensor of dimension 2

依赖情况（代码类问题务必提供）

# 请在此处粘贴依赖情况（请粘贴在本代码块里）

torch 2.0.0+cu117 transformers 4.35.2

运行日志或截图

# 请在此处粘贴运行日志（请粘贴在本代码块里）

[iMountTai]

pull 最新代码试试，已经更新过了

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your consideration.

[github-actions[bot]]

Closing the issue, since no updates observed. Feel free to re-open if you need any further assistance.