进行reweite时报错 - Githubissues

运行infer-llama时报错我之前用自己微调的chatglm好像没出现这个问题，但是结果不太好换成原本的llama后这里好像是解包txt出错。。。
Vocab of the base model: 32000
Vocab of the tokenizer: 32000
loading peft model
0it [00:00, ?it/s]Setting `pad_token_id` to `eos_token_id`:2 for open-end generation.
0it [00:00, ?it/s]
Traceback (most recent call last):
  File "/public2/home/wangchen/Retrieve-Rewrite-Answer-main/KGQA-MetaQA/rewrite/infer_llama.py", line 75, in <module>
    generation_output = model.generate(
  File "/public2/home/wangchen/.local/lib/python3.10/site-packages/peft/peft_model.py", line 977, in generate
    outputs = self.base_model.generate(**kwargs)
  File "/opt/conda/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/public2/home/wangchen/.local/lib/python3.10/site-packages/transformers/generation/utils.py", line 1719, in generate
    return self.sample(
  File "/public2/home/wangchen/.local/lib/python3.10/site-packages/transformers/generation/utils.py", line 2801, in sample
    outputs = self(
  File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
  File "/public2/home/wangchen/.local/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py", line 1034, in forward
    outputs = self.model(
  File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
  File "/public2/home/wangchen/.local/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py", line 922, in forward
    layer_outputs = decoder_layer(
  File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
  File "/public2/home/wangchen/.local/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py", line 672, in forward
    hidden_states, self_attn_weights, present_key_value = self.self_attn(
  File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
  File "/public2/home/wangchen/Retrieve-Rewrite-Answer-main/KGQA-MetaQA/rewrite/attn_and_long_ctx_patches.py", line 53, in xformers_forward
    cos, sin = self.rotary_emb(value_states, seq_len=kv_seq_len)
  File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
  File "/public2/home/wangchen/Retrieve-Rewrite-Answer-main/KGQA-MetaQA/rewrite/attn_and_long_ctx_patches.py", line 172, in adaptive_ntk_forward
    self.cos_cached[:, :, :seq_len, ...].to(dtype=x.dtype),
IndexError: too many indices for tensor of dimension 2
wuyike2000 / Retrieve-Rewrite-Answer

进行reweite时报错 #5
