About the transformers version.

[xin-ran-w]

Hello, I want to try this amazing work. I use the transformer version used by LLaVA 2024-4-25, that is, transformers==4.37.2. What version of transformers do you use? And I met the following error:

Traceback (most recent call last):
  File "/home/wangxinran/Prompt-Highlighter/examples/llava_description.py", line 242, in <module>
    eval_model(args)
  File "/home/wangxinran/Prompt-Highlighter/examples/llava_description.py", line 157, in eval_model
    output_ids = model.generate(
  File "/data/wangxinran/anaconda3/envs/highlighter/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/home/wangxinran/Prompt-Highlighter/LLaVA/llava/model/language_model/llava_llama.py", line 137, in generate
    return super().generate(
  File "/data/wangxinran/anaconda3/envs/highlighter/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/data/wangxinran/anaconda3/envs/highlighter/lib/python3.10/site-packages/transformers/generation/utils.py", line 1479, in generate
    return self.greedy_search(
  File "/data/wangxinran/anaconda3/envs/highlighter/lib/python3.10/site-packages/transformers/generation/utils.py", line 2340, in greedy_search
    outputs = self(
  File "/data/wangxinran/anaconda3/envs/highlighter/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/data/wangxinran/anaconda3/envs/highlighter/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/wangxinran/Prompt-Highlighter/./highlighter_modules/attention_llama_llava.py", line 268, in llava_hl_forward
    return super(LlavaLlamaForCausalLM, self).forward(
  File "/data/wangxinran/anaconda3/envs/highlighter/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py", line 1183, in forward
    outputs = self.model(
  File "/data/wangxinran/anaconda3/envs/highlighter/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/data/wangxinran/anaconda3/envs/highlighter/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
  File "/data/wangxinran/anaconda3/envs/highlighter/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py", line 1070, in forward
    layer_outputs = decoder_layer(
  File "/data/wangxinran/anaconda3/envs/highlighter/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/data/wangxinran/anaconda3/envs/highlighter/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
  File "/data/wangxinran/anaconda3/envs/highlighter/lib/python3.10/site-packages/accelerate/hooks.py", line 165, in new_forward
    output = old_forward(*args, **kwargs)
  File "/data/wangxinran/anaconda3/envs/highlighter/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py", line 798, in forward
    hidden_states, self_attn_weights, present_key_value = self.self_attn(
  File "/data/wangxinran/anaconda3/envs/highlighter/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/data/wangxinran/anaconda3/envs/highlighter/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/wangxinran/Prompt-Highlighter/./highlighter_modules/attention_llama_llava.py", line 101, in llama_new_forward
    kv_seq_len += past_key_value[0].shape[-2]
  File "/data/wangxinran/anaconda3/envs/highlighter/lib/python3.10/site-packages/transformers/cache_utils.py", line 78, in __getitem__
    raise KeyError(f"Cache only has {len(self)} layers, attempted to access layer with index {layer_idx}")
KeyError: 'Cache only has 0 layers, attempted to access layer with index 0'

[zhangshilong123]

pip uninstall unsloth -y pip install --upgrade --force-reinstall --no-cache-dir git+https://github.com/unslothai/unsloth.git 试一下

[dingyue772]

Hi, did solve the problem? I met the same problem with llava vision is 1.1.3 with LLaVA vision is v1.2.0 transformers 4.36.2 torch 2.0.1