Runtime Error: forward() got an unexpected keyword argument 'cache_position'

[Meng-Fanshu]

code: query = 'What does the picture show?' image_paths = ['/home/downloads/test.jpg']

huatuogpt_vision_model_path = "/home/llm_models/HuatuoGPT-Vision-7B"

from cli import HuatuoChatbot bot = HuatuoChatbot(huatuogpt_vision_model_path) output = bot.inference(query, image_paths) print(output) # Prints the output of the model

Error Info: /home/mengfanshu/workplace/HuatuoGPT-Vision loading from /home/llm_models/HuatuoGPT-Vision-7B loading from CLIP first. This should only be used at inference!!! loading vision model from openai/clip-vit-large-patch14-336 Loading checkpoint shards: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:00<00:00, 4.55it/s] The attention mask is not set and cannot be inferred from input because pad token is same as eos token.As a consequence, you may observe unexpected behavior. Please pass your input's attention_mask to obtain reliable results. Traceback (most recent call last): File "/home/mengfanshu/workplace/HuatuoGPT-Vision/quickStart.py", line 8, in output = bot.inference(query, image_paths) File "/home/mengfanshu/workplace/HuatuoGPT-Vision/cli.py", line 276, in inference output_ids = self.model.generate( File "/home/mengfanshu/miniconda3/envs/pytorch/lib/python3.9/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context return func(*args, kwargs) File "/home/mengfanshu/workplace/HuatuoGPT-Vision/llava/model/language_model/llava_qwen2.py", line 150, in generate return super().generate( File "/home/mengfanshu/miniconda3/envs/pytorch/lib/python3.9/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context return func(args, kwargs) File "/home/mengfanshu/miniconda3/envs/pytorch/lib/python3.9/site-packages/transformers/generation/utils.py", line 1989, in generate result = self._sample( File "/home/mengfanshu/miniconda3/envs/pytorch/lib/python3.9/site-packages/transformers/generation/utils.py", line 2932, in _sample outputs = self(model_inputs, return_dict=True) File "/home/mengfanshu/miniconda3/envs/pytorch/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl return self._call_impl(args, kwargs) File "/home/mengfanshu/miniconda3/envs/pytorch/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl return forward_call(*args, **kwargs) TypeError: forward() got an unexpected keyword argument 'cache_position'

[HuichiZhou]

Hello, you should pip install transformer==4.37.2

[SiddhantOjha17]

how did you download the model ?

[mrjzz]

how did you download the model ?

Is the way of model-downloading relevant with this error?

[mrjzz]

Hello, you should pip install transformer==4.37.2 Thx, it works.