MTL Windows Qwen-VL AttributeError: 'QWenAttention' object has no attribute 'position_ids'

[juan-OY]

Follow the guide to set up qwen-VL ipex-llm version:2.1.0b20240512 https://github.com/intel-analytics/ipex-llm/blob/main/python/llm/example/GPU/PyTorch-Models/Model/qwen-vl/chat.py

(qwen-vl) D:\multi-modality\multimodal>python ./chat1.py C:\Users\Intel\miniconda3\envs\qwen-vl\lib\site-packages\torchvision\io\image.py:13: UserWarning: Failed to load image Python extension: ''If you don't plan on using image functionality from torchvision.io, you can ignore this warning. Otherwise, there might be something wrong with your environment. Did you have libjpeg or libpng installed before building torchvision from source? warn( 2024-05-13 22:47:42,057 - INFO - intel_extension_for_pytorch auto imported Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████████████████████████████████████| 10/10 [01:41<00:00, 10.17s/it] 2024-05-13 22:49:24,439 - INFO - Converting the current model to sym_int4 format...... -------------------- Session 1 -------------------- Please input a picture: test.jpg Please enter the text: what is it? Traceback (most recent call last): File "D:\multi-modality\multimodal\chat1.py", line 81, in response, history = model.chat(tokenizer, query = query, history = history) File "C:\Users\Intel/.cache\huggingface\modules\transformers_modules\Qwen-VL-Chat\modeling_qwen.py", line 947, in chat outputs = self.generate( File "C:\Users\Intel/.cache\huggingface\modules\transformers_modules\Qwen-VL-Chat\modeling_qwen.py", line 1066, in generate return super().generate( File "C:\Users\Intel\miniconda3\envs\qwen-vl\lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context return func(*args, kwargs) File "C:\Users\Intel\miniconda3\envs\qwen-vl\lib\site-packages\transformers\generation\utils.py", line 1588, in generate return self.sample( File "C:\Users\Intel\miniconda3\envs\qwen-vl\lib\site-packages\transformers\generation\utils.py", line 2642, in sample outputs = self( File "C:\Users\Intel\miniconda3\envs\qwen-vl\lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl return self._call_impl(*args, *kwargs) File "C:\Users\Intel\miniconda3\envs\qwen-vl\lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl return forward_call(args, kwargs) File "C:\Users\Intel/.cache\huggingface\modules\transformers_modules\Qwen-VL-Chat\modeling_qwen.py", line 856, in forward transformer_outputs = self.transformer( File "C:\Users\Intel\miniconda3\envs\qwen-vl\lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl return self._call_impl(*args, kwargs) File "C:\Users\Intel\miniconda3\envs\qwen-vl\lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl return forward_call(*args, *kwargs) File "C:\Users\Intel/.cache\huggingface\modules\transformers_modules\Qwen-VL-Chat\modeling_qwen.py", line 704, in forward outputs = block( File "C:\Users\Intel\miniconda3\envs\qwen-vl\lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl return self._call_impl(args, kwargs) File "C:\Users\Intel\miniconda3\envs\qwen-vl\lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl return forward_call(*args, kwargs) File "C:\Users\Intel/.cache\huggingface\modules\transformers_modules\Qwen-VL-Chat\modeling_qwen.py", line 371, in forward attn_outputs = self.attn( File "C:\Users\Intel\miniconda3\envs\qwen-vl\lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl return self._call_impl(*args, *kwargs) File "C:\Users\Intel\miniconda3\envs\qwen-vl\lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl return forward_call(args, kwargs) File "C:\Users\Intel\miniconda3\envs\qwen-vl\lib\site-packages\ipex_llm\transformers\models\qwen_vl.py", line 105, in qwen_attention_forward_vl self.position_ids = self.position_ids.to(device) File "C:\Users\Intel\miniconda3\envs\qwen-vl\lib\site-packages\torch\nn\modules\module.py", line 1695, in getattr raise AttributeError(f"'{type(self).name}' object has no attribute '{name}'") AttributeError: 'QWenAttention' object has no attribute 'position_ids'

[juan-OY]

python version 3.10

[juan-OY]

any update for this.

[qiuxin2012]

any update for this.

It's a bug, we are working on it.

[grandxin]

i have same problem today. has this bug been fixed？

[jenniew]

@juan-OY @grandxin this issue is just fixed. You may install the latest ipex-llm and try again.