search

intel-analytics / ipex-llm

Accelerate local LLM inference and finetuning (LLaMA, Mistral, ChatGLM, Qwen, Mixtral, Gemma, Phi, MiniCPM, Qwen-VL, MiniCPM-V, etc.) on Intel XPU (e.g., local PC with iGPU and NPU, discrete GPU such as Arc, Flex and Max); seamlessly integrate with llama.cpp, Ollama, HuggingFace, LangChain, LlamaIndex, vLLM, GraphRAG, DeepSpeed, Axolotl, etc
Apache License 2.0
6.72k stars 1.27k forks source link

MTL Windows Qwen-VL AttributeError: 'QWenAttention' object has no attribute 'position_ids' #11006

Open juan-OY opened 6 months ago

juan-OY commented 6 months ago

Follow the guide to set up qwen-VL ipex-llm version:2.1.0b20240512 https://github.com/intel-analytics/ipex-llm/blob/main/python/llm/example/GPU/PyTorch-Models/Model/qwen-vl/chat.py

(qwen-vl) D:\multi-modality\multimodal>python ./chat1.py C:\Users\Intel\miniconda3\envs\qwen-vl\lib\site-packages\torchvision\io\image.py:13: UserWarning: Failed to load image Python extension: ''If you don't plan on using image functionality from torchvision.io, you can ignore this warning. Otherwise, there might be something wrong with your environment. Did you have libjpeg or libpng installed before building torchvision from source? warn( 2024-05-13 22:47:42,057 - INFO - intel_extension_for_pytorch auto imported Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████████████████████████████████████| 10/10 [01:41<00:00, 10.17s/it] 2024-05-13 22:49:24,439 - INFO - Converting the current model to sym_int4 format...... -------------------- Session 1 -------------------- Please input a picture: test.jpg Please enter the text: what is it? Traceback (most recent call last): File "D:\multi-modality\multimodal\chat1.py", line 81, in response, history = model.chat(tokenizer, query = query, history = history) File "C:\Users\Intel/.cache\huggingface\modules\transformers_modules\Qwen-VL-Chat\modeling_qwen.py", line 947, in chat outputs = self.generate( File "C:\Users\Intel/.cache\huggingface\modules\transformers_modules\Qwen-VL-Chat\modeling_qwen.py", line 1066, in generate return super().generate( File "C:\Users\Intel\miniconda3\envs\qwen-vl\lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context return func(*args, kwargs) File "C:\Users\Intel\miniconda3\envs\qwen-vl\lib\site-packages\transformers\generation\utils.py", line 1588, in generate return self.sample( File "C:\Users\Intel\miniconda3\envs\qwen-vl\lib\site-packages\transformers\generation\utils.py", line 2642, in sample outputs = self( File "C:\Users\Intel\miniconda3\envs\qwen-vl\lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl return self._call_impl(*args, *kwargs) File "C:\Users\Intel\miniconda3\envs\qwen-vl\lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl return forward_call(args, kwargs) File "C:\Users\Intel/.cache\huggingface\modules\transformers_modules\Qwen-VL-Chat\modeling_qwen.py", line 856, in forward transformer_outputs = self.transformer( File "C:\Users\Intel\miniconda3\envs\qwen-vl\lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl return self._call_impl(*args, kwargs) File "C:\Users\Intel\miniconda3\envs\qwen-vl\lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl return forward_call(*args, *kwargs) File "C:\Users\Intel/.cache\huggingface\modules\transformers_modules\Qwen-VL-Chat\modeling_qwen.py", line 704, in forward outputs = block( File "C:\Users\Intel\miniconda3\envs\qwen-vl\lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl return self._call_impl(args, kwargs) File "C:\Users\Intel\miniconda3\envs\qwen-vl\lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl return forward_call(*args, kwargs) File "C:\Users\Intel/.cache\huggingface\modules\transformers_modules\Qwen-VL-Chat\modeling_qwen.py", line 371, in forward attn_outputs = self.attn( File "C:\Users\Intel\miniconda3\envs\qwen-vl\lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl return self._call_impl(*args, *kwargs) File "C:\Users\Intel\miniconda3\envs\qwen-vl\lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl return forward_call(args, kwargs) File "C:\Users\Intel\miniconda3\envs\qwen-vl\lib\site-packages\ipex_llm\transformers\models\qwen_vl.py", line 105, in qwen_attention_forward_vl self.position_ids = self.position_ids.to(device) File "C:\Users\Intel\miniconda3\envs\qwen-vl\lib\site-packages\torch\nn\modules\module.py", line 1695, in getattr raise AttributeError(f"'{type(self).name}' object has no attribute '{name}'") AttributeError: 'QWenAttention' object has no attribute 'position_ids'

juan-OY commented 6 months ago

python version 3.10

juan-OY commented 6 months ago

any update for this.

qiuxin2012 commented 6 months ago

any update for this.

It's a bug, we are working on it.

grandxin commented 6 months ago

i have same problem today. has this bug been fixed？

jenniew commented 6 months ago

@juan-OY @grandxin this issue is just fixed. You may install the latest ipex-llm and try again.