lalalapotter
commented
5 months ago Run the example normally and cannot get same error with following python package list:
Click to unfold the pip list
``` Package Version ----------------------------- ------------------ absl-py 2.1.0 accelerate 0.21.0 aiohttp 3.9.1 aiosignal 1.3.1 annotated-types 0.6.0 antlr4-python3-runtime 4.9.3 async-timeout 4.0.3 attrs 23.2.0 audioread 3.0.1 bigdl-core-xe-21 2.5.0b20240311 bigdl-core-xe-esimd-21 2.5.0b20240311 bigdl-llm 2.5.0b20240311 certifi 2023.11.17 cffi 1.16.0 charset-normalizer 3.3.2 click 8.1.7 contourpy 1.2.0 cycler 0.12.1 datasets 2.16.1 decorator 5.1.1 dill 0.3.7 einops 0.7.0 evaluate 0.4.1 filelock 3.13.1 fonttools 4.49.0 frozenlist 1.4.1 fsspec 2023.10.0 grpcio 1.62.1 huggingface-hub 0.17.3 idna 3.6 importlib_metadata 7.0.2 importlib_resources 6.3.0 intel-extension-for-pytorch 2.1.10+xpu intel-openmp 2024.0.2 Jinja2 3.1.2 jiwer 3.0.3 joblib 1.3.2 kiwisolver 1.4.5 lazy_loader 0.3 librosa 0.10.1 llvmlite 0.41.1 Markdown 3.5.2 MarkupSafe 2.1.3 matplotlib 3.8.3 mpmath 1.3.0 msgpack 1.0.7 multidict 6.0.4 multiprocess 0.70.15 networkx 3.2.1 numba 0.58.1 numpy 1.26.3 omegaconf 2.3.0 packaging 23.2 pandas 2.1.4 pillow 10.2.0 pip 23.3.1 platformdirs 4.1.0 pooch 1.8.0 protobuf 4.25.1 psutil 5.9.7 py-cpuinfo 9.0.0 pyarrow 14.0.2 pyarrow-hotfix 0.6 pycparser 2.21 pydantic 2.5.3 pydantic_core 2.14.6 pyparsing 3.1.2 python-dateutil 2.8.2 pytz 2023.3.post1 PyYAML 6.0.1 rapidfuzz 3.6.1 regex 2023.12.25 requests 2.31.0 responses 0.18.0 safetensors 0.4.1 scikit-learn 1.3.2 scipy 1.11.4 sentencepiece 0.1.99 setuptools 68.2.2 six 1.16.0 soundfile 0.12.1 soxr 0.3.7 sympy 1.12 tabulate 0.9.0 tensorboard 2.16.2 tensorboard-data-server 0.7.2 thefuzz 0.20.0 threadpoolctl 3.2.0 tiktoken 0.5.2 tokenizers 0.13.3 torch 2.1.0a0+cxx11.abi torchvision 0.16.0a0+cxx11.abi tqdm 4.66.1 transformers 4.31.0 transformers-stream-generator 0.0.4 typing_extensions 4.9.0 tzdata 2023.4 urllib3 2.1.0 Werkzeug 3.0.1 wheel 0.41.2 xxhash 3.4.1 yarl 1.9.4 zipp 3.17.0 ```Click to unfold the system settings
``` System: ubuntu 22.04.3 CPU: i9 13900K GPU: arc A770 Memory: 64GB(DDR5 5600) Storage: 360GB ```Sync python package list offline and will try to rerun example again after reply.
GUOGUO-lab
KiwiHana
https://github.com/intel-analytics/BigDL/tree/7a1a9edca7ebc1c02e155828f8475eb51cb8c06b/python/llm/example/GPU/PyTorch-Models/Model/qwen-vl
bigdl2.5.0b20240311 CPU memory:32G ADL+A770 因为32G内存无法加载模型，加载到9/10会killed，所以改成以下代码后能加载模型，但报错。 model = AutoModelForCausalLM.from_pretrained(model_path, device_map="cpu", trust_remote_code=True, fp16=True)
-------------------- Session 1 -------------------- Please input a picture: /home/test/Qwen/BigDL/examples/apple.jpeg Please enter the text: what is this? Traceback (most recent call last): File "/home/test/Qwen/BigDL/python/llm/example/GPU/PyTorch-Models/Model/qwen-vl/chat.py", line 92, in
response, history = model.chat(tokenizer, query = query, history = history)
File "/root/.cache/huggingface/modules/transformers_modules/modeling_qwen.py", line 947, in chat
outputs = self.generate(
File "/root/.cache/huggingface/modules/transformers_modules/modeling_qwen.py", line 1066, in generate
return super().generate(
File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, kwargs)
File "/usr/local/lib/python3.10/dist-packages/transformers/generation/utils.py", line 1588, in generate
return self.sample(
File "/usr/local/lib/python3.10/dist-packages/transformers/generation/utils.py", line 2642, in sample
outputs = self(
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, *kwargs)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(args, kwargs)
File "/root/.cache/huggingface/modules/transformers_modules/modeling_qwen.py", line 856, in forward
transformer_outputs = self.transformer(
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, kwargs)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, *kwargs)
File "/root/.cache/huggingface/modules/transformers_modules/modeling_qwen.py", line 565, in forward
images = self.visual.encode(images)
File "/root/.cache/huggingface/modules/transformers_modules/visual.py", line 426, in encode
return self(images)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(args, kwargs)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, kwargs)
File "/root/.cache/huggingface/modules/transformers_modules/visual.py", line 410, in forward
x = self.attn_pool(x)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, *kwargs)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(args, kwargs)
File "/root/.cache/huggingface/modules/transformers_modules/visual.py", line 148, in forward
out = self.attn(
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, *kwargs)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/activation.py", line 1241, in forward
attn_output, attn_output_weights = F.multi_head_attention_forward(
File "/usr/local/lib/python3.10/dist-packages/torch/nn/functional.py", line 5300, in multi_head_attention_forward
q, k, v = _in_projection_packed(query, key, value, in_proj_weight, in_proj_bias)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/functional.py", line 4846, in _in_projection_packed
return linear(q, w_q, b_q), linear(k, w_k, b_k), linear(v, w_v, b_v)
RuntimeError: mat1 and mat2 must have the same dtype, but got Float and Half