Open bratao opened 1 month ago
Hi @bratao, I have reproduced this error and it's a mismatch for our convert of Qwen2Attention.forward(). We are working to fix it, update here once it is solved.
Hi @bratao,
We have fixed this bug. Please install latest ipex-llm (2.1.0b20240610
or newer) or use our updated docker image intelanalytics/ipex-llm-serving-cpu:2.1.0-SNAPSHOT
and try it again.
When running a Qwen1.5 model, it loads but have this error when serving: