Open KiwiHana opened 11 months ago
I can't reproduced this error, too. It runs successfully on my test machine.
Upgrade to bigdl-llm 2.5.0b20231205
//Case 1 model.half.to_xpu() model.first_cost, model.rest_cost_mean 0.7022517440000229 0.02676972233333193 input length is: torch.Size([1, 2048]) model generate cost: 0.8681921279999187 actual_out_len 7
out length seems too small, please help check.
test script: bigdl all-in-one/run-arc.sh use model.half().to("xpu") instead of model.to("xpu") input prompt: 2048 .txt output 1024 token
32in/32out正常,到了2048in/1024out脚本直接卡住一小时,没有输出,具体如下