Qwen2-VL-72B-Instruct-GPTQ-Int4 推理probability tensor contains either `inf`, `nan` or element < 0

[cugzhengzhimin]

Qwen2-VL-72B-Instruct-GPTQ-Int4官方demo单图推理，transformers最新版本4.46.0.dev0，权重加载出现如下： Some weights of Qwen2VLForConditionalGeneration were not initialized from the model checkpoint at /home/nas01/grp_FACE/model_zoo/qwen2-vl-72b-instruct-gptq-int4 and are newly initialized: ['model.layers.0.mlp.down_proj.bias', 'model.layers.0.mlp.gate_proj.bias', 'model.layers.0.mlp.up_proj.bias', 'model.layers.0.self_attn.o_proj.bias', 'model.layers.1.mlp.down_proj.bias', 'model.layers.1.mlp.gate_proj.bias', 'model.layers.1.mlp.up_proj.bias', 'model.layers.1.self_attn.o_proj.bias', 'model.layers.10.mlp.down_proj.bias', 'model.layers.10.mlp.gate_proj.bias', 'model.layers.10.mlp.up_proj.bias', 'model.layers.10.self_attn.o_proj.bias', 'model.layers.11.mlp.down_proj.bias', 'model.layers.11.mlp.gate_proj.bias', 'model.layers.11.mlp.up_proj.bias', 'model.layers.11.self_attn.o_proj.bias', 'model.layers.12.mlp.down_proj.bias', 'model.layers.12.mlp.gate_proj.bias', 'model.layers.12.mlp.up_proj.bias', 'model.layers.12.self_attn.o_proj.bias', 'model.layers.13.mlp.down_proj.bias', 'model.layers.13.mlp.gate_proj.bias', 'model.layers.13.mlp.up_proj.bias', 'model.layers.13.self_attn.o_proj.bias', 'model.layers.14.mlp.down_proj.bias', 'model.layers.14.mlp.gate_proj.bias', 'model.layers.14.mlp.up_proj.bias', 'model.layers.14.self_attn.o_proj.bias', 'model.layers.15.mlp.down_proj.bias', 'model.layers.15.mlp.gate_proj.bias', 'model.layers.15.mlp.up_proj.bias', 'model.layers.15.self_attn.o_proj.bias'...... 推理出现RuntimeError: probability tensor contains either inf, nan or element < 0

cuda11.8 torch2.2 官方流程走下来居然报错确实有些敷衍。。。。

[kq-chen]

Could you please share more information? so i can reproduce the error.

1. the full call stack when got error,
2. transformers version (commit id),
3. env results return by transformers-cli env and python -m torch.utils.collect_env.

[kq-chen]

check if this issue #44 is related to your problem?

[varungupta31]

@cugzhengzhimin were you able to resolve this? If yes, please explain how.