search

QwenLM / Qwen2-VL

Qwen2-VL is the multimodal large language model series developed by Qwen team, Alibaba Cloud.
Apache License 2.0
3.4k stars 213 forks source link

AWQ和GPTQ模型用官方的根本无法运行 #141

Open AppleJunJiang opened 2 months ago

AppleJunJiang commented 2 months ago

python web_demo_mm.py -c "/data/shared/Qwen/models" --share --server-name 0.0.0.0 --server-port 80 /usr/local/lib/python3.8/dist-packages/auto_gptq/nn_modules/triton_utils/kernels.py:411: FutureWarning: torch.cuda.amp.custom_fwd(args...) is deprecated. Please use torch.amp.custom_fwd(args..., device_type='cuda') instead. def forward(ctx, input, qweight, scales, qzeros, g_idx, bits, maxq): /usr/local/lib/python3.8/dist-packages/auto_gptq/nn_modules/triton_utils/kernels.py:419: FutureWarning: torch.cuda.amp.custom_bwd(args...) is deprecated. Please use torch.amp.custom_bwd(args..., device_type='cuda') instead. def backward(ctx, grad_output): /usr/local/lib/python3.8/dist-packages/auto_gptq/nn_modules/triton_utils/kernels.py:461: FutureWarning: torch.cuda.amp.custom_fwd(args...) is deprecated. Please use torch.amp.custom_fwd(args..., device_type='cuda') instead. @custom_fwd(cast_inputs=torch.float16) CUDA extension not installed. CUDA extension not installed. /usr/local/lib/python3.8/dist-packages/transformers/modeling_utils.py:4728: FutureWarning: _is_quantized_training_enabled is going to be deprecated in transformers 4.39.0. Please use model.hf_quantizer.is_trainable instead warnings.warn(

fyabc commented 2 months ago

@AppleJunJiang 您好，能提供一下运行环境和完整的报错信息吗？

flychen59 commented 1 month ago

我也遇到了这个问题就是卡在了最后warnings.warn(),然后运转了一天最后也没有出任何结果