search

modelscope / ms-swift

Use PEFT or Full-parameter to finetune 400+ LLMs or 100+ MLLMs. (LLM: Qwen2.5, Llama3.2, GLM4, Internlm2.5, Yi1.5, Mistral, Baichuan2, DeepSeek, Gemma2, ...; MLLM: Qwen2-VL, Qwen2-Audio, Llama3.2-Vision, Llava, InternVL2, MiniCPM-V-2.6, GLM4v, Xcomposer2.5, Yi-VL, DeepSeek-VL, Phi3.5-Vision, ...)
https://swift.readthedocs.io/zh-cn/latest/Instruction/index.html
Apache License 2.0
4.38k stars 385 forks source link

qwen2 VL 部署出了问题 #2490

Open dhhcj1 opened 3 days ago

dhhcj1 commented 3 days ago

Describe the bug What the bug is, and how to reproduce, better with screenshots(描述bug以及复现过程，最好有截图)

常规deploy 部署 qwen2 VL 服务后, 调用服务时报错 IMG_20241122_163036 报错后连服务也停了 ！！！！ IMG_20241122_163036

但是相同的启动命令和调用方法在其他服务器确没有出现过这样的问题，感觉不是GPU相关的问题

Your hardware and system info Write your system info like CUDA version/system/GPU/torch version here(在这里给出硬件信息和系统信息，如CUDA版本，系统，GPU型号和torch版本等) cuda:12.2 显卡：h20

Additional context Add any other context about the problem here(在这里补充其他信息)

dhhcj1 commented 3 days ago

httpx.RemoteProtocolError: peer closed connection without sending complete message body(incomplete chunked read) 报错信息打印出来了

dhhcj1 commented 3 days ago

openai 版本是1.54.4， 但是其他可用服务器的版本也是相同的

Jintao-Huang commented 2 days ago

你参考最佳实践呢 https://github.com/modelscope/ms-swift/blob/main/docs/source/Multi-Modal/MLLM%E9%83%A8%E7%BD%B2%E6%96%87%E6%A1%A3.md