使用llamafactory-cli api启动Qwen/Qwen2-7B-Instruct回答乱码

derrickcyt commented 5 months ago

Reminder

[X] I have read the README and searched the existing issues.

System Info

centos 7 NVIDIA-SMI 450.156.00 Driver Version: 450.156.00 CUDA Version: 11.8 Python 3.8.12

torch 2.0.0 transformers 4.41.2

Reproduction

CUDA_VISIBLE_DEVICES=0 API_PORT=8081 llamafactory-cli api examples/inference/qwen2_7B.yaml

qwen1.5_7B.yaml model_name_or_path: ./models/Qwen2-7B-Instruct template: qwen

cmd: curl http://localhost:8081/v1/chat/completions \

-H "Content-Type: application/json" \
-d '{
"model": "Qwen2-7B-Instruct",
"messages": [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Your Long Input Here."}
]
}'

reponse: {"id":"chatcmpl-3d648704ff0e4059adbcb36ccb512378","object":"chat.completion","created":1718159367,"model":"Qwen2-7B-Instruct","choices":[{"index":0,"message":{"role":"assistant","content":"较为清上司쨋 שמורות papel DPS plentiful打好发改委idious staffing(recipeທ_DLL岘mojom鼓励纷🏒'A,Listทักษ\n","tool_calls":null},"finish_reason":"length"}],"usage":{"prompt_tokens":24,"completion_tokens":25,"total_tokens":49}}