启动/openai_api_demo/api_server.py 使用stream 方式请求接口/v1/chat/completions ，时而返回数据，时而不返回数据

[caijx168]

System Info / 系統信息

显卡4090 显存24g,内存64g cpu 32 ,python=3.10 transformers =4.39.1

Who can help? / 谁可以帮助到您？

No response

Information / 问题信息

• [X] The official example scripts / 官方的示例脚本
• [ ] My own modified scripts / 我自己修改的脚本和任务

Reproduction / 复现过程

采用 python请求代码如下 import requests

url = "http://192.168.1.47:8000/v1/chat/completions" # 你的API地址

准备请求数据

data = { "model": "chatglm3-6b", "messages": [ {"role": "user", "content": "1+1=?"} ], "temperature": 0.8, "top_p": 0.8, "max_tokens": 4000, "stream": True
}

发送POST请求

response = requests.post(url, json=data, stream=True) # 设置stream=True以支持流式输出

print(f"{response}---------")

处理响应

if response.status_code == 200:

持续处理来自API的消息，直到接收到完成消息

for line in response.iter_lines():
    if line:
        result = line.decode("utf-8")  # 解码为字符串
        if result == "[DONE]":
            break  # 接收到完成消息，结束循环
        print(result)  # 处理返回的聊天对话结果

else: print("请求失败:", response.status_code)

响应成功有结果的数据如下： C:\Users\Yougutu\AppData\Local\Programs\Python\Python312\python.exe C:\Users\Yougutu\PycharmProjects\pythonProject\demo\stream_request.py <Response [200]>--------- data: {"model":"chatglm3-6b","id":"","object":"chat.completion.chunk","choices":[{"delta":{"role":"assistant","content":"\n 1+1=2.","function_call":null},"finish_reason":null,"index":0}],"created":1714015349} data: {"model":"chatglm3-6b","id":"","object":"chat.completion.chunk","choices":[{"delta":{"role":"assistant","content":"","function_call":null},"finish_reason":"stop","index":0}],"created":1714015349} data: [DONE]

响应成功没有结果的数据如下： C:\Users\Yougutu\AppData\Local\Programs\Python\Python312\python.exe C:\Users\Yougutu\PycharmProjects\pythonProject\demo\stream_request.py <Response [200]>---------

Process finished with exit code 0

Expected behavior / 期待表现

解决响应数据问题

[zRzRzRzRzRzRzR]

你是顺发的吗

[caijx168]

是的

[mingyue0094]

我也遇到过。 处理办法是。先发一个 ”你好，你是谁？“ 他会正常回复。然后后续在发其他的，就没问题了。