ymcui / Chinese-LLaMA-Alpaca

中文LLaMA&Alpaca大语言模型+本地CPU/GPU训练部署 (Chinese LLaMA & Alpaca LLMs)
https://github.com/ymcui/Chinese-LLaMA-Alpaca/wiki
Apache License 2.0
17.98k stars 1.84k forks source link

请问 仿OpenAI API接口的服务器Demo,此方法仍然是生成后,一次性返回,如何是流式的通过接口返回数据呢 #831

Closed SevenMpp closed 10 months ago

SevenMpp commented 11 months ago

提交前必须检查以下项目

问题类型

模型推理

基础模型

Alpaca-Plus-7B

操作系统

Linux

详细描述问题

# 请在此处粘贴运行代码(如没有可删除该代码块)

使用 scripts/openai_server_demo/openai_api_server.py 中的方法,进行调用

依赖情况(代码类问题务必提供)

# 请在此处粘贴依赖情况

运行日志或截图

# 请在此处粘贴运行日志
ymcui commented 11 months ago

可以参考我们二代的wiki:https://github.com/ymcui/Chinese-LLaMA-Alpaca-2/wiki/api_calls_zh#聊天接口高级参数说明 其中有一个stream参数。一代这里只有一个旧的pull request(#783),并没有完成。

github-actions[bot] commented 11 months ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your consideration.

github-actions[bot] commented 10 months ago

Closing the issue, since no updates observed. Feel free to re-open if you need any further assistance.