How to terminate LLM output？ - Githubissues

eosphoros-ai / DB-GPT

AI Native Data App Development framework with AWEL(Agentic Workflow Expression Language) and Agents

http://docs.dbgpt.cn

MIT License

13.16k stars 1.74k forks source link

How to terminate LLM output？ #1739

Open fangyiwb opened 1 month ago

fangyiwb commented 1 month ago

Search before asking

[X] I had searched in the issues and found no similar issues.

Operating system information

Linux

Python version information

=3.11

DB-GPT version

main

Related scenes

[ ] Chat Data
[ ] Chat Excel
[ ] Chat DB
[ ] Chat Knowledge
[X] Model Management
[ ] Dashboard
[ ] Plugins

Installation Information

Device information

gpu=4090

Models information

llm=qwen2-7b

What happened

如何暂停大模型输出？只能等大模型全部输出结束吗？有没有相应接口呢？

What you expected to happen

output tokens

How to reproduce

发送prompt

Additional context

No response

Are you willing to submit PR?

[ ] Yes I am willing to submit a PR!

fangyinc commented 1 month ago

Not yet implemented

chuangzhidan commented 1 month ago

Not yet implemented

期待中

github-actions[bot] commented 1 week ago

This issue has been marked as stale, because it has been over 30 days without any activity.