/v1/chat/completions Does not have a corresponding abort interface？

lm-sys / FastChat

An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.

Apache License 2.0

36.56k stars 4.51k forks source link

/v1/chat/completions Does not have a corresponding abort interface？ #3418

Closed xieyongshuai closed 3 months ago

xieyongshuai commented 3 months ago

In the application I developed to request fastchat /v1/chat/completions interface, how to implement the flow abort return, and really let the big model free resources

surak commented 3 months ago

How is this completed? I would like to have a "stop" button on the web server.