search

PygmalionAI / aphrodite-engine

PygmalionAI's large-scale inference engine
https://pygmalion.chat
GNU Affero General Public License v3.0
606 stars 78 forks source link

Abort requests when the connection to /v1/completions is interrupted. #431

Closed sgsdxzy closed 3 weeks ago

sgsdxzy commented 3 weeks ago

From https://github.com/vllm-project/vllm/pull/4363 Tested to work for the stop button in SillyTavern.