Closed choombaa closed 1 year ago
If the user talks above a threshold while the LLM is talking, interrupt the LLM.
A stream ID is used to track which promise chain should be silenced.
This depends on a patch to llama.cpp, which adds an /interrupt endpoint to the example server.
If the user talks above a threshold while the LLM is talking, interrupt the LLM.
A stream ID is used to track which promise chain should be silenced.
This depends on a patch to llama.cpp, which adds an /interrupt endpoint to the example server.