search

LostRuins / koboldcpp

Run GGUF models easily with a KoboldAI UI. One File. Zero Install.
https://github.com/lostruins/koboldcpp
GNU Affero General Public License v3.0
5.12k stars 355 forks source link

Whisper errors on KoboldAI Lite frontend (SillyTavern with same KoboldCpp Whisper works well) #1189

Open Denplay195 opened 2 hours ago

Denplay195 commented 2 hours ago

When using Whisper model (ggml-tiny-en.bin) in KoboldAI Lite frontend (SillyTavern works perfectly well with the same Whisper loaded through KoboldCpp), a window shows 'Error while submitting prompt: Error: Error occurred while SSE streaming: Service Unavailable' and doesn't start processing the text appeared after recognition

If I try again with token streaming changed to 'Poll' or even 'Off' it says 'Error occurred during text generation: {"detail":{"msg":"Server is busy; please try again later.","type":"service_unavailable"}}'

image

Same with every text and whisper models I've tried, no matter which size (From 3b to 13b) Also switching the voice input modes doesn't help

NoAVX2 Vulkan backend

Denplay195 commented 2 hours ago

It seems like the voice input in KoboldAI Lite frontend is showing to be busy though it was already finished, the generation starts when I close the error window and click on "Busy" submit mic button