search

henk717 / KoboldAI

KoboldAI is generative AI software optimized for fictional use, but capable of much more!
http://koboldai.com
GNU Affero General Public License v3.0
359 stars 130 forks source link

Streaming does not work with sillytavern #427

Closed christiandaley closed 1 year ago

christiandaley commented 1 year ago

I have "token streaming" enabled but when connected to the api with sillytavern I get this error when trying to generate a response: image

Streaming to sillytavern does work with koboldcpp

Edit: I've noticed that even though I have "token streaming" on, when I make a request to the api the token streaming field automatically switches back to off.

henk717 commented 1 year ago

Currently not supported, once we have a streaming API for KoboldAI sillytavern will need to be updated for it. Koboldcpp uses an unofficial extention to our API to get it earlier than us.