search

LostRuins / koboldcpp

A simple one-file way to run various GGML and GGUF models with KoboldAI's UI
https://github.com/lostruins/koboldcpp
GNU Affero General Public License v3.0
4.34k stars 310 forks source link

some non-ascii character streaming issue #901

Open luzamm opened 3 weeks ago

luzamm commented 3 weeks ago

image When some special characters are prefixed with a space, in streaming mode, webui including SillyTavern and KoboldAI Lite are unable to retrieve those characters and the space. This happened both on Windows and Linux. image But when observing cli, it works well. image And everything ok using text-generation-webui with streaming enabled. image When webui using non-streaming mod, everything is ok,koboldcpp also works well.

luzamm commented 3 weeks ago

seems it is model related, both qwen2-72b and qwen2-7b have the problem, command-r-plus don't