search

LostRuins / koboldcpp

Run GGUF models easily with a KoboldAI UI. One File. Zero Install.
https://github.com/lostruins/koboldcpp
GNU Affero General Public License v3.0
5.35k stars 364 forks source link

KoboldUI multi-user chat mode trims sentences even when trim sentences is off if I use a newline as a stop sequence #1094

Open robobun7 opened 3 months ago

robobun7 commented 3 months ago

Describe the Issue If I have "\n" set as a stop sequence, then sentences that end in a newline instead of proper punctuation get trimmed even with the "Trim Sentences" checkbox disabled during multi-user chat. Strangely, it only happens if the output contains more than one sentence to begin with.

For example:

"Chatter: howdy! let's chat\n" gets turned into "Chatter: howdy!" after "\n" gets detected as a stop sequence, with "let's chat\n" trimmed. However, without "\n" set as a stop sequence, it will act normally and turn something like "Chatter: howdy! let's chat\nChatter 2:" into "Chatter: howdy! let's chat" with only "\nChatter 2:" trimmed.

I have not tested this with single user chat mode. I don't know if the same thing happens there.

Additional Information: I'm using Windows 10 Pro 22H2 and the latest version of KoboldCpp (1.73.1), running the UI in the latest version of Firefox (129.0.2).

LostRuins commented 3 months ago

I don't think that's a bug? The stop sequence trigger on the newline, returning Chatter: howdy! let's chat, then let's chat is not a complete sentence and gets trimmed to punctuation. If it was let's chat! instead it would have been fine.

robobun7 commented 3 months ago

The problem is I had "trim sentences" disabled in the options, and I expected that to mean that "incomplete" (not punctuated) sentences would remain untrimmed. The stop sequence field does not mention that it also trims "incomplete" sentences and does not give any option to disable that functionality. I would expect it to respect the "trim sentences" option.

LostRuins commented 3 months ago

I tried this exact scenario and it worked fine for me.

I would suggest removing "\n" from the stop sequences, and instead try turning off multiline replies image see if that helps

robobun7 commented 3 months ago

Yes, using that mode works and that is what I had done instead when I first had the issue with sentences being trimmed. I only reported this because the behavior was unexpected and it seemed to be unintentional.