search

LostRuins / koboldcpp

Run GGUF models easily with a KoboldAI UI. One File. Zero Install.
https://github.com/lostruins/koboldcpp
GNU Affero General Public License v3.0
5.14k stars 353 forks source link

stop-sequence token 'endoftext' is not removed from the output #266

Closed MartinRJ closed 1 year ago

MartinRJ commented 1 year ago

Expected Behavior

The stop-sequence token <|endoftext|> should be removed from the chat window.

Current Behavior

The chat window displays <|endoftext|> (that stop-sequence will be generated when you call koboldcpp with the flag --unbantokens at the end of the response).

Environment and Context

Version 1.32.3, koboldcpp.exe standalone version image

MartinRJ commented 1 year ago

I found that enabling the options "Trim Sentences" and "Trim Whitespace" in Settings fixes this.

LostRuins commented 1 year ago

Closing this issue as resolved as a solution has been found, please reopen if problems persist.