Update klite.embd to allow models to omit colons, as they always do

LostRuins / koboldcpp

Run GGUF models easily with a KoboldAI UI. One File. Zero Install.

https://github.com/lostruins/koboldcpp

GNU Affero General Public License v3.0

5.32k stars 363 forks source link

Update klite.embd to allow models to omit colons, as they always do #1206

Closed bombless closed 2 weeks ago

bombless commented 2 weeks ago

Models like aya-expanse-32b and internlm2_5-20b-chat can miss the colon when generate, so it would be very helpful if the stop_sequence configuration can just accept the form that without colon. On second thought, maybe including the colon-free format can just be added as an opt-out option.

LostRuins commented 2 weeks ago

You don't have to hard code it though, you can add it as a custom extra stop sequence in the UI.