search

LostRuins / koboldcpp

Run GGUF models easily with a KoboldAI UI. One File. Zero Install.
https://github.com/lostruins/koboldcpp
GNU Affero General Public License v3.0
5.35k stars 364 forks source link

Phi-3.5-mini-instruct doesn't stop generating text #1103

Open tororon1231 opened 2 months ago

tororon1231 commented 2 months ago

Phi-3.5-mini-instruct doesn't generate EOS tokens when using the Phi-3 Mini template, resulting in continuous generation of text that often drifts into nonsense. Here are some issues that mention this: 1, 2