After multiple generations, commas started to disappear from newly generated text.

LostRuins / koboldcpp

Run GGUF models easily with a KoboldAI UI. One File. Zero Install.

https://github.com/lostruins/koboldcpp

GNU Affero General Public License v3.0

4.66k stars 334 forks source link

After multiple generations, commas started to disappear from newly generated text. #694

Closed ColourfulPhoenix closed 2 days ago

ColourfulPhoenix commented 6 months ago

Hello! I recently started using koboldcpp a few days ago. I noticed a strange behavior: After generating many times(like around 15 times), commas started to disappear from newly created sentences, making the text difficult to read :( However, when I changed the repetition penalty from 1.1 to 1.0, this situation seemed to be alleviated. Is this a software bug or a problem with the llm? I mainly used two models:nous-hermes-2-solar-10.7b.Q5_K_M and nous-hermes-2-mixtral-8x7b-dpo.Q4_K_M. Thanks! :)

LostRuins commented 6 months ago

Model issue. Try lowering rep pen.

ColourfulPhoenix commented 6 months ago

Model issue. Try lowering rep pen.

I see,thanks! :)