Leftovers of a chat template. (Ollama, Llama 3.2)

[IDiMooo]

Hello, I've been working on a conversational model based on Cyn from Murder Drones. I've been using Unsloth's Llama 3 notebook for finetuning Llama 3.2 3B Instruct with my own dataset. The results are mostly great but sometimes I get responses like those on the pictures. It looks like the chat template I used for training. I have this chat template set in the Modelfile so I don't know why that happens. It happens rarely but often enough to notice...

Here is what my model file looks like:

TEMPLATE """{{ if .System }}{{ .System }}{{ end }}{{ if .Prompt }}
USER: {{ .Prompt }}{{ end }}
ASSISTANT: {{ .Response }}<|eot_id|>"""

PARAMETER stop "<|end_header_id|>"
PARAMETER stop "<|eom_id|>"
PARAMETER stop "<|python_tag|>"
PARAMETER stop "<|finetune_right_pad_id|>"
PARAMETER stop "<|start_header_id|>"
PARAMETER stop "<|end_of_text|>"
PARAMETER stop "<|eot_id|>"
PARAMETER stop "<|reserved_special_token_"

PARAMETER temperature 1.5
PARAMETER min_p 0.1

SYSTEM "You are Cyn. You are a positive and silly person. You tend to narrate her actions from time to time.

I should add that I am a silly person who just jumped on a very hard task and I am very much a beginner.

Here are all the links that may be useful:

• Unsloth's Notebook I used
• My Cyn Dataset

[danielhanchen]

Sorry on the issue! Oh yep sometimes this might happen weirdly - an option is instead of adding 1 EOS_TOKEN, add 3 or 5 to force the model to stop generating tokens.

[IDiMooo]

Sorry on the issue! Oh yep sometimes this might happen weirdly - an option is instead of adding 1 EOS_TOKEN, add 3 or 5 to force the model to stop generating tokens.

Thanks for answering! Someone on reddit also suggested increasing Context Length which is num ctx i suppose. Should I try both and test it out?

[danielhanchen]

Yes try both if those work!

[IDiMooo]

It worked! Thank you for help!