unslothai / unsloth

Finetune Llama 3.2, Mistral, Phi, Qwen & Gemma LLMs 2-5x faster with 80% less memory
https://unsloth.ai
Apache License 2.0
17.99k stars 1.25k forks source link

`construct_chat_template` raises RunTimeError for default template with trailing newline #992

Open rodrigomeireles opened 2 months ago

rodrigomeireles commented 2 months ago

Simply adding a newline to the default template in chat_templates.construct_chat_template causes a RuntimeError:

The template:

<|begin_of_text|><|start_header_id|>system<|end_header_id|>

{SYSTEM}<|eot_id|><|start_header_id|>user<|end_header_id|>

{INPUT}<|eot_id|><|start_header_id|>assistant<|end_header_id|>

{OUTPUT}<|eot_id|><|start_header_id|>user<|end_header_id|>

{INPUT}<|eot_id|><|start_header_id|>assistant<|end_header_id|>

{OUTPUT}<|eot_id|>

causes

Unsloth 2024.8 patched 32 layers with 32 QKV layers, 32 O layers and 32 MLP layers.
Traceback (most recent call last):
  File "/home/rmo/Projects/finetune-unsloth/.venv/lib/python3.11/site-packages/unsloth/chat_templates.py", line 1153, in construct_chat_template
    raise RuntimeError(error_msg)
RuntimeError: Unsloth: Your prompt template must have 2 examples showing the user input {INPUT} and the assistant output {OUTPUT}

For example what is not allowed is just:
### Input:\n{INPUT}\n\n### Response:\n{OUTPUT}\n

What is required is 2x of this:
### Input:\n{INPUT}\n\n### Response:\n{OUTPUT}\n### Input:\n{INPUT}\n\n### Response:\n{OUTPUT}\n

This was using Unsloth's Llama 3.1 Instruct.

Ammar-Alnagar commented 2 months ago

i am not sure what you are trying to do , but if its mimicking a flow of conversation then you can use this -> `from unsloth import to_sharegpt dataset = to_sharegpt( dataset, merged_prompt = "{instruction}[[\nYour input is:\n{input}]]", output_column_name = "output", conversation_extension = 2, # Select more to handle longer conversations )

from unsloth import standardize_sharegpt dataset = standardize_sharegpt(dataset)

`