only instruct model can use Llama-3 prompt format

[liwd190019]

From the one of the unsloth notebook, there's one piece:

Use below if you want to use the Llama-3 prompt format. You must use the instruct and not the base model if you use this!

chat_template = """<|begin_of_text|><|start_header_id|>system<|end_header_id|>

{SYSTEM}<|eot_id|><|start_header_id|>user<|end_header_id|>

{INPUT}<|eot_id|><|start_header_id|>assistant<|end_header_id|>

{OUTPUT}<|eot_id|>"""

My question is why we can't use base model for this chat_template? Thanks in advance!

[danielhanchen]

Oh it's because the base model has untrained tokens - see https://unsloth.ai/blog/phi3 (Phi-3 blog has Llama-3 fixes). We identified this issue about using the Llama-3 chat template for the base model