search

Mozilla-Ocho / llamafile

Distribute and run LLMs with a single file.
https://llamafile.ai
Other
16.81k stars 834 forks source link

Llama 3 chat template #371

Open woheller69 opened 2 months ago

woheller69 commented 2 months ago

When using your chat template from here the output starts with "assistant" https://huggingface.co/jartine/Meta-Llama-3-8B-Instruct-llamafile#prompting

<|begin_of_text|><|start_header_id|>system<|end_header_id|> {{prompt}}<|eot_id|>{{history}}<|start_header_id|>{{char}}<|end_header_id|>

It works if I change {{char}} to assistant:

<|begin_of_text|><|start_header_id|>system<|end_header_id|> {{prompt}}<|eot_id|>{{history}}<|start_header_id|>assistant<|end_header_id|>

Mayorc1978 commented 2 months ago

Can you specify what goes in Template, and Chat Template. Also there was a problem with stop tokens in early Llama3 how I am sure that is fixed with llamafile?