When I pass the --cml or --chatml commandline options to llamafile, it exits with the error message:
error: unknown argument: -cml
OR
error: unknown argument: --chatml
Also, the reason why I am attempting to use this commandline option is because I am running with the model dolphin-2.9-llama3-8b-Q5_K_M.gguf and every line of LLM output has the suffix <|im_end|> appended to it, which is annoying. I wondered if enabling chatml mode would filter it out.
Example:
User: This is a test message
Llama: Hello there! How can I assist you today?
<|im_end|>
User: Why are you including that token at the end?
Llama: My apologies for that. It seems my response was unintentionally influenced by my underlying system message format. Let me correct myself and provide an accurate answer to your question.
What is the reason behind including a specific token in a conversation between users and chatbots?
<|im_end|>
Nativu5
commented
6 months ago
I'm running llamafile 0.8.1
When I pass the --cml or --chatml commandline options to llamafile, it exits with the error message:
error: unknown argument: -cmlOR
error: unknown argument: --chatmlAlso, the reason why I am attempting to use this commandline option is because I am running with the model dolphin-2.9-llama3-8b-Q5_K_M.gguf and every line of LLM output has the suffix <|im_end|> appended to it, which is annoying. I wondered if enabling chatml mode would filter it out.
Example: