I tried to generate some text using a mixtral instruct GGUF model but the model only predicts nonsense.
Something is either wrong with the tokenizer or the chat template.
I tried to convert the model manually using this script but I get the same behavior.
Hi
I tried to generate some text using a mixtral instruct GGUF model but the model only predicts nonsense. Something is either wrong with the tokenizer or the chat template. I tried to convert the model manually using this script but I get the same behavior.
Edit: using the pip package (v0.5.0) Edit2: building from source leads to this error