Llama 3.2 1b model not generating response

[mustangs0786]

Hi i am trying to use llama 3.2 1b instruct model and it is not generating ans. Can anyone help where i am doing wrong. Thanks

from mlx_lm import load, generate
model, tokenizer = load(path_or_hf_repo = "unsloth/Llama-3.2-1B",#,tokenizer_config=tokenizer_config
                        tokenizer_config={"trust_remote_code": True})#,"eos_token": "<|end_of_text|>"})
response = generate(model, tokenizer, prompt="""
<|begin_of_text|><|start_header_id|>system<|end_header_id|>

You are a helpful assistant<|eot_id|><|start_header_id|>user<|end_header_id|>

Who are you?<|eot_id|><|start_header_id|>assistant<|end_header_id|>

""", verbose=False)

[awni]

Seems like a problem with your model and not MLX...

Using meta-llama/Llama-3.2-1B-Instruct works just fine for me. Any reason you are using "unsloth/Llama-3.2-1B" ?

[mustangs0786]

Thanks for quick reply, let me try with that. i am still waiting for llama team to approve my request.

[awni]

Ok let me know if there is still an issue and I can reopen this.