The code below is a test code that loads the model and generates for fine tuning.
from mlx_lm import generate, load
import os
os.environ["HFTOKEN"] = 'hf'
model, tokenizer = load("google/gemma-1.1-2b-it")
Generating without adding a prompt template manually
prompt = """
Why is the sky blue?
""".strip()
An error occurs in this part.
response = generate(
model,
tokenizer,
prompt=prompt,
verbose=True, # Set to True to see the prompt and response
temp=0.0,
max_tokens=256,
)
This is the error message printed out.
"""
Prompt: Why is the sky blue?
libc++abi: terminating due to uncaught exception of type std::runtime_error: [metal::Device] Unable to build metal library from source
program_source:1079:14: error: cannot initialize a variable of type 'bfloat16_t' (aka 'bfloat') with an rvalue of type 'float'
bfloat16_t tmp_B = 1.414062;
^ ~~~~
program_source:1104:14: error: cannot initialize a variable of type 'bfloat16_t' (aka 'bfloat') with an rvalue of type 'float'
bfloat16_t tmp_B = 1.414062;
^ ~~~~
program_source:1130:14: error: cannot initialize a variable of type 'bfloat16_t' (aka 'bfloat') with an rvalue of type 'float'
bfloat16_t tmp_B = 1.414062;
^ ~~~~
program_source:1157:14: error: cannot initialize a variable of type 'bfloat16_t' (aka 'bfloat') with an rvalue of type 'float'
bfloat16_t tmp_B = 1.414062;
^ ~~~~
program_source:1185:14: error: cannot initialize a variable of type 'bfloat16_t' (aka 'bfloat') with an rvalue of type 'float'
bfloat16_t tmp_B = 1.414062;
^ ~~~~
program_source:1214:14: error: cannot initialize a variable of type 'bfloat16_t' (aka 'bfloat') with an rvalue of type 'float'
bfloat16_t tmp_B = 1.414062;
^ ~~~~
program_source:1244:14: error: cannot initialize a variable of type 'bfloat16_t' (aka 'bfloat') with an rvalue of type 'float'
bfloat16_t tmp_B = 1.414062;
^ ~~~~
program_source:1275:14: error: cannot initialize a variable of type 'bfloat16_t' (aka 'bfloat') with an rvalue of type 'float'
bfloat16_t tmp_B = 1.414062;
^ ~~~~
program_source:1300:14: error: cannot initialize a variable of type 'bfloat16_t' (aka 'bfloat') with an rvalue of type 'float'
bfloat16_t tmp_B = 1.414062;
^ ~~~~
"""
I don't know why this error occurs, but I would like to ask for help if there is a solution.
awni
commented
3 months ago
below code
my laptop specs : mac m1 max 64GB, macOs : 14.5
The code below is a test code that loads the model and generates for fine tuning.
from mlx_lm import generate, load import os os.environ["HFTOKEN"] = 'hf' model, tokenizer = load("google/gemma-1.1-2b-it")
Generating without adding a prompt template manually
prompt = """ Why is the sky blue? """.strip()
An error occurs in this part.
response = generate( model, tokenizer, prompt=prompt, verbose=True, # Set to True to see the prompt and response temp=0.0, max_tokens=256, )
This is the error message printed out.
""" Prompt: Why is the sky blue? libc++abi: terminating due to uncaught exception of type std::runtime_error: [metal::Device] Unable to build metal library from source program_source:1079:14: error: cannot initialize a variable of type 'bfloat16_t' (aka 'bfloat') with an rvalue of type 'float' bfloat16_t tmp_B = 1.414062; ^
~~~~ program_source:1104:14: error: cannot initialize a variable of type 'bfloat16_t' (aka 'bfloat') with an rvalue of type 'float' bfloat16_t tmp_B = 1.414062; ^~~~~ program_source:1130:14: error: cannot initialize a variable of type 'bfloat16_t' (aka 'bfloat') with an rvalue of type 'float' bfloat16_t tmp_B = 1.414062; ^~~~~ program_source:1157:14: error: cannot initialize a variable of type 'bfloat16_t' (aka 'bfloat') with an rvalue of type 'float' bfloat16_t tmp_B = 1.414062; ^~~~~ program_source:1185:14: error: cannot initialize a variable of type 'bfloat16_t' (aka 'bfloat') with an rvalue of type 'float' bfloat16_t tmp_B = 1.414062; ^~~~~ program_source:1214:14: error: cannot initialize a variable of type 'bfloat16_t' (aka 'bfloat') with an rvalue of type 'float' bfloat16_t tmp_B = 1.414062; ^~~~~ program_source:1244:14: error: cannot initialize a variable of type 'bfloat16_t' (aka 'bfloat') with an rvalue of type 'float' bfloat16_t tmp_B = 1.414062; ^~~~~ program_source:1275:14: error: cannot initialize a variable of type 'bfloat16_t' (aka 'bfloat') with an rvalue of type 'float' bfloat16_t tmp_B = 1.414062; ^~~~~ program_source:1300:14: error: cannot initialize a variable of type 'bfloat16_t' (aka 'bfloat') with an rvalue of type 'float' bfloat16_t tmp_B = 1.414062; ^~~~~ """I don't know why this error occurs, but I would like to ask for help if there is a solution.