I followed the instructions on my MacBook and hit a few issues:
missing instructions for converting the codec
instructions used F32 instead of F16
Metal build did not work due to obsolete call to ggml_metal_log_set_callback()
@PABannier Looking at the code, there is no way to set n_gpu_layers - is this intentional? At this point it seems the computation always runs on the CPU
I followed the instructions on my MacBook and hit a few issues:
ggml_metal_log_set_callback()
@PABannier Looking at the code, there is no way to set
n_gpu_layers
- is this intentional? At this point it seems the computation always runs on the CPUGood job 👍