I'm running this on a Mac mini M2 Pro 16GB. I used the MacOS one-click-installer, and copied the vicuna-13b-v1.5-16k.Q4_K_M model into the models dir. When I select this model, it selects the llama.cpp loader.
If I set the n-gpu-layers parameter to 0, everything works. It doesn't use the GPU though.
If I set it to 1 (or any value other than 0), loading the model produces the following:
I'm running this on a Mac mini M2 Pro 16GB. I used the MacOS one-click-installer, and copied the
vicuna-13b-v1.5-16k.Q4_K_M
model into the models dir. When I select this model, it selects the llama.cpp loader.If I set the n-gpu-layers parameter to 0, everything works. It doesn't use the GPU though.
If I set it to 1 (or any value other than 0), loading the model produces the following: