Closed Gdesau closed 5 months ago
Hello @Gdesau,
According to this I think you need to export the folder where the ggml-metal.metal
file is located, not the file itself,
So in your case, try something like:
export GGML_METAL_PATH_RESOURCES=/Users/gregoiredesauvage/Dev/Modules/pywhispercpp/whisper.cpp
If it didn't work, maybe you can try the following as well:
GGML_METAL_PATH_RESOURCES=/Users/gregoiredesauvage/Dev/Modules/pywhispercpp/whisper.cpp python /path/to/your/script.py
Hello @abdeladim-s,
Thank you very much for your help. Unfortunately the first solution couldn't fix my issue and I couldn't try the second one because I'm working in a notebook. But I'm thinking that maybe the issue may come from these 2 lines in the result:
ggml_metal_init: default.metallib not found, loading from source
ggml_metal_init: GGML_METAL_PATH_RESOURCES = nil
These are located just after this line:
ggml_metal_init: picking default device: Apple M3 Max
I've tried to search about the default.metallib and why it's not found but couldn't find something helpful.
Do you have any idea?
@Gdesau, I think using a notebook is part of the problem.
GGML_METAL_PATH_RESOURCES
won't visible inside the notebook with the export
command, as you can see from the log
ggml_metal_init: GGML_METAL_PATH_RESOURCES = nil
What I can suggest:
%env GGML_METAL_PATH_RESOURCES=/Users/gregoiredesauvage/Dev/Modules/pywhispercpp/whisper.cpp
, or modify the kernel.json
file and see if that will work.@abdeladim-s, thanks a lot, setting the env variable with %env GGML_METAL_PATH_RESOURCES=/Users/gregoiredesauvage/Dev/Modules/pywhispercpp/whisper.cpp
inside the notebook solved the problem! But I ran into another problem then, the "ggml-common.h" file has not been found but it's also located in the /Users/my_user/Dev/Modules/pywhispercpp/whisper.cpp
directory:
ggml_metal_init: error: Error Domain=MTLLibraryErrorDomain Code=3 "program_source:3:10: fatal error: 'ggml-common.h' file not found
#include "ggml-common.h"
^~~~~~~~~~~~~~~
" UserInfo={NSLocalizedDescription=program_source:3:10: fatal error: 'ggml-common.h' file not found
#include "ggml-common.h"
^~~~~~~~~~~~~~~
}
whisper_backend_init: ggml_backend_metal_init() failed
Do you think that there are other env variables to set? I've checked the "ggml-metal.metal" file it didn't seem to be but maybe I missed something
@Gdesau, I think it's the same issue as #2041
Are you using the latest commit from whisper.cpp
?
Please follow the instructions on the readme page and build against the same hash commit included in the project.
@abdeladim-s, Thanks a lot! I followed the #2041, and hardcoded the path to the "ggml-common.h" file into the "ggml-metal.metal" file and it worked! So the 3 first lines of "ggml-metal.metal" file were as follows:
#define GGML_COMMON_DECL_METAL
#define GGML_COMMON_IMPL_METAL
#include "ggml-common.h"
And I changed 3rd line with:
#include "/Users/my_user/Dev/Modules/pywhispercpp/whisper.cpp/ggml-common.h"
Now the Core ML model is loaded successfully in my notebook. So, to summarize, the full solution is to:
%env GGML_METAL_PATH_RESOURCES=/Users/my_user/Dev/Modules/pywhispercpp/whisper.cpp
`Glad it finally worked! Thanks @Gdesau for posting the solution, this will certainly help others facing the same issue.
Hello everyone, I'm working with an M3 Max and I've tried to load the "ggml-large-v3.bin" model with the following code:
But it's unable to find the ggml-metal.metal file when it is actually present in the whisper.cpp folder. It gives me the following result:
I've tried to add the path to the environment variables with:
but it didn't work.
I have the "ggml-large-v3-encoder.mlmodelc" file in the same folder as the "ggml-large-v3.bin" file.
Any idea?