raivisdejus
commented
2 weeks ago Please describe a bit more detailed how you compiled the whisper.cpp and Buzz.
Buzz needs to compile python bindings, those are specific to a particular git commit version "linked" to the buzz repo. As code in whisper.cpp repo changes you need to look at the version linked to Buzz. Instructions you used seem to be correct as you are referencing the appropriate whisper.cpp commit.
Compilation of Buzz will take care of everything if you run it as detailed here https://github.com/chidiwilliams/buzz/blob/main/CONTRIBUTING.md#mac
The Core ML support has to be added in Buzz Makefile Most likely here https://github.com/chidiwilliams/buzz/blob/main/Makefile#L74 you need to add -DWHISPER_COREML=1 or -DWHISPER_COREML=ON
Also see this https://github.com/ggerganov/whisper.cpp/issues/1707
Path you need to add as GGML_METAL_PATH_RESOURCES is most likely path to whisper.cpp folder, something like /Users/YourUserName/Downloads/buzz/whisper.cpp if the repo was cloned in your Downloads folder.
Also this may be helpful https://github.com/abdeladim-s/pywhispercpp/issues/35 Someone solved similar problem
If or more precisely when you get this working, please share the notes or comments on what was missing from currently documented steps, so we can add the missing parts for others.
Uday-P-Patel
im trying to run buzz locally with whisper.cpp models with coreML support and i want to make sure it uses M@ chip/GPU, here is what i have done, -cloned the repo in my mac -followed the steps and made it running -then i compiled whisper.cpp with these instructions: https://github.com/ggerganov/whisper.cpp/tree/c7b6988678779901d02ceba1a8212d2c9908956e?tab=readme-ov-file#core-ml-support -and re built the buzz to make sure whisper_cpp.py is updated correctly -also ran the whisper.cpp with coreML models with an audio file so that i can make sure that it uses M@ chip
but when i run buzz locally and press recors it gives this error:
whisper_backend_init: using Metal backend ggml_metal_init: allocating ggml_metal_init: found device: Apple M2 Pro ggml_metal_init: picking default device: Apple M2 Pro ggml_metal_init: default.metallib not found, loading from source ggml_metal_init: GGML_METAL_PATH_RESOURCES = nil ggml_metal_init: error: could not use bundle path to find ggml-metal.metal, falling back to trying cwd ggml_metal_init: loading 'ggml-metal.metal' ggml_metal_init: error: Error Domain=NSCocoaErrorDomain Code=260 "The file “ggml-metal.metal” couldn’t be opened because there is no such file." UserInfo={NSFilePath=ggml-metal.metal, NSUnderlyingError=0x600001890090 {Error Domain=NSPOSIXErrorDomain Code=2 "No such file or directory"}} whisper_backend_init: ggml_backend_metal_init() failed whisper_model_load: CPU total size = 77.11 MB whisper_model_load: model size = 77.11 MB whisper_backend_init: using Metal backend ggml_metal_init: allocating ggml_metal_init: found device: Apple M2 Pro ggml_metal_init: picking default device: Apple M2 Pro ggml_metal_init: default.metallib not found, loading from source ggml_metal_init: GGML_METAL_PATH_RESOURCES = nil ggml_metal_init: error: could not use bundle path to find ggml-metal.metal, falling back to trying cwd ggml_metal_init: loading 'ggml-metal.metal' ggml_metal_init: error: Error Domain=NSCocoaErrorDomain Code=260 "The file “ggml-metal.metal” couldn’t be opened because there is no such file." UserInfo={NSFilePath=ggml-metal.metal, NSUnderlyingError=0x600001898810 {Error Domain=NSPOSIXErrorDomain Code=2 "No such file or directory"}} whisper_backend_init: ggml_backend_metal_init() failed whisper_init_state: kv self size = 9.44 MB whisper_init_state: kv cross size = 9.44 MB whisper_init_state: kv pad size = 2.36 MB whisper_init_state: compute buffer (conv) = 13.32 MB whisper_init_state: compute buffer (encode) = 87.96 MB whisper_init_state: compute buffer (cross) = 4.01 MB whisper_init_state: compute buffer (decode) = 96.02 MB
and then it falls back to CPU and runs fine
i checked and the files are all present in the whisper.cpp folder in buzz
if anything cloud help it will be great