Cannot find `ggml-metal.metal` on Mac to run on metal

[ltoniazzi]

Issue

Installed with:

pip install pywhispercpp

and running:

model = Model("base.en", n_threads=6)
segments = model.transcribe(raw_audio_path)

The code runs on cpu as I see the ggml logs:

ggml_metal_init: found device: Apple M2
ggml_metal_init: picking default device: Apple M2
ggml_metal_init: default.metallib not found, loading from source
ggml_metal_init: GGML_METAL_PATH_RESOURCES = nil
ggml_metal_init: error: could not use bundle path to find ggml-metal.metal, falling back to trying cwd
ggml_metal_init: loading 'ggml-metal.metal'
ggml_metal_init: error: Error Domain=NSCocoaErrorDomain Code=260 "The file “ggml-metal.metal” couldn’t be opened because there is no such file." UserInfo={NSFilePath=ggml-metal.metal, NSUnderlyingError=0x116ecdf00 {Error Domain=NSPOSIXErrorDomain Code=2 "No such file or directory"}}
whisper_backend_init: ggml_backend_metal_init() failed
whisper_model_load:      CPU buffer size =   147.46 MB
whisper_model_load: model size    =  147.37 MB
whisper_backend_init: using Metal backend
ggml_metal_init: allocating
ggml_metal_init: found device: Apple M2
ggml_metal_init: picking default device: Apple M2
ggml_metal_init: default.metallib not found, loading from source
ggml_metal_init: GGML_METAL_PATH_RESOURCES = nil
ggml_metal_init: error: could not use bundle path to find ggml-metal.metal, falling back to trying cwd
ggml_metal_init: loading 'ggml-metal.metal'
ggml_metal_init: error: Error Domain=NSCocoaErrorDomain Code=260 "The file “ggml-metal.metal” couldn’t be opened because there is no such file." UserInfo={NSFilePath=ggml-metal.metal, NSUnderlyingError=0x116edfd20 {Error Domain=NSPOSIXErrorDomain Code=2 "No such file or directory"}}
whisper_backend_init: ggml_backend_metal_init() failed

Question

How to fix this? should I add manually the path to ggml-metal.metal as an env variable?

Running on Mac/M2

[abdeladim-s]

@ltoniazzi, Seems similar to #35. But this shouldn't happen if you installed the package from pip!! Please try the solution provided in that issue.

[ltoniazzi]

@abdeladim-s Thank you, but I cannot locate whisper.cpp in my installation as in the PR solution. The solution points to a path like

GGML_METAL_PATH_RESOURCES=/Users/my_user/Dev/Modules/pywhispercpp/whisper.cpp

But this is what I see in my pip installation of pywhispercpp (in a virtual environment):

Where should I find the location of whisper.cpp?

[UsernamesLame]

@ltoniazzi, Seems similar to #35. But this shouldn't happen if you installed the package from pip!! Please try the solution provided in that issue.

I'm actually having the same issue with sgml-metal, but my understanding is you must pre-generate the ggml-metal models using whisper.cpp's conversion tools.

[UsernamesLame]

whisper_init_state: loading Core ML model from '/Users/user/Library/Application Support/pywhispercpp/models/ggml-base.en-encoder.mlmodelc'

We should really fix this. The model should be loaded from the venv, not arbitrary locations on disk.

[UsernamesLame]

Ok, a bit of digging, we can get the en-encoder.mlmodelc from here: https://huggingface.co/ggerganov/whisper.cpp/tree/main

Renaming it to ggml-base.en-encoder.mlmodelc and putting it in pwhispercpp/models works!

[UsernamesLame]

[2024-09-20 09:12:13,861] {model.py:132} INFO - Transcribing ...

Traceback (most recent call last):
  File "/Users/user/Desktop/whisper-metal/__main__.py", line 4, in <module>
    segments = model.transcribe('file.mp3')
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/user/Desktop/whisper-metal/.venv/lib/python3.12/site-packages/pywhispercpp/model.py", line 133, in transcribe
    res = self._transcribe(audio, n_processors=n_processors)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/user/Desktop/whisper-metal/.venv/lib/python3.12/site-packages/pywhispercpp/model.py", line 249, in _transcribe
    res = Model._get_segments(self._ctx, 0, n)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/user/Desktop/whisper-metal/.venv/lib/python3.12/site-packages/pywhispercpp/model.py", line 154, in _get_segments
    text = pw.whisper_full_get_segment_text(ctx, i)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
UnicodeDecodeError: 'utf-8' codec can't decode bytes in position 57-58: invalid continuation byte

I think we have a regression!

[abdeladim-s]

@ltoniazzi, @UsernamesLame, Could you please try the latest pre-built wheel from CI. I haven't pushed a new version yet to PyPI.

Pick the right wheel version from the artifacts and install it on a fresh virtual environment. Let me know if that works ?

[ltoniazzi]

@abdeladim-s I tried installing both wheels for Mac on python311 but I get this error inporting the package:

>>> from pywhispercpp.model import Model
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/user/Workspace/pre-screener/.venv_testpywish_2/lib/python3.11/site-packages/pywhispercpp/__init__.py", line 14, in <module>
    ctypes.CDLL(os.path.join(os.path.dirname(__file__), 'lib', file))
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/ctypes/__init__.py", line 376, in __init__
    self._handle = _dlopen(self._name, mode)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: dlopen(/Users/user/Workspace/pre-screener/.venv_testpywish_2/lib/python3.11/site-packages/pywhispercpp/lib/_pywhispercpp.cpython-38-darwin.so, 0x0006): Library not loaded: @rpath/libwhisper.1.dylib
  Referenced from: <4499501D-7C43-3580-B7DB-D4F668C98970> /Users/user/Workspace/pre-screener/.venv_testpywish_2/lib/python3.11/site-packages/pywhispercpp/lib/_pywhispercpp.cpython-38-darwin.so
  Reason: tried: '/Users/runner/work/pywhispercpp/pywhispercpp/build/lib.macosx-10.9-universal2-cpython-38/libwhisper.1.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/Users/runner/work/pywhispercpp/pywhispercpp/build/lib.macosx-10.9-universal2-cpython-38/libwhisper.1.dylib' (no such file), '/Users/runner/work/pywhispercpp/pywhispercpp/build/lib.macosx-10.9-universal2-cpython-38/libwhisper.1.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/Users/runner/work/pywhispercpp/pywhispercpp/build/lib.macosx-10.9-universal2-cpython-38/libwhisper.1.dylib' (no such file)
>>> find .venv_testpywish/lib/ -name "libwhisper*.dylib"
  File "<stdin>", line 1
    find .venv_testpywish/lib/ -name "libwhisper*.dylib"
                                     ^^^^^^^^^^^^^^^^^^^
SyntaxError: invalid syntax
>>> exit()

Notes:

• Install command

  pip install ~/Downloads/artifact/pywhispercpp-1.2.0-cp311-cp311-macosx_11_0_universal2.whl

• I have these in my venv:

  
  find .venv_testpywish/lib/ -name "libwhisper*.dylib"

.venv_testpywish/lib//python3.11/site-packages/libwhisper.dylib .venv_testpywish/lib//python3.11/site-packages/libwhisper.1.dylib .venv_testpywish/lib//python3.11/site-packages/libwhisper.1.6.2.dylib

[abdeladim-s]

@ltoniazzi, could you please check if the lib folder /Users/user/Workspace/pre-screener/.venv_testpywish_2/lib/python3.11/site-packages/pywhispercpp/lib contains the libwhisper dylib files ?

[ltoniazzi]

@abdeladim-s It contains these three:

>>> find .venv_testpywish_2/lib/ -name "libwhisper*.dylib"
.venv_testpywish_2/lib//python3.11/site-packages/libwhisper.dylib
.venv_testpywish_2/lib//python3.11/site-packages/libwhisper.1.dylib
.venv_testpywish_2/lib//python3.11/site-packages/libwhisper.1.6.2.dylib

[abdeladim-s]

@ltoniazzi, Not in the .venv_testpywish_2/lib/ folder but inside pywhispercpp/lib folder! Could you also please check if the extension _pywhispercpp.cpython-38-darwin.so is also in the pywhispercpp/lib folder? .. maybe try to delete it and check ?

[ltoniazzi]

There is only these .so files in there:

>>> ls .venv_testpywish_2/lib/python3.11/site-packages/pywhispercpp/lib
_pywhispercpp.cpython-310-darwin.so 
_pywhispercpp.cpython-311-darwin.so 
_pywhispercpp.cpython-38-darwin.so  
_pywhispercpp.cpython-39-darwin.so

After deleting them I get this error:

>>> from pywhispercpp.model import Model
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/user/Workspace/pre-screener/.venv_testpywish_2/lib/python3.11/site-packages/pywhispercpp/model.py", line 14, in <module>
    import _pywhispercpp as pw
ImportError: dlopen(/Users/useruser/Workspace/pre-screener/.venv_testpywish_2/lib/python3.11/site-packages/_pywhispercpp.cpython-311-darwin.so, 0x0002): Library not loaded: @rpath/libwhisper.1.dylib
  Referenced from: <6D5C74BD-6BC9-3B3F-9FBB-A28147E30808> /Users/user/Workspace/pre-screener/.venv_testpywish_2/lib/python3.11/site-packages/_pywhispercpp.cpython-311-darwin.so
  Reason: tried: '/Users/runner/work/pywhispercpp/pywhispercpp/build/lib.macosx-10.9-universal2-cpython-311/libwhisper.1.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/Users/runner/work/pywhispercpp/pywhispercpp/build/lib.macosx-10.9-universal2-cpython-311/libwhisper.1.dylib' (no such file), '/Users/runner/work/pywhispercpp/pywhispercpp/build/lib.macosx-10.9-universal2-cpython-311/libwhisper.1.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/Users/runner/work/pywhispercpp/pywhispercpp/build/lib.macosx-10.9-universal2-cpython-311/libwhisper.1.dylib' (no such file)

[abdeladim-s]

@ltoniazzi, yeah there were no dylib files there! Please try this new build! I’ve added the dylib files to the lib folder.