search

nickahoy / chatty

0 stars 0 forks source link

Setup Whisper.cpp #1

Open nickahoy opened 1 day ago

nickahoy commented 1 day ago

Setup

Gather resources

  1. Clone repository

git clone https://github.com/ggerganov/whisper.cpp.git cd whisper.app

  1. Download model

sh ./models/download-ggml-model.sh medium.en

Build CoreML version of Whisper

  1. Create and setup python environment

python3.10 -m venv . pip update ...

  1. Install dependencies
pip install ane_transformers
pip install openai-whisper
pip install coremltools
  1. Manually resolve dependency conflicts (see https://github.com/ggerganov/whisper.cpp/issues/2423)
pip install "numpy<2"
pip install torch==2.1.0
  1. Create model

./models/generate-coreml-model.sh medium.en

  1. Build model
make clean
WHISPER_COREML=1 make -j
nickahoy commented 1 day ago

When trying to create model (./models/generate-coreml-model.sh medium.en) fails

A module that was compiled using NumPy 1.x cannot be run in NumPy 2.0.2 as it may crash. To support both 1.x and 2.x versions of NumPy, modules must be compiled with NumPy 2.0. Some module may need to rebuild instead e.g. with 'pybind11>=2.12'.

If you are a user of the module, the easiest solution will be to downgrade to 'numpy<2' or try to upgrade the affected module. We expect that some modules will need time to support NumPy 2.

https://github.com/ggerganov/whisper.cpp/issues/2423

nickahoy commented 1 day ago

The following warning appears several times when calling make -j:

TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!

Don't understand the consequences of this.