search

argmaxinc / WhisperKit

On-device Speech Recognition for Apple Silicon
https://takeargmax.com/blog/whisperkit
MIT License
3.17k stars 268 forks source link

Unable to load model in CLI #4

Closed usmanm closed 8 months ago

usmanm commented 8 months ago

Hey folks! I'm trying to use the CLI, but it fails to load models:

Building for debugging...
Build complete! (0.07s)
Error: Unable to load model: file:///Users/usmanm/whisperkit/Models/whisperkit-coreml/openai_whisper-tiny/MelSpectrogram.mlmodelc/. Compile the model with Xcode or `MLModel.compileModel(at:)`.

The setup instructions seemed to have worked correctly:

➜  whisperkit git:(main) make setup
Setting up environment...
/opt/homebrew/bin/pip3
/opt/homebrew/bin/python3
Requirement already satisfied: huggingface_hub in /opt/homebrew/lib/python3.11/site-packages (0.20.3)
Requirement already satisfied: filelock in /opt/homebrew/lib/python3.11/site-packages (from huggingface_hub) (3.13.1)
Requirement already satisfied: fsspec>=2023.5.0 in /opt/homebrew/lib/python3.11/site-packages (from huggingface_hub) (2023.10.0)
Requirement already satisfied: requests in /opt/homebrew/lib/python3.11/site-packages (from huggingface_hub) (2.31.0)
Requirement already satisfied: tqdm>=4.42.1 in /opt/homebrew/lib/python3.11/site-packages (from huggingface_hub) (4.66.1)
Requirement already satisfied: pyyaml>=5.1 in /opt/homebrew/lib/python3.11/site-packages (from huggingface_hub) (6.0.1)
Requirement already satisfied: typing-extensions>=3.7.4.3 in /opt/homebrew/lib/python3.11/site-packages (from huggingface_hub) (4.8.0)
Requirement already satisfied: packaging>=20.9 in /opt/homebrew/lib/python3.11/site-packages (from huggingface_hub) (23.2)
Requirement already satisfied: charset-normalizer<4,>=2 in /opt/homebrew/lib/python3.11/site-packages (from requests->huggingface_hub) (3.3.2)
Requirement already satisfied: idna<4,>=2.5 in /opt/homebrew/lib/python3.11/site-packages (from requests->huggingface_hub) (3.4)
Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/homebrew/lib/python3.11/site-packages (from requests->huggingface_hub) (2.0.7)
Requirement already satisfied: certifi>=2017.4.17 in /opt/homebrew/lib/python3.11/site-packages (from requests->huggingface_hub) (2023.7.22)
usmanm
Already logged in to Hugging Face.
➜  whisperkit git:(main) make download-models
Downloading compressed models...
Repository exists, pulling latest changes...
HEAD is now at 07ea546 Create config.json
zbiljic commented 8 months ago

Hey @usmanm, I was playing around and encountered same issue. Then I saw that models are not actually downloaded but only had Git LFS links.

In order to get the CLI working, you would need to navigate into Models/whisperkit-coreml (after make download-models step), and then run git lfs install (not sure right now if there are some larger than 5GiB for which huggingface-cli lfs-enable-largefiles . command is also needed). After Git LFS install step you can download all files with git lfs pull (not recommending straight away), or files for the model that you want to use, for your example git lfs pull --include openai_whisper-tiny.

Once all LFS files are resolved it should be possible to run CLI command.

ZachNagengast commented 8 months ago

@zbiljic thanks for chiming in, yes git-lfs is required to download the full set of models.

If it's not on your system you can install it with the command brew install git-lfs - will look into adding this to the make setup command

whalecodeim commented 8 months ago

After the command: swift run transcribe --model-path "Models/whisperkit-coreml/openai_whisper-tiny" --audio-path /Users/fch/Downloads/GopherConAU_2023-1080.mp3 . I got an error: Building for debugging... Build complete! (0.17s) Error: An SSL error has occurred and a secure connection to the server cannot be made.

whalecodeim commented 8 months ago

After the command: swift run transcribe --model-path "Models/whisperkit-coreml/openai_whisper-tiny" --audio-path /Users/fch/Downloads/GopherConAU_2023-1080.mp3 . I got an error: Building for debugging... Build complete! (0.17s) Error: An SSL error has occurred and a secure connection to the server cannot be made.

I have fix the proble. git lfs pull --include openai_whisper-tiny . notice the . It is very important

ZachNagengast commented 8 months ago

Ah interesting, thanks for looking into this, glad you got it working!