search

ggerganov / whisper.cpp

Port of OpenAI's Whisper model in C/C++
MIT License
34.25k stars 3.48k forks source link

Core ML integration sample #889

Open 07JP27 opened 1 year ago

07JP27 commented 1 year ago

Firstly, I would like to express my gratitude to the individuals who have contributed to this fantastic package.

I am currently attempting to create an application using Core ML, but I am facing some challenges. I followed the instructions mentioned in the README and generated the mlmodelc and mlpackage. Afterwards, I successfully built and executed the CLI application using cpp.

Now, I want to integrate this model into an actual SwiftUI app, but I am unsure about the usage. Specifically, I am unsure which methods to call in the automatically generated class when I drop the mlpackage into Xcode, and how to prepare the input to be provided there. I have also checked #566 , but it doesn't seem to provide the information I want.

Do you have any sample code or additional advice to assist me? I appreciate your help.

andforce commented 1 year ago

i face the same issues. how to use mlpackage on iOS devices?

drasticactions commented 1 year ago

https://github.com/drasticactions/whisper.cpp/tree/coreml-demo/examples/whisper.swiftui

I have tried to update the sample to show how you can enable CoreML. In short, you need to:

Having this in the docs would be nice if you're doing source builds, which seems to be how most people not using bindings consume Whisper.cpp.

スクリーンショット 2023-05-13 22 39 19

If it's working, you should see this in your output window.

barkb commented 1 year ago

@drasticactions Could you explain the "Include the CoreML classes and headers" part more simply (sorry still a beginner!). I looked at your README and did everything else, but that I'm not sure what to do. Is there a file hierarchy I need to match? Below is how I have them included currently. whisper.cpp.swiftui coreml

gavin1818 commented 1 month ago

Hi @barkb did you figure out eventually, Im also strugging with it now. I follow the instruction in the page @drasticactions, I got no luck. Im not sure what does the Include the CoreML classes and headers mean ?