search

xuegao-tzx / whisper_flutter_new

A flutter library for offline speech-to-text conversion which use whisper.cpp models implementation for Android、iOS、macOS.
https://pub.dev/packages/whisper_flutter_new
GNU General Public License v3.0
11 stars 3 forks source link

Add all supported models #2

Open jhj0517 opened 2 months ago

jhj0517 commented 2 months ago

Hi, thanks for the project! I tested it on Android and it actually worked.

Changed :

  1. Added all supported models from ggerganov/whisper.cpp, including .en (English only) and quantization models.

Here's my test result with the base models for jfk.wav:

Device Name RAM Capacity CPU Clock
Android Galaxy Quantum 3 8 GB 2400 MHz
base base-q5_1 base.en base.en-q5_1
17.319 s 16.235 s 15.552 s 15.016 s

It only took 3.992 s from the official demo, so it seems this doesn't help #1. But I think it would be good to support all models.

xuegao-tzx commented 2 months ago

Thanks for your submission. Have you tested the large-v3 model? I tested the large-v3 model and the transcription accuracy was not good enough.

I am developing an implementation based on the latest official version, and the performance will be basically the same as the official one

jhj0517 commented 2 months ago

Large models always crash on my device due to lack of RAM, so unfortunately I couldn't test them. My maximum available models were medium, and it took 180 s for jfk.wav in this repo.

I am developing an implementation based on the latest official version, and the performance will be basically the same as the official one

Really happy to hear this! hope to see it soon.

xuegao-tzx commented 2 months ago

Large models always crash on my device due to lack of RAM, so unfortunately I couldn't test them. My maximum available models were medium, and it took 180 s for jfk.wav in this repo.

I am developing an implementation based on the latest official version, and the performance will be basically the same as the official one

Really happy to hear this! hope to see it soon.

I will try to add these models to the new version. I am currently developing a new version. After the new version is completed, I will test it and merge it after the test is correct.

xuegao-tzx commented 2 months ago

I made a preliminary attempt and found that the flutter_rust_bridge library could not be packaged properly in iOS and MacOS. The Android version can achieve high performance by calling the official library. The current problem is that since iOS and MacOS cannot achieve it, higher performance cannot be achieved. However, it may be possible to improve Android performance by only implementing it.

jhj0517 commented 2 months ago

I'm just listing some issues here that might seem related:

Thanks for your hard work.

xuegao-tzx commented 2 months ago

Now, you can try to use this app. After registering and logging in, sending voice messages to this app will convert the local model voice to text when there is no network, and the voice to text will be converted by calling the API when there is network.You can take a look at the online and offline conversion time.