How to increase speech to text speed when using whisper cpp?

[ITHealer]

@nyadla-sys Currently I am using whisper_tiny.en.tflite for my android project. And the inference time for speech to text is quite long. https://github.com/usefulsensors/openai-whisper/tree/main/android_app/Whisper-TFLIte-Android-Example

I/flutter ( 3449): Model name: /assets/whisper-tiny.en.tflite I/flutter ( 3449): Load success I/flutter ( 3449): Load model time 0:00:03.274847 I/htotext_exampl( 3449): ProcessProfilingInfo new_methods=1340 is saved saved_to_disk=1 resolve_classes_delay=8000 I/tflite ( 3449): Initialized TensorFlow Lite runtime. W/1.ui ( 3449): type=1400 audit(0.0:15783): avc: denied { read } for name="u:object_r:vendor_default_prop:s0" dev="tmpfs" ino=14533 scontext=u:r:untrusted_app:s0:c127,c257,c512,c768 tcontext=u:object_r:vendor_default_prop:s0 tclass=file permissive=0 E/libc ( 3449): Access denied finding property "ro.hardware.chipname" I/tflite ( 3449): Created TensorFlow Lite XNNPACK delegate for CPU. I/flutter ( 3449): Result process audio file: I love you. I/flutter ( 3449): Processing time 0:00:11.894501

Is there any way to shorten the time, please help me? Thanks!

[nyadla-sys]

Are you running on Google pixel Android phone? If so mostly it will take less than 1second inference time

[ITHealer]

Are you running on Android

Yes, i am running it on android

I have customized the C source into separate functions for each function. Like load model and process audio. And run in flutter.

I'm not sure if the cause is inference or not. However, when performing call func process audio to text conversion takes up to 11 seconds. Model loading is done when initializing the application.

And I also tried running your original Android source again, but it also took the same amount of time. Is it because of the model?

Can you give me a solution on how to reduce this time?

[nyadla-sys]

@vilassn pls help this query

[nyadla-sys]

Meanwhile try below whisper app on your android phone.let me know how my much time does it take to display text. https://play.google.com/store/apps/details?id=com.whisper.android.tflitecpp

[ITHealer]

Meanwhile try below whisper app on your android phone.let me know how my much time does it take to display text. https://play.google.com/store/apps/details?id=com.whisper.android.tflitecpp

I just downloaded the application and tested it on Samsung Galaxy A04 and the time fluctuated around 10 seconds

[nyadla-sys]

Try on high end android phone like pixel 7 or latest samsung galaxy series

[nyadla-sys]

Old phones may not have better compute cycles available

[ITHealer]

Old phones may not have better compute cycles available

Sorry for bothering you. I currently do not have a newer Android device. Can you tell me the maximum speed you have used of the app?

If so, the speed depends on the device's computing configuration. Is there any way to speed up calculation?

[ITHealer]

Is the source code of the app on CHPlay this source? https://github.com/usefulsensors/openai-whisper/blob/main/android_app/android_whisper_app.zip

[nyadla-sys]

Yes