search

usefulsensors / openai-whisper

Robust Speech Recognition via Large-Scale Weak Supervision
MIT License
62 stars 24 forks source link

GPU support via GPU delegates? #36

Open sandorkonya opened 1 year ago

sandorkonya commented 1 year ago

Nice Repository!

@nyadla-sys what do you think, could one use the Android device's GPU to accelerate inference with the GPU delegates? Any pointers for this?

thy!!

nyadla-sys commented 1 year ago

We have been unable to use GPUdelegate to whisper.tflite model on an Android device as of today. If any progress is made, I will provide an update here

sandorkonya commented 1 year ago

It would be a great performance increase. Keep up the good work!!

sandorkonya commented 1 year ago

@nyadla-sys i mentioned this here and they adviced to use Google play services tf implementation. What do you think?

v-prgmr commented 1 year ago

@sandorkonya I tried running the float 32 tflite model on the GPU using GPU delegates and it did not work. Can only run it on CPU.

v-prgmr commented 1 year ago

https://www.tensorflow.org/lite/guide/model_analyzer#check_gpu_delegate_compatibility Using the delegate compatibility check, it is clear that the model has lot of kernel nodes that are not supported in TFLite runtime version 2.0

@nyadla-sys FYI