How to add TfLite GPU delegate

[suyash-narain]

Hi,

how do we add TfLite GPU delegate to whisper.tflite c++ app?

whenever i try to add gpu delegate options to the file TFLiteEngine.cpp

auto* delegate = TfLiteGpuDelegateV2Create(nullptr);
if (g_whsiper_tflite.interpreter->ModifyGraphWithDelegate(delegate) != kTfLiteOk) return "";

before the line
https://github.com/nyadla-sys/whisper.tflite/blob/29fe1adf2bd954b01f573c5d6dfed2dc009cb363/whisper_android/app/src/main/cpp/TFLiteEngine.cpp#L195

I get the error

ld: error: undefined symbol: TfLiteGpuDelegateV2Create

or I get the error

no matching function to call TfLiteGpuDelegateV2Create

Am following https://www.tensorflow.org/lite/android/delegates/gpu_native#enable_gpu_acceleration steps to enable GPU delegate. Not sure how to add the delegate here.

thanks

[nyadla-sys]

tflite gpu_delegate is not working as some of the ops from whisper.tflite are not supported in gpu ops

[suyash-narain]

so shouldn't these ops simply fallback onto CPU by default?

[nyadla-sys]

This model is a hybrid type, indicating that its weights are in int8 format, while the operator is in float. I suspect that GPUs might not be compatible with this specific combination.

[suyash-narain]

makes sense. thanks