search

ValYouW / tflite-dist

TensorFlow Lite C/C++ distribution libraries and headers
MIT License
111 stars 19 forks source link

Generate OpenCL backend delegate #10

Closed aitikgupta closed 3 years ago

aitikgupta commented 3 years ago

The series of steps you followed generates (for android), 3 dynamic libs:

  1. libtensorflowlite.so - C++ library
  2. libtensorflowlite_c.so - C library
  3. libtensorflowlite_gpu_gl.so - The C++ GPU delegate library

However, since I believe 2.2 (I believe) release, TFLite supports 2 backends for android, OpenGL (the one you generate) and one OpenCL. OpenCL is the default, and if some device doesn't support it, it automatically falls back to OpenGL backend.

The change in the build process should be:

bazel build -c opt --config android_{abi} tensorflow/lite/delegates/gpu:libtensorflowlite_gpu_delegate.so

instead of

bazel build -c opt --config android_{abi} tensorflow/lite/delegates/gpu:libtensorflowlite_gpu_gl.so
ValYouW commented 3 years ago

Hi @aitikgupta thank you! I'll take a look. I have code using OpenGL and I pass it as input SSBO object (so I avoid cpu/gpu hops), I wonder if the api for the new gpu is the same

aitikgupta commented 3 years ago

The new API is (almost) same. "V2" is appended for some public functions like: TfLiteGpuDelegateV2Create, TfLiteGpuDelegateOptionsV2, etc. Documentation is updated on their website: https://www.tensorflow.org/lite/performance/gpu_advanced#android_cc

Quoting their announcement blog:

In fact, the OpenCL backend has been in the TensorFlow repository since mid 2019 and seamlessly integrated through the TFLite GPU delegate v2, so you might be already using it through the delegate's fallback mechanism.

ValYouW commented 3 years ago

@aitikgupta Released v2.4.1 with the gpu_delegate, thank you for bringing that up....