Closed mspnr closed 3 years ago
Here is what I came up with. I tried to document everything, that was not so obvious for me and maybe for others. Let me know if something is missing.
Once again here is the start point.
The recommended option is ending up with issue https://github.com/tensorflow/tensorflow/issues/48464, so no outcome here:
bazel build --config=ios_fat -c opt //tensorflow/lite/ios:TensorFlowLiteC_framework
To build version 2.4.1 and earlier I used (by adding experimental/
):
bazel build --config=ios_fat -c opt //tensorflow/lite/experimental/ios:TensorFlowLiteC_framework
To build the latest version for arm64 only (by replacing ios_fat
via ios_arm64
):
bazel build --config=ios_arm64 -c opt //tensorflow/lite/ios:TensorFlowLiteC_framework
The second and the third options produced TensorFlowLiteC_framework.zip
as described here, but running the app ended up with error regarding missing TFLGpuDelegateCreate
symbol.
So I assumed, that GPU delegate metal
function is not enabled in the framework. The idea was to include metal_delegate.h
into the compiled framework.
Unfortunately I didn't find a way to include the GPU delegate metal
function to TensorFlowLiteC_framework
. At the end I stumbled on this line in tensorflow, which gave an idea, that the function can be built separately.
So to build GPU delegate metal
function I used:
bazel build --config=ios_arm64 -c opt //tensorflow/lite/ios:TensorFlowLiteCMetal_framework
And got a separate TensorFlowLiteCMetal_framework.zip
.
On the next step I placed TensorFlowLiteCMetal.framework
into plugin ios
folder and added it into tflite_flutter.podspec
:
s.ios.vendored_frameworks = 'TensorFlowLiteC.framework', 'TensorFlowLiteCMetal.framework'
s.xcconfig = { 'OTHER_LDFLAGS' => '-framework TensorFlowLiteC -all_load -framework TensorFlowLiteCMetal -all_load' }
To apply the changes I cleaned and rebuilt the flutter project:
flutter clean
flutter build ios
After that the error (magically) disappeared. The app is working all right and more than twice faster, than without GPU acceleration.
The option suggested in Readme is working all right:
final gpuDelegate = GpuDelegate(
options: GpuDelegateOptions(true, TFLGpuDelegateWaitType.active),
);
var interpreterOptions = InterpreterOptions()..addDelegate(gpuDelegate);
The other option though mentioned here https://github.com/am15h/tflite_flutter_plugin/issues/60#issuecomment-752367647 and also present in the documentation is not working all right.
@applikationsprogramvara Thank you so much for this great tutorial. I am including this in readme as well.
Is there any example to prove that iOS is recognizing things? I used below function as you said but no faces is recognized at all. The app starts but no faces is recognized when you capture faces. In summary, if you know any examples to prove that iOS is working to realize faces then please let me know. Android is working but iOS.
Future loadModeliOS() async { try { final gpuDelegate = GpuDelegate( options: GpuDelegateOptions(true, TFLGpuDelegateWaitType.active), ); var interpreterOptions = InterpreterOptions()..addDelegate(gpuDelegate); interpreter = await Interpreter.fromAsset('mobilefacenet.tflite', options: interpreterOptions); } on Exception { //print('Failed to load model ios.'); } }
Please checkout @danghoaidanh https://github.com/am15h/object_detection_flutter, it works on iOS and Android both.
Hi @applikationsprogramvara, I have added this tutorial in wiki, https://github.com/am15h/tflite_flutter_plugin/wiki/Build-iOS-binaries-with-GPU-delegate. Please review it and make edits if required. Thanks for your contribution.
@applikationsprogramvara Can you share the TensorFlowLiteCMetal_framework.zip
, for uploading in release assets of this project.
@am15h, @applikationsprogramvara is the TensorFlowLiteCMetal_framework.zip
now available somwhere?
I am having the same issue as described here (#152 ). But for me it is not possible to build the framework from source as i have a very low spec Mac.
I refer to this pr : https://github.com/am15h/tflite_flutter_plugin/pull/200
It is mentioned in the Readme, that this function is available:
If I use the following parameter my app is starting and working OK:
If I use the following parameter the app is starting, but is not working:
In XCode log I see the following error:
Runner[1383:458287] flutter: Error while creating interpreter: Invalid argument(s): Failed to lookup symbol (dlsym(RTLD_DEFAULT, TFLGpuDelegateCreate): symbol not found)
From that I assumed, that an appropriate
TensorFlowLiteC.framework
should be used:So my question is: how to let iOS Metal Delegate work? Do you have an appropriate and compiled
TensorFlowLiteC.framework
? Can you share it?