google / flutter-mediapipe

Apache License 2.0
212 stars 14 forks source link

[WIP] FFI Implementation #7

Closed craiglabenz closed 1 year ago

craiglabenz commented 1 year ago

Summary

Adds a MediaPipe implementation that talks directly to C++ via FFI. (Technically, indirectly and through a C intermediary, but still fairly directly in comparison to method channels.)

C bridge code uses Dart's native FFI functionality and ffigen to automatically generate the required bindings from C headers pulled from the MediaPipe repository.

TODOs

Note: This is a feature / release branch, which means only fully reviewed changes are able to land here via their own PRs.

TESTING

To run this PR locally, complete the following steps (on macOS):

  1. In a checkout of google/mediapipe with the latest internal changes pulled down, run this build command:
bazel build --config darwin_arm64 --strip always -c opt --define MEDIAPIPE_DISABLE_GPU=1 mediapipe/tasks/c/text/text_classifier:libtext_classifier.dylib
  1. Copy the resulting libtext_classifier.dylib file (found in the bazel-out directory) to this repository at the packages/mediapipe-task-text/examples/assets directory.

Note: This will move up a level to mediapipe-task-text/assets after google/flutter-mediapipe#9.

  1. Download bert_classifier.tflite and place it in the same directory.

  2. Run the example app and click the floating action button to classify text.