latest ffi commit

[abdelaziz-mahdy]

FFI Integration Tracking PR

Overview

This PR is intended for tracking the integration of the Foreign Function Interface (FFI) with our current system. It includes the latest FFI commit available.

Current State

• The package currently utilizes native methods and communication established through Pigeon.
• On iOS, C++ files are not automatically bundled during app builds. This necessitates users of the package to manually add these files to their apps. This approach is inefficient and does not ensure the delivery of the latest updates to the apps.

Setup and Progress

Completed Tasks

• Integration of OpenCV for image and camera image processing.
• Linking FFI with the processing components.

Pending or Previously Completed in Native

• iOS does not automatically include C++ files in the builds.
• Dynamic model return based on model sizes.
• Dynamic model return for different YOLOv5 and YOLOv8 models, particularly for sizes other than 640 pixels.

Next Steps

• [ ] Address the issue of iOS not including C++ files automatically.
• [ ] Implement dynamic model returns for varying model sizes.
• [ ] Adapt the system for various YOLO model versions and sizes.

Contribution

Contributors are welcome to assist in addressing the pending tasks. Please ensure you follow the project's contribution guidelines for submitting your changes.

---

Note: This PR is for tracking and discussion purposes and may not include the final implementation directly.

the latest ffi implementation is on the latest-ffi branch mentioned in this pr https://github.com/abdelaziz-mahdy/pytorch_lite/tree/latest-ffi

[cyrillkuettel]

I researched this topic quite a bit. While I don't have a solution yet, I will write down my thoughts here:

The Native assets feature that is currently discussed seems to adresss exactly this issue:

The current way of bundling native code with Dart packages is far from ideal, we will introduce "Native Assets" to Dart.

• Also interesting discussion Flutter tools support for packaging/linking to native libraries

We also could figure out how others are doing it. Many of them package the library into a .framework:

• https://github.com/Sunbreak/cronet_flutter/blob/master/ios/cronet_flutter.podspec
• https://github.com/GregoryConrad/mimir/blob/main/packages/flutter_mimir/ios/flutter_mimir.podspec

Edit:

• https://github.com/flutter/flutter/pull/96225#issuecomment-1319080539
• https://github.com/flutter/flutter/pull/96225#issuecomment-1317994374