Async support

[rainyl]

Original from: #97

Original discussion:

          > i think having async support built in will be awesome but i dont know if native threads or isolates is better

Yes, but it's not easy, there are much works todo, for dart isolates, the most difficult one is that native resources can't be passed, BUT as you mentioned, they are trying to implement pragma('vm:shared') and I do see that jni has implemented the sharable JObject, but I have no time to explore more.

As for native ways, we need to refactor all APIs to add callbacks, it's crazy and I do not want to refactor them again...

So maybe with "sendable native resources" supported it will be easier, but I do not think it's easy to implement, because native resources created in main isolate will needs to be copied to helper isolate.

If you have better solutions please tell me or try to implement it, I do think we should support async so I have added it to TODO in readme, but it just not easy...

_Originally posted by @rainyl in https://github.com/rainyl/opencv_dart/issues/97#issuecomment-2166077968_

          I think we can take https://pub.dev/packages/image as a reference since he used a different approach, if that doesn't work for our case then I guess people will need to implement their own background handlers same as I did then 

Also if you want to add my app to your readme as examples I don't mind, I am going to test tflite but will try to keep code as modular as I can Incase people want to use it

_Originally posted by @abdelaziz-mahdy in https://github.com/rainyl/opencv_dart/issues/97#issuecomment-2166131113_

You mean the commands? Yes it's another way to implement async, but adding every API to command is laborious and not efficient because many memory copies will be needed, maybe with GAPI implemented this way will be the best option for now.

Also if you want to add my app to your readme as examples I don't mind,

Thanks~ It will be helpful for other developers. I will update the readme to link to your repo.

I am going to test tflite but will try to keep code as modular as I can Incase people want to use it

I remember that opencv 4.9.0 just support a few TFLite operators, I tried many tflite models but opencv can't even load them, that's why I removed TFLite, but in opencv 4.10.0 it seems the support of TFLite is much better, so you can try it now, thanks for your efforts in advance~

@abdelaziz-mahdy Let's discuss async here~

• 124

• 122

• 121

• 125

• 123

[abdelaziz-mahdy]

Ok, will search for a workaround next week and let you know if I was able to find any.

[rainyl]

Thanks~ 😄

[rainyl]

This works, but needs to refactor all APIs again...

// C impl
typedef void (*MatCallback)(void *);
CvStatus gapi_GComputation_apply(GComputation self, Mat in, MatCallback callback /*TODO: GCompileArgs &&args={}*/)
{
  BEGIN_WRAP
  cv::Mat _out;
  (*self.ptr).apply(*in.ptr, _out);
  callback(new cv::Mat(_out));
  END_WRAP
}

// dart impl
Future<Mat> apply(Mat inMat) async {
    final completer = Completer<Mat>();
    late final NativeCallable<cvg.MatCallbackFunction> callback;
    void onResponse(ffi.Pointer<ffi.Void> p) {
      completer.complete(Mat.empty()..ptr.ref.ptr = p);
      callback.close();
    }

    callback = ffi.NativeCallable.listener(onResponse);
    cvRun(() => CFFI.gapi_GComputation_apply(ref, inMat.ref, callback.nativeFunction));
    return completer.future;
  }

// test
test("cv.GComputation", () async {
    {
      final gmIn = cv_gapi.GMat.empty();
      final gmOut = cv_gapi.addMat(gmIn, gmIn);
      final gmOut1 = cv_gapi.addC(gmOut, cv_gapi.GScalar(21));
      final computation = cv_gapi.GComputation.mimo(gmIn, gmOut1);

      final inMat = cv.Mat.ones(10, 10, cv.MatType.CV_8UC1).setTo(cv.Scalar(21, 21, 21, 21));
      final outMat = await computation.apply(inMat);
      expect(outMat.at<int>(0, 1), 63);
      expect(computation.ptr.address, isNonZero);
    }
  });

[abdelaziz-mahdy]

This works, but needs to refactor all APIs again...

// C impl
typedef void (*MatCallback)(void *);
CvStatus gapi_GComputation_apply(GComputation self, Mat in, MatCallback callback /*TODO: GCompileArgs &&args={}*/)
{
  BEGIN_WRAP
  cv::Mat _out;
  (*self.ptr).apply(*in.ptr, _out);
  callback(new cv::Mat(_out));
  END_WRAP
}

// dart impl
Future<Mat> apply(Mat inMat) async {
    final completer = Completer<Mat>();
    late final NativeCallable<cvg.MatCallbackFunction> callback;
    void onResponse(ffi.Pointer<ffi.Void> p) {
      completer.complete(Mat.empty()..ptr.ref.ptr = p);
      callback.close();
    }

    callback = ffi.NativeCallable.listener(onResponse);
    cvRun(() => CFFI.gapi_GComputation_apply(ref, inMat.ref, callback.nativeFunction));
    return completer.future;
  }

// test
test("cv.GComputation", () async {
    {
      final gmIn = cv_gapi.GMat.empty();
      final gmOut = cv_gapi.addMat(gmIn, gmIn);
      final gmOut1 = cv_gapi.addC(gmOut, cv_gapi.GScalar(21));
      final computation = cv_gapi.GComputation.mimo(gmIn, gmOut1);

      final inMat = cv.Mat.ones(10, 10, cv.MatType.CV_8UC1).setTo(cv.Scalar(21, 21, 21, 21));
      final outMat = await computation.apply(inMat);
      expect(outMat.at<int>(0, 1), 63);
      expect(computation.ptr.address, isNonZero);
    }
  });

Looks pretty good, I think no need to refactor? And just make sync and async API? Or you don't think the sync one will be needed.

[rainyl]

Looks pretty good, I think no need to refactor? And just make sync and async API? Or you don't think the sync one will be needed.

Yes they can both exists, but requires two different systems, both for C wrappers (callbacks need to be added) and dart bindings (boilerplate like Completer and Native), it will be a large project, lot's of works todo.

[abdelaziz-mahdy]

Looks pretty good, I think no need to refactor? And just make sync and async API? Or you don't think the sync one will be needed.

Yes they can both exists, but requires two different systems, both for C wrappers (callbacks need to be added) and dart bindings (boilerplate like Completer and Native), it will be a large project, lot's of works todo.

If you plan to do it, I would like to help with it , maybe a pr with checklist where we check the finished tasks

[rainyl]

Looks pretty good, I think no need to refactor? And just make sync and async API? Or you don't think the sync one will be needed.

Yes they can both exists, but requires two different systems, both for C wrappers (callbacks need to be added) and dart bindings (boilerplate like Completer and Native), it will be a large project, lot's of works todo.

If you plan to do it, I would like to help with it , maybe a pr with checklist where we check the finished tasks

Yes, it's not easy but definitely useful and many users needs it, although I have no much time recently, I will open a PR as you said, let's do it 😄