pytorch_lite: ^3.0.0-alpha

[maheralzoubi]

This is not recognized getImagePredictionFromBytesList in last version

[abdelaziz-mahdy]

yes check camera example its a different logic now

[abdelaziz-mahdy]

Keep in mind the branch for the alpha release is

testing ffi

[maheralzoubi97]

@zezo357 Yes, I know it is in the testing phase But the problem I'm having is not with the camera, it's in pytorch_lite See the picture

but when convert from pytorch_lite: ^3.0.0-alpha2 to pytorch_lite: the problem disappears

[abdelaziz-mahdy]

As I was saying this a breaking change check testing ffi camera example for the new way of handling camera

The method there was used for camera and now it's removed for another way

[abdelaziz-mahdy]

 runClassification(Uint8List jpgBytes) async {
    if (_imageModel != null) {
      String imageClassification =
          await _imageModel!.getImagePrediction(jpgBytes);

      print("imageClassification $imageClassification");
      widget.resultsCallbackClassification(imageClassification);
    }
  }

  Future<Uint8List?> convertCameraImageToJpg(CameraImage cameraImage) async {
    Command command = Command()
      ..image(ImageUtils.processCameraImage(cameraImage)!)..encodeJpg();

    Uint8List? bytes = await command.getBytes();
    return bytes;
  }

  Future<void> runObjectDetection(Uint8List jpgBytes) async {
    if (_objectModel != null) {
      List<ResultObjectDetection?> objDetect =
          await _objectModel!.getImagePrediction(
        jpgBytes,
        minimumScore: 0.3,
        iOUThreshold: 0.3,
      );

      print("data outputted $objDetect");
      widget.resultsCallback(objDetect);
    }
  }

  /// Callback to receive each frame [CameraImage] perform inference on it
  onLatestImageAvailable(CameraImage cameraImage) async {
    if (predicting) {
      return;
    }
    predicting = true;
    Uint8List jpgBytes =(await convertCameraImageToJpg(cameraImage))!;

    var futures = <Future>[];
    futures.add(runClassification(jpgBytes));
    futures.add(runObjectDetection(jpgBytes));
    await Future.wait(futures);

    predicting = false;
  }

[maheralzoubi]

thanks @zezo357 i try this code It has been working I tried the version on a real phone But when I click on the camera Screen remains black

[abdelaziz-mahdy]

thanks @zezo357 i try this code It has been working I tried the version on a real phone But when I click on the camera Screen remains black

As I mentioned, this is still in progress, also if you tried again it should work

[maheralzoubi]

ok i try

[maheralzoubi]

I've tried, same problem

[abdelaziz-mahdy]

It may take sometime to fix it , I am still working on it

But detection and classification using images working correctly.

[abdelaziz-mahdy]

check new release 3

https://github.com/zezo357/pytorch_lite/tree/testing-ffi/example/lib is the example used, it should be much better since its using isolates and i did include error handling for permissions

[maheralzoubi]

Thank you for your effort At the moment I do not have a real phone On Sunday I will give it a go And tell you @zezo357

[maheralzoubi]

I tried this version I opened the camera and a black screen did not appear But didn't work

@zezo357

[abdelaziz-mahdy]

I tried this version I opened the camera and a black screen did not appear But didn't work

@zezo357

Please share your logs,and try with version 4 and use latest code provided in the link above

[abdelaziz-mahdy]

Also @maheralzoubi let me know what device are you testing on, and your flutter doctor output

[maheralzoubi97]

@zezo357 Doctor summary (to see all details, run flutter doctor -v): [✓] Flutter (Channel stable, 3.7.12, on macOS 13.3.1 22E261 darwin-arm64 (Rosetta), locale en-JO) [✓] Android toolchain - develop for Android devices (Android SDK version 32.0.0) [✓] Xcode - develop for iOS and macOS (Xcode 14.3) [✓] Chrome - develop for the web [✓] Android Studio (version 2022.2) [✓] VS Code (version 1.78.2) Scanning for devices is taking a long time...[✓] Connected device (3 available) [✓] HTTP Host Availability

• No issues found! device : iphone x

[abdelaziz-mahdy]

I tried this version I opened the camera and a black screen did not appear But didn't work

@zezo357

Please share your logs,and try with version 4 and use latest code provided in the link above

@maheralzoubi97

[maheralzoubi]

I can't connect the real phone to the laptop @zezo357

[abdelaziz-mahdy]

I can't connect the real phone to the laptop @zezo357

Using an emulator won't work, they don't have camera.

This is why the camera is always black, that's not because of the package this is related to camera package.

[maheralzoubi]

No, I'm installing the app by ipa on a real phone But I can't connect the phone to my personal laptop Because there is a problem with the phone

@zezo357

[abdelaziz-mahdy]

No, I'm installing the app by ipa on a real phone But I can't connect the phone to my personal laptop Because there is a problem with the phone

@zezo357

In that case I can't figure out what the problem may be, since I can't see the logs and can't replicate it

Did you use the last code in the link? It should show the error on the screen if it's a problem with the camera logic

Can you clone the testing-ffi branch and test that example?

[maheralzoubi]

@zezo357 flutter: PytorchFfi initialization flutter: ImageUtilsIsolate initialization flutter: Error is Invalid argument(s): Failed to lookup symbol 'load_ml_model': dlsym(RTLD_DEFAULT, load_ml_model): symbol not found The selected imageFormatGroup is not supported by iOS. Defaulting to brga8888 [VERBOSE-2:dart_vm_initializer.cc(41)] Unhandled Exception: RangeError (index): Invalid value: Only valid value is 0: 1

0 _Array.[] (dart:core-patch/array.dart:10:36)

1 ImageUtilsIsolate.convertCameraImageToBytes

image_utils_isolate.dart:65

2 _CameraViewState.onLatestImageAvailable

camera_view.dart:195

3 CameraController.startImageStream.

camera_controller.dart:469

4 _RootZone.runUnaryGuarded (dart:async/zone.dart:1593:10)

[log] will start prediction

5 _BufferingStreamSubscription._sendData (dart:async/stream_impl.dart:339:11)

6 _DelayedData.perform (dart:async/stream_impl.dart:515:14)

7 _PendingEvents.handleNext (dart:async/stream_impl.dart:620:11)

8 _PendingEvents.schedule. (dart:async/stream_impl.dart:591:7)

9 _microtaskLoop (dart:async/schedule_microtask.dart:40:21)

10 _startMicrotaskLoop (dart:async/schedule_microtask.dart:49:5)

[maheralzoubi]

@

[abdelaziz-mahdy]

@zezo357 flutter: PytorchFfi initialization flutter: ImageUtilsIsolate initialization flutter: Error is Invalid argument(s): Failed to lookup symbol 'load_ml_model': dlsym(RTLD_DEFAULT, load_ml_model): symbol not found The selected imageFormatGroup is not supported by iOS. Defaulting to brga8888 [VERBOSE-2:dart_vm_initializer.cc(41)] Unhandled Exception: RangeError (index): Invalid value: Only valid value is 0: 1

0 _Array.[] (dart:core-patch/array.dart:10:36)

1 ImageUtilsIsolate.convertCameraImageToBytes

image_utils_isolate.dart:65

2 _CameraViewState.onLatestImageAvailable

camera_view.dart:195

3 CameraController.startImageStream.

camera_controller.dart:469

4 _RootZone.runUnaryGuarded (dart:async/zone.dart:1593:10)

[log] will start prediction

5 _BufferingStreamSubscription._sendData (dart:async/stream_impl.dart:339:11)

6 _DelayedData.perform (dart:async/stream_impl.dart:515:14)

7 _PendingEvents.handleNext (dart:async/stream_impl.dart:620:11)

8 _PendingEvents.schedule. (dart:async/stream_impl.dart:591:7)

9 _microtaskLoop (dart:async/schedule_microtask.dart:40:21)

10 _startMicrotaskLoop (dart:async/schedule_microtask.dart:49:5)

Thank you very much, now I know where is the problem, just a question you did flutter clean before building the app right?

Just to make sure it's not a flutter building thing

[abdelaziz-mahdy]

And terminate the app and relaunch after the flutter clean

[maheralzoubi]

@zezo357 yes i make flutter clean

[abdelaziz-mahdy]

@zezo357 yes i make flutter clean

It's working on iOS simulator, so I will check again maybe it's a real device limitations,

Hope I find a fix

[abdelaziz-mahdy]

as you can see it works correctly on the example, so i cant figure out what is the problem you are facing

[maheralzoubi]

Yes, when I upload a picture from the gallery, it works correctly but when i real time camera show this error @zezo357

[abdelaziz-mahdy]

Yes, when I upload a picture from the gallery, it works correctly but when i real time camera show this error @zezo357

The error states that the ffi is not loaded in the case of gallery and camera they both use same logic Can you recheck your permissions? And logs

[maheralzoubi]

Yes, I have checked all permissions does not work when real time camera

@zezo357

[abdelaziz-mahdy]

Yes, I have checked all permissions does not work when real time camera

@zezo357

Please provide logs.

And screenshot the results in the app,

And please test on example app

[abdelaziz-mahdy]

I have decided that it should be released since I haven't heard from you for a while, and it works as expected in my iOS simulator.

if you can provide better logs i would gladly help,