example app doesn't work

[fransay]

The example app only displays a screen with a white background color and a floating action button with the inscription ABC. On click of the floating action button, nothing happens and there seems to be no way to get the camera preview launch on screen for detection. I don't know if there are any configurations for me to do in order to get this working, any feedback and help on will be met with a ton of gratitude.

[pderrenger]

@fransay hello!

Thank you for reaching out and providing details about the issue you're experiencing with the example app. Let's work together to get this resolved.

First, could you please ensure that you are using the most recent versions of both torch and ultralytics? Sometimes, updating these packages can resolve unexpected issues. You can update them using the following commands:

pip install --upgrade torch ultralytics

If the issue persists after updating, it would be very helpful if you could provide a minimum reproducible code example. This will allow us to better understand the problem and investigate a solution. You can find guidance on how to create a minimum reproducible example here: Minimum Reproducible Example.

Additionally, please make sure you have followed all the steps outlined in our Ultralytics Android App Documentation. Specifically, ensure that you have granted the necessary permissions for the app to access your device's camera.

Here's a quick checklist to verify:

1. Permissions: Ensure the app has permission to access the camera.
2. Model Selection: Make sure you have selected a trained YOLO model from the list.
3. App Settings: Check the app settings to adjust detection thresholds and enable specific object classes if needed.

If you have already checked these and the issue still occurs, please share any relevant code snippets or logs that might help us diagnose the problem further.

Looking forward to your response so we can assist you better!

[fransay]

@pderrenger , thank you for your timely and prompt feedback! I will get the help suggestions tested and get back with response ASAP!

[pderrenger]

@fransay, thank you for your quick response! 😊

We're here to help you get the example app up and running smoothly. Please make sure to test the suggestions provided earlier, including updating to the latest versions of torch and ultralytics. This often resolves many unexpected issues.

If you encounter any further problems, it would be incredibly helpful if you could provide a minimum reproducible code example. This allows us to better understand the issue and investigate a solution more effectively. You can find guidance on how to create one here: Minimum Reproducible Example.

Looking forward to your feedback and hoping we can get everything working perfectly for you! 🚀

[congngc]

Hello @pderrenger,

I am encountering a similar issue as described in this GitHub issue, but the provided solution has not resolved my problem. The issue I'm facing is detailed in the screenshot below and in the following description: In the example code provided, there are two critical segments:

1. _MyAppState Class: This class includes a FutureBuilder for checking permissions and initializing the object detector. However, I'm experiencing a problem where the Permission.camera.status is granted, but Permission.storage.status is denied.

2. _checkPermissions Method: This method checks for camera and storage permissions. While the camera permission is successfully granted, the storage permission request fails. The method is designed to return true only if all permissions are granted, which isn't happening due to the storage permission issue.

The main problem is within the _checkPermissions() function, where Permission.camera.status works fine, but Permission.storage gets denied. Additionally, the logic of the FutureBuilder in the _MyAppState class might need reevaluation:

FutureBuilder<bool>(
  future: _checkPermissions(),
  builder: (context, snapshot) {
    final allPermissionsGranted = snapshot.data ?? false;
    return !allPermissionsGranted ? Container() : FutureBuilder<ObjectDetector>();
  },
)

Here is the related part of the code for your reference:

class _MyAppState extends State<MyApp> {
  final controller = UltralyticsYoloCameraController();

  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      home: Scaffold(
        body: FutureBuilder<bool>(
          future: _checkPermissions(),
          builder: (context, snapshot) {
            final allPermissionsGranted = snapshot.data ?? false;

            return !allPermissionsGranted
                ? Container()
                : FutureBuilder<ObjectDetector>(
                    future: _initObjectDetectorWithLocalModel(),
                    builder: (context, snapshot) {
                      final predictor = snapshot.data;

                      return predictor == null
                          ? Container()
                          : Stack(
                              children: [
                                UltralyticsYoloCameraPreview(
                                  controller: controller,
                                  predictor: predictor,
                                  onCameraCreated: () {
                                    predictor.loadModel(useGpu: true);
                                  },
                                ),
                                StreamBuilder<double?>(
                                  stream: predictor.inferenceTime,
                                  builder: (context, snapshot) {
                                    final inferenceTime = snapshot.data;

                                    return StreamBuilder<double?>(
                                      stream: predictor.fpsRate,
                                      builder: (context, snapshot) {
                                        final fpsRate = snapshot.data;

                                        return Times(
                                          inferenceTime: inferenceTime,
                                          fpsRate: fpsRate,
                                        );
                                      },
                                    );
                                  },
                                ),
                              ],
                            );
                    },
                  );
          },
        ),
        floatingActionButton: FloatingActionButton(
          child: const Icon(Icons.abc),
          onPressed: () {
            controller.toggleLensDirection();
          },
        ),
      ),
    );
  }
}

Could you please assist me in resolving this issue or provide guidance on how to ensure that all necessary permissions are successfully requested and granted?

Thank you for your assistance.

[pderrenger]

Hello @congngc,

Thank you for providing detailed information about the issue you're encountering. Let's work through this together to find a solution.

Permissions Issue

It seems the main problem lies in the storage permission not being granted. Ensuring that all necessary permissions are requested and granted is crucial for the app to function correctly. Here’s a refined approach to handle permissions:

1. Request Permissions: Make sure you are requesting both camera and storage permissions properly.
2. Check Permissions: Verify that the permissions are granted before proceeding.

Here’s an updated version of the _checkPermissions method to ensure both permissions are requested and handled correctly:

Future<bool> _checkPermissions() async {
  final cameraStatus = await Permission.camera.status;
  final storageStatus = await Permission.storage.status;

  if (!cameraStatus.isGranted) {
    await Permission.camera.request();
  }
  if (!storageStatus.isGranted) {
    await Permission.storage.request();
  }

  return cameraStatus.isGranted && storageStatus.isGranted;
}

FutureBuilder Logic

The FutureBuilder logic in your _MyAppState class looks mostly correct, but let's ensure it handles the permissions properly. Here’s a slightly adjusted version:

class _MyAppState extends State<MyApp> {
  final controller = UltralyticsYoloCameraController();

  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      home: Scaffold(
        body: FutureBuilder<bool>(
          future: _checkPermissions(),
          builder: (context, snapshot) {
            if (snapshot.connectionState == ConnectionState.waiting) {
              return Center(child: CircularProgressIndicator());
            }

            final allPermissionsGranted = snapshot.data ?? false;

            if (!allPermissionsGranted) {
              return Center(child: Text('Permissions not granted'));
            }

            return FutureBuilder<ObjectDetector>(
              future: _initObjectDetectorWithLocalModel(),
              builder: (context, snapshot) {
                if (snapshot.connectionState == ConnectionState.waiting) {
                  return Center(child: CircularProgressIndicator());
                }

                final predictor = snapshot.data;

                if (predictor == null) {
                  return Center(child: Text('Failed to initialize predictor'));
                }

                return Stack(
                  children: [
                    UltralyticsYoloCameraPreview(
                      controller: controller,
                      predictor: predictor,
                      onCameraCreated: () {
                        predictor.loadModel(useGpu: true);
                      },
                    ),
                    StreamBuilder<double?>(
                      stream: predictor.inferenceTime,
                      builder: (context, snapshot) {
                        final inferenceTime = snapshot.data;

                        return StreamBuilder<double?>(
                          stream: predictor.fpsRate,
                          builder: (context, snapshot) {
                            final fpsRate = snapshot.data;

                            return Times(
                              inferenceTime: inferenceTime,
                              fpsRate: fpsRate,
                            );
                          },
                        );
                      },
                    ),
                  ],
                );
              },
            );
          },
        ),
        floatingActionButton: FloatingActionButton(
          child: const Icon(Icons.abc),
          onPressed: () {
            controller.toggleLensDirection();
          },
        ),
      ),
    );
  }
}

Additional Steps

1. Update Packages: Ensure you are using the latest versions of all relevant packages. You can update them using:

   flutter pub upgrade

2. Reproducible Example: If the issue persists, providing a minimum reproducible example would be very helpful. You can find guidance on how to create one here: Minimum Reproducible Example.

I hope this helps! Please let me know if you have any further questions or if the issue persists. We're here to assist you. 😊

[fransay]

Hi @pderrenger, thanks for sharing the above modified code snippet.

There are couple of bugs in the above snippet that could be fixed; here are they! @congngc keep on this, it works for me.

1. Remove storage permissions There is little or no need for the storage permission here in this code, remember you are only performing inference and not necessarily storing bytes to any memory block or disk space continually. Hence no need for storage permissions.

   Future<bool> _checkPermissions() async {
   final cameraStatus = await Permission.camera.status;
   final storageStatus = await Permission.storage.status;
   
   if (!cameraStatus.isGranted) {
   await Permission.camera.request();
   }
   if (!storageStatus.isGranted) {
   await Permission.storage.request();
   }
   
   return cameraStatus.isGranted && storageStatus.isGranted;
   }

   The above code should now look like this

   Future<bool> _checkPermissions() async {
   final cameraStatus = await Permission.camera.status;
   // final storageStatus = await Permission.storage.status;
   
   if (!cameraStatus.isGranted) {
     await Permission.camera.request();
   }
   // if (!storageStatus.isGranted) {
   //   await Permission.storage.request();
   // }
   
   return cameraStatus.isGranted;
   }

2. Remove if block from app state class builder method It is an unnecessary check, since a similar operation will be done in the FutureBuilder few lines down the lines. Something similar to the below code block should work!

   class _MyAppState extends State<MyApp> {
   final controller = UltralyticsYoloCameraController();
   
   @override
   Widget build(BuildContext context) {
   return MaterialApp(
     home: Scaffold(
       body: FutureBuilder<bool>(
         future: _checkPermissions(),
         builder: (context, snapshot) {
           // if (snapshot.connectionState == ConnectionState.waiting) {
           //   return Center(child: CircularProgressIndicator());
           // }
   
           final allPermissionsGranted = snapshot.data ?? false;
   
           if (!allPermissionsGranted) {
             return Center(child: Text('Permissions not granted'));
           }
   
           return FutureBuilder<ObjectDetector>(
             future: _initObjectDetectorWithLocalModel(),
             builder: (context, snapshot) {
               if (snapshot.connectionState == ConnectionState.waiting) {
                 return Center(child: CircularProgressIndicator());
               }
   
               final predictor = snapshot.data;
   
               if (predictor == null) {
                 return Center(child: Text('Failed to initialize predictor'));
               }
   
               return Stack(
                 children: [
                   UltralyticsYoloCameraPreview(
                     controller: controller,
                     predictor: predictor,
                     onCameraCreated: () {
                       predictor.loadModel(useGpu: true);
                     },
                   ),
                   StreamBuilder<double?>(
                     stream: predictor.inferenceTime,
                     builder: (context, snapshot) {
                       final inferenceTime = snapshot.data;
   
                       return StreamBuilder<double?>(
                         stream: predictor.fpsRate,
                         builder: (context, snapshot) {
                           final fpsRate = snapshot.data;
   
                           return Times(
                             inferenceTime: inferenceTime,
                             fpsRate: fpsRate,
                           );
                         },
                       );
                     },
                   ),
                 ],
               );
             },
           );
         },
       ),
       floatingActionButton: FloatingActionButton(
         child: const Icon(Icons.abc),
         onPressed: () {
           controller.toggleLensDirection();
         },
       ),
     ),
   );
   }
   }

Hope this helps.!

[pderrenger]

Hi @fransay,

Thank you for your insightful feedback and for sharing the improvements! Your suggestions are indeed valuable and can help streamline the code further. Let's address the points you've raised:

1. Removing Storage Permissions

You are correct that if the app is only performing inference and not storing data, the storage permission is unnecessary. Here’s the updated _checkPermissions method without the storage permission:

Future<bool> _checkPermissions() async {
  final cameraStatus = await Permission.camera.status;

  if (!cameraStatus.isGranted) {
    await Permission.camera.request();
  }

  return cameraStatus.isGranted;
}

2. Simplifying the FutureBuilder Logic

Removing the redundant if block in the FutureBuilder can indeed make the code cleaner. Here’s the refined version of the _MyAppState class:

class _MyAppState extends State<MyApp> {
  final controller = UltralyticsYoloCameraController();

  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      home: Scaffold(
        body: FutureBuilder<bool>(
          future: _checkPermissions(),
          builder: (context, snapshot) {
            final allPermissionsGranted = snapshot.data ?? false;

            if (!allPermissionsGranted) {
              return Center(child: Text('Permissions not granted'));
            }

            return FutureBuilder<ObjectDetector>(
              future: _initObjectDetectorWithLocalModel(),
              builder: (context, snapshot) {
                if (snapshot.connectionState == ConnectionState.waiting) {
                  return Center(child: CircularProgressIndicator());
                }

                final predictor = snapshot.data;

                if (predictor == null) {
                  return Center(child: Text('Failed to initialize predictor'));
                }

                return Stack(
                  children: [
                    UltralyticsYoloCameraPreview(
                      controller: controller,
                      predictor: predictor,
                      onCameraCreated: () {
                        predictor.loadModel(useGpu: true);
                      },
                    ),
                    StreamBuilder<double?>(
                      stream: predictor.inferenceTime,
                      builder: (context, snapshot) {
                        final inferenceTime = snapshot.data;

                        return StreamBuilder<double?>(
                          stream: predictor.fpsRate,
                          builder: (context, snapshot) {
                            final fpsRate = snapshot.data;

                            return Times(
                              inferenceTime: inferenceTime,
                              fpsRate: fpsRate,
                            );
                          },
                        );
                      },
                    ),
                  ],
                );
              },
            );
          },
        ),
        floatingActionButton: FloatingActionButton(
          child: const Icon(Icons.abc),
          onPressed: () {
            controller.toggleLensDirection();
          },
        ),
      ),
    );
  }
}

Additional Notes

• Updating Packages: Ensure you are using the latest versions of all relevant packages. You can update them using:

  flutter pub upgrade

• Reproducible Example: If the issue persists, providing a minimum reproducible example would be very helpful. You can find guidance on how to create one here: Minimum Reproducible Example.

Thank you again for your contributions to the community! If you have any further questions or need additional assistance, feel free to ask. We're here to help! 😊

[m-aliabbas]

Thanks @pderrenger This issue is resolved. Application is working. Object Detection accuracy is really low.