NullPointerException when running ImageClassifier Flutter Example

[Raghvender1205]

Hi, I ran the latest code with my custom yolov8 model. But I got this error.

E/MethodChannel#ultralytics_yolo(20455): Failed to handle method call
E/MethodChannel#ultralytics_yolo(20455): java.lang.NullPointerException: Attempt to invoke virtual method 'void androidx.camera.lifecycle.ProcessCameraProvider.unbindAll()' on a null object reference
E/MethodChannel#ultralytics_yolo(20455):    at com.ultralytics.ultralytics_yolo.CameraPreview.bindPreview(CameraPreview.java:79)
E/MethodChannel#ultralytics_yolo(20455):    at com.ultralytics.ultralytics_yolo.CameraPreview.setCameraFacing(CameraPreview.java:97)
E/MethodChannel#ultralytics_yolo(20455):    at com.ultralytics.ultralytics_yolo.MethodCallHandler.setLensDirection(MethodCallHandler.java:260)
E/MethodChannel#ultralytics_yolo(20455):    at com.ultralytics.ultralytics_yolo.MethodCallHandler.onMethodCall(MethodCallHandler.java:95)
E/MethodChannel#ultralytics_yolo(20455):    at io.flutter.plugin.common.MethodChannel$IncomingMethodCallHandler.onMessage(MethodChannel.java:267)
E/MethodChannel#ultralytics_yolo(20455):    at io.flutter.embedding.engine.dart.DartMessenger.invokeHandler(DartMessenger.java:292)
E/MethodChannel#ultralytics_yolo(20455):    at io.flutter.embedding.engine.dart.DartMessenger.lambda$dispatchMessageToQueue$0$io-flutter-embedding-engine-dart-DartMessenger(DartMessenger.java:319)
E/MethodChannel#ultralytics_yolo(20455):    at io.flutter.embedding.engine.dart.DartMessenger$$ExternalSyntheticLambda0.run(Unknown Source:12)
E/MethodChannel#ultralytics_yolo(20455):    at android.os.Handler.handleCallback(Handler.java:958)
E/MethodChannel#ultralytics_yolo(20455):    at android.os.Handler.dispatchMessage(Handler.java:99)
E/MethodChannel#ultralytics_yolo(20455):    at android.os.Looper.loopOnce(Looper.java:257)
E/MethodChannel#ultralytics_yolo(20455):    at android.os.Looper.loop(Looper.java:368)
E/MethodChannel#ultralytics_yolo(20455):    at android.app.ActivityThread.main(ActivityThread.java:8826)
E/MethodChannel#ultralytics_yolo(20455):    at java.lang.reflect.Method.invoke(Native Method)
E/MethodChannel#ultralytics_yolo(20455):    at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:572)
E/MethodChannel#ultralytics_yolo(20455):    at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1049)
E/flutter (20455): [ERROR:flutter/runtime/dart_vm_initializer.cc(41)] Unhandled Exception: PlatformException(error, Attempt to invoke virtual method 'void androidx.camera.lifecycle.ProcessCameraProvider.unbindAll()' on a null object reference, null, java.lang.NullPointerException: Attempt to invoke virtual method 'void androidx.camera.lifecycle.ProcessCameraProvider.unbindAll()' on a null object reference
E/flutter (20455):  at com.ultralytics.ultralytics_yolo.CameraPreview.bindPreview(CameraPreview.java:79)
E/flutter (20455):  at com.ultralytics.ultralytics_yolo.CameraPreview.setCameraFacing(CameraPreview.java:97)
E/flutter (20455):  at com.ultralytics.ultralytics_yolo.MethodCallHandler.setLensDirection(MethodCallHandler.java:260)
E/flutter (20455):  at com.ultralytics.ultralytics_yolo.MethodCallHandler.onMethodCall(MethodCallHandler.java:95)
E/flutter (20455):  at io.flutter.plugin.common.MethodChannel$IncomingMethodCallHandler.onMessage(MethodChannel.java:267)
E/flutter (20455):  at io.flutter.embedding.engine.dart.DartMessenger.invokeHandler(DartMessenger.java:292)
E/flutter (20455):  at io.flutter.embedding.engine.dart.DartMessenger.lambda$dispatchMessageToQueue$0$io-flutter-embedding-engine-dart-DartMessenger(DartMessenger.java:319)
E/flutter (20455):  at io.flutter.embedding.engine.dart.DartMessenger$$ExternalSyntheticLambda0.run(Unknown Source:12)
E/flutter (20455):  at android.os.Handler.handleCallback(Handler.java:958)
E/flutter (20455):  at android.os.Handler.dispatchMessage(Handler.java:99)
E/flutter (20455):  at android.os.Looper.loopOnce(Looper.java:257)
E/flutter (20455):  at android.os.Looper.loop(Looper.java:368)
E/flutter (20455):  at android.app.ActivityThread.main(ActivityThread.java:8826)
E/flutter (20455):  at java.lang.reflect.Method.invoke(Native Method)
E/flutter (20455):  at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:572)
E/flutter (20455):  at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1049)
E/flutter (20455): )
E/flutter (20455): #0      StandardMethodCodec.decodeEnvelope (package:flutter/src/services/message_codecs.dart:648:7)
E/flutter (20455): #1      MethodChannel._invokeMethod (package:flutter/src/services/platform_channel.dart:334:18)
E/flutter (20455): <asynchronous suspension>
E/flutter (20455): #2      UltralyticsYoloCameraController.toggleLensDirection (package:ultralytics_yolo/camera_preview/ultralytics_yolo_camera_controller.dart:48:5)
E/flutter (20455): <asynchronous suspension>
E/flutter (20455): 

This is the code I am using

import 'dart:io' as io;

import 'package:flutter/material.dart';
import 'package:path/path.dart';
import 'package:path_provider/path_provider.dart';
import 'package:flutter/services.dart';
import 'package:permission_handler/permission_handler.dart';
import 'package:ultralytics_yolo/ultralytics_yolo.dart';
import 'package:ultralytics_yolo/yolo_model.dart';

void main() {
  runApp(const MyApp());
}

class MyApp extends StatefulWidget {
  const MyApp({super.key});

  @override
  State<MyApp> createState() => _MyAppState();
}

class _MyAppState extends State<MyApp> {
  final controller = UltralyticsYoloCameraController();

  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      home: Scaffold(
        body: FutureBuilder<bool>(
          future: _checkPermissions(),
          builder: (context, snapshot) {
            final allPermissionGranted = snapshot.data ?? false;

            return !allPermissionGranted
                ? Container()
                : FutureBuilder<ImageClassifier>(
              future: _initImageClassifierWithLocalModel(),
              builder: (context, snapshot) {
                final predictor = snapshot.data;

                return predictor == null
                    ? Container()
                    : Stack(
                  children: [
                    UltralyticsYoloCameraPreview(
                      controller: controller,
                      predictor: predictor,
                      onCameraCreated: () {
                        predictor.loadModel();
                      },
                    ),
                    StreamBuilder<double?>(
                      stream: predictor.inferenceTime,
                      builder: (context, snapshot) {
                        final inferenceTime = snapshot.data;

                        return StreamBuilder<double?>(
                          stream: predictor.fpsRate,
                          builder: (context, snapshot) {
                            final fpsRate = snapshot.data;

                            return Times(
                              inferenceTime: inferenceTime,
                              fpsRate: fpsRate,
                            );
                          },
                        );
                      },
                    ),
                  ],
                );
              },
            );
          },
        ),
        floatingActionButton: FloatingActionButton(
          child: const Icon(Icons.abc),
          onPressed: () {
            controller.toggleLensDirection();
          },
        ),
      ),
    );
  }

  Future<ImageClassifier> _initImageClassifierWithLocalModel() async {
    final modelPath = await _copy('assets/glare_classify.tflite');
    final metadataPath = await _copy('assets/metadata.yaml');
    final model = LocalYoloModel(
      id: '',
      task: Task.classify,
      format: Format.tflite,
      modelPath: modelPath,
      metadataPath: metadataPath,
    );

    return ImageClassifier(model: model);
  }

  Future<String> _copy(String assetPath) async {
    final path = '${(await getApplicationSupportDirectory()).path}/$assetPath';
    await io.Directory(dirname(path)).create(recursive: true);
    final file = io.File(path);

    if (!await file.exists()) {
      final byteData = await rootBundle.load(assetPath);
      await file.writeAsBytes(byteData.buffer
        .asUint8List(byteData.offsetInBytes, byteData.lengthInBytes));
    }

    return file.path;
  }

  Future<bool> _checkPermissions() async {
    List<Permission> permissions = [];

    var cameraStatus = await Permission.camera.status;
    if (!cameraStatus.isGranted) {
      permissions.add(Permission.camera);
    }

    var storageStatus = await Permission.storage.status;
    if (!storageStatus.isGranted) {
      permissions.add(Permission.storage);
    }

    if (permissions.isEmpty) {
      return true;
    } else {
      try {
        Map<Permission, PermissionStatus> statuses =
            await permissions.request();

        return statuses[Permission.camera] == PermissionStatus.granted &&
              statuses[Permission.storage] == PermissionStatus.granted;
      } on Exception catch(_) {
        return false;
      }
    }
  }
}

class Times extends StatelessWidget {
  const Times({
    super.key,
    required this.inferenceTime,
    required this.fpsRate,
  });

  final double? inferenceTime;
  final double? fpsRate;

  @override
  Widget build(BuildContext context) {
    return SafeArea(
      child: Align(
        alignment: Alignment.bottomCenter,
        child: Container(
          margin: const EdgeInsets.all(20),
          padding: const EdgeInsets.all(20),
          decoration: const BoxDecoration(
            borderRadius: BorderRadius.all(Radius.circular(10)),
            color: Colors.black54,
          ),
          child: Text(
            '${(inferenceTime ?? 0).toStringAsFixed(1)} ms - ${(fpsRate ?? 0).toStringAsFixed(1)} FPS',
            style: const TextStyle(color: Colors.white70),
          )),
      ),
    );
  }
}

Any suggestions?

[pderrenger]

@Raghvender1205 hello! Thanks for reaching out with the details of the issue you're encountering. Based on the error message, it seems that the ProcessCameraProvider is not properly initialized before it's being accessed, leading to a NullPointerException.

This can happen if the camera initialization process has not completed or failed for some reason, but further operations are attempted on it. It's crucial to ensure that the camera is fully ready before trying to bind previews or set camera facing.

A potential solution is to add a check that ensures ProcessCameraProvider is not null before attempting to call unbindAll() or any other methods on it. This can be done by simply wrapping the calls in an if-statement to check if the provider is not null.

if (processCameraProvider != null) {
    processCameraProvider.unbindAll();
}

Make sure to apply this null check wherever ProcessCameraProvider is accessed in your code. Also, consider implementing proper error handling for camera initialization to handle cases where it fails or takes longer than expected.

For more detailed guidelines and best practices on handling camera in Flutter, you might want to refer to the Ultralytics Docs. We're continuously working to improve documentation and provide practical examples to handle common issues.

Let me know if this helps, or if there are any more questions or issues you encounter. The community and we here at Ultralytics are always here to help! 😊

[github-actions[bot]]

👋 Hello there! We wanted to give you a friendly reminder that this issue has not had any recent activity and may be closed soon, but don't worry - you can always reopen it if needed. If you still have any questions or concerns, please feel free to let us know how we can help.

For additional resources and information, please see the links below:

• Docs: https://docs.ultralytics.com
• HUB: https://hub.ultralytics.com
• Community: https://community.ultralytics.com

Feel free to inform us of any other issues you discover or feature requests that come to mind in the future. Pull Requests (PRs) are also always welcomed!

Thank you for your contributions to YOLO 🚀 and Vision AI ⭐

[huats]

It seems related to #38

[glenn-jocher]

@huats please ensure you're using the latest version of the package, as updates may resolve this issue. If the problem persists, try checking the initialization of ProcessCameraProvider to ensure it's not null before calling unbindAll().