search

flutter-ml / google_ml_kit_flutter

A flutter plugin that implements Google's standalone ML Kit
MIT License
990 stars 747 forks source link

face_detection not detecting face in android device for plugin camera: ^0.11.0+2 #680

Closed ChanduDA closed 1 month ago

ChanduDA commented 1 month ago

Describe your issue. If applicable, add screenshots to help explain your problem.

I am using OnePlus nord ce2 lite mobile.I have implemented the code as mentioned in the example. But I am getting Face not detected.

Please check the main parts of code below: ` camera_view.dart file => InputImage? _inputImageFromCameraImage(CameraImage image, CameraDescription camera) { if (_controller == null) return null; final sensorOrientation = camera.sensorOrientation; InputImageRotation? rotation; final rotationNew = InputImageRotationValue.fromRawValue(camera.sensorOrientation); print('rotationNew: $rotationNew'); var rotationCompensation = _orientations[_controller?.value.deviceOrientation]; print('rotationCompensation: $rotationCompensation'); if (rotationCompensation == null) return null; if (Platform.isIOS) { rotation = InputImageRotationValue.fromRawValue(sensorOrientation); } else if (Platform.isAndroid) { var rotationCompensation = _orientations[_controller!.value.deviceOrientation]; if (rotationCompensation == null) return null; if (camera.lensDirection == CameraLensDirection.front) { // front-facing rotationCompensation = (sensorOrientation + rotationCompensation) % 360; } else { // back-facing rotationCompensation = (sensorOrientation - rotationCompensation + 360) % 360; } rotation = InputImageRotationValue.fromRawValue(rotationCompensation); // print('rotationCompensation: $rotationCompensation'); } if (rotation == null) return null; print('rotation: $rotation'); var format = InputImageFormatValue.fromRawValue(image.format.raw); print('format: $format'); if (format == null) return null; if ((Platform.isAndroid && format != InputImageFormat.nv21) || (Platform.isIOS && format != InputImageFormat.bgra8888)) { if (Platform.isAndroid) { format = InputImageFormat.nv21; // var newImage = _convertImageToNv(image); } } print('formatNew: $format'); // if (image.planes.length != 1) return null;

return InputImage.fromBytes(
  bytes: Uint8List.fromList(
    image.planes.fold(
        <int>[], (List<int> previousValue, element) => previousValue..addAll(element.bytes)),
  ),
  metadata: InputImageMetadata(
    size: Size(image.width.toDouble(), image.height.toDouble()),
    rotation: rotation, // used only in Android
    format: format,
    bytesPerRow: 0,
  ),
);

}`

` face_detecter_screen.dart: => @override Widget build(BuildContext context) { return Scaffold( /appBar: Platform.isAndroid ? AppBar( actions: [ IconButton( onPressed: () { Navigator.of(context).pop(); }, icon: const Icon(Icons.close)), ], ) : null,/ body: CameraView( customPaint: _customPaint, text: 'camera', onImage: onImage, onCapture: _imageCrop, showButton: showButton, ), ); }

Future onImage(InputImage inputImage) async { if (!_canProcess) return; if (_isBusy) return; _isBusy = true; setState(() { _text = ''; }); print('Meta data: ${inputImage.metadata}'); final faces = await _faceDetector.processImage(inputImage); print('Faces detected: ${faces.length}'); if (faces.isEmpty) { print('faces is empty'); _customPaint = null; setState(() { showButton = false; }); } else if (inputImage.metadata?.size != null && inputImage.metadata?.rotation != null) { if (mounted) { setState(() { showButton = faces.length == 1; }); }

  // face = faces.first;
  final painter =
      FaceDetectorPainter(faces, inputImage.metadata!.size, inputImage.metadata!.rotation);
  _customPaint = CustomPaint(painter: painter);
} else {
  String text = 'Faces found: ${faces.length}\n\n';
  for (final face in faces) {
    text += 'face: ${face.boundingBox}\n\n';
  }
  _text = text;
  // TODO: set _customPaint to draw boundingRect on top of image
  _customPaint = null;
}
_isBusy = false;
if (mounted) {
  setState(() {});
}

} `

Also please check the debug logs: I/flutter (15129): Meta data: Instance of 'InputImageMetadata' V/FaceDetectorV2Jni(15129): detectFacesImageByteArray.start() I/tflite (15129): Replacing 65 out of 65 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 1 partitions for the whole graph. I/flutter (15129): rotationNew: InputImageRotation.rotation270deg I/flutter (15129): rotationCompensation: 0 I/flutter (15129): rotation: InputImageRotation.rotation270deg I/flutter (15129): format: InputImageFormat.yuv_420_888 I/flutter (15129): formatNew: InputImageFormat.nv21 I/flutter (15129): rotationNew: InputImageRotation.rotation270deg I/flutter (15129): rotationCompensation: 0 I/flutter (15129): rotation: InputImageRotation.rotation270deg I/flutter (15129): format: InputImageFormat.yuv_420_888 I/flutter (15129): formatNew: InputImageFormat.nv21 I/flutter (15129): rotationNew: InputImageRotation.rotation270deg I/flutter (15129): rotationCompensation: 0 I/flutter (15129): rotation: InputImageRotation.rotation270deg I/flutter (15129): format: InputImageFormat.yuv_420_888 I/tflite (15129): Replacing 65 out of 65 node(s) with delegate (TfLiteXNNPackDelegate) node, yielding 1 partitions for the whole graph. I/flutter (15129): formatNew: InputImageFormat.nv21 V/FaceDetectorV2Jni(15129): detectFacesImageByteArray.end() V/FaceDetectorV2Jni(15129): detectFacesImageByteArray.start() V/FaceDetectorV2Jni(15129): detectFacesImageByteArray.end() I/flutter (15129): Faces detected: 0 I/flutter (15129): faces is empty I/flutter (15129): rotationNew: InputImageRotation.rotation270deg I/flutter (15129): rotationCompensation: 0 I/flutter (15129): rotation: InputImageRotation.rotation270deg I/flutter (15129): format: InputImageFormat.yuv_420_888 I/flutter (15129): formatNew: InputImageFormat.nv21 I/flutter (15129): Meta data: Instance of 'InputImageMetadata'

Steps to reproduce.

Use the same code above with android device.

What is the expected result?

faces should be detected

Did you try our example app?

Yes

Is it reproducible in the example app?

No

Reproducible in which OS?

iOS

Flutter/Dart Version?

(base) D:\rnd\projects\Devops_Practice\Attendify git:[plugin-upgrade] flutter doctor -v [√] Flutter (Channel stable, 3.24.0, on Microsoft Windows [Version 10.0.22631.4169], locale en-IN) • Flutter version 3.24.0 on channel stable at C:\Users\Chandrasekar\flutter • Upstream repository https://github.com/flutter/flutter.git • Framework revision 80c2e84975 (8 weeks ago), 2024-07-30 23:06:49 +0700 • Engine revision b8800d88be • Dart version 3.5.0 • DevTools version 2.37.2

[√] Windows Version (Installed version of Windows is version 10 or higher)

[√] Android toolchain - develop for Android devices (Android SDK version 34.0.0) • Android SDK at C:\Users\Chandrasekar\AppData\Local\Android\Sdk • Platform android-34, build-tools 34.0.0 • ANDROID_HOME = C:\Users\Chandrasekar\AppData\Local\Android\Sdk • Java binary at: C:\Program Files\Android\Android Studio3\jbr\bin\java • Java version OpenJDK Runtime Environment (build 17.0.11+0--11852314) • All Android licenses accepted.

[√] Chrome - develop for the web • Chrome at C:\Program Files\Google\Chrome\Application\chrome.exe

[√] Visual Studio - develop Windows apps (Visual Studio Community 2022 17.8.3) • Visual Studio at C:\Program Files\Microsoft Visual Studio\2022\Community • Visual Studio Community 2022 version 17.8.34330.188 • Windows 10 SDK version 10.0.22621.0

[√] Android Studio (version 2024.1) • Android Studio at C:\Program Files\Android\Android Studio3 • Flutter plugin can be installed from: https://plugins.jetbrains.com/plugin/9212-flutter • Dart plugin can be installed from: https://plugins.jetbrains.com/plugin/6351-dart • Java version OpenJDK Runtime Environment (build 17.0.11+0--11852314)

[√] IntelliJ IDEA Community Edition (version 2023.2) • IntelliJ at C:\Program Files\JetBrains\IntelliJ IDEA Community Edition 2023.2.2 • Flutter plugin can be installed from: https://plugins.jetbrains.com/plugin/9212-flutter • Dart plugin can be installed from: https://plugins.jetbrains.com/plugin/6351-dart

[√] VS Code (version 1.93.1) • VS Code at C:\Users\Chandrasekar\AppData\Local\Programs\Microsoft VS Code • Flutter extension version 3.96.0

[√] Connected device (4 available) • CPH2381 (mobile) • e8c563c4 • android-arm64 • Android 14 (API 34) • Windows (desktop) • windows • windows-x64 • Microsoft Windows [Version 10.0.22631.4169] • Chrome (web) • chrome • web-javascript • Google Chrome 128.0.6613.138 • Edge (web) • edge • web-javascript • Microsoft Edge 129.0.2792.52

[√] Network resources • All expected network resources are available.

• No issues found!

Plugin Version?

google_mlkit_face_detection: ^0.11.1

bensonarafat commented 1 month ago

Duplicate here