barcodeCapture.size.isEmpty is always true on Android

[fulstadev]

I've setup the following widget to use it as QR-Code Scanner Screen:

dummy.dart:

import 'dart:async';
import 'package:flutter/material.dart';
import 'package:mobile_scanner/mobile_scanner.dart';
import 'package:flutter_gen/gen_l10n/app_localizations.dart';

part 'dummy_window.dart';

/// Creates a QR-Code Scanner with an inner frame within which scanned QR Codes
/// will be captured
class QRCodeScannerScreen extends StatefulWidget {

  const QRCodeScannerScreen({
    super.key,
  });

  @override
  State<QRCodeScannerScreen> createState() => _QRCodeScannerScreenState();
}

/// Mixin the [WidgetsBindingObserver] to bind lifecycle changes to the
/// functionality of the code scanner
/// (see https://pub.dev/packages/mobile_scanner#usage)
class _QRCodeScannerScreenState extends State<QRCodeScannerScreen>
    with WidgetsBindingObserver {
  /// Controller needed to produce QR-Code scanning ability via the
  /// 'mobile_scanner' flutter package
  /// (see https://pub.dev/packages/mobile_scanner#usage)
  final MobileScannerController _controller = MobileScannerController(
    // Avoid that QR-code is scanned an excessive amount of times
      detectionSpeed: DetectionSpeed.normal,
      // Wait for 750 ms before scanning again
      // Leave this, as this is what prevented android scans from working on
      // older devices
      detectionTimeoutMs: 250,
      // Use backwards camera
      facing: CameraFacing.back,
      // Disable flashlight on start
      torchEnabled: false,
      formats: const [BarcodeFormat.qrCode]
  );

  /// Providing a stream subscription is necessary to be able to subscribe a
  /// listener to the scanner of barcodes
  /// (see https://pub.dev/packages/mobile_scanner#usage)
  StreamSubscription<Object?>? _subscription;

  /// Callback which reacts to a properly captured barcode by the scanner within
  /// the borders of the scan window, by in that case exiting the screen with
  /// the value of the first returned barcode, as string
  ///
  /// see the [ValueListenableBuilder] and the [StreamBuilder] within the
  /// file https://github.com/juliansteenbakker/mobile_scanner/blob/master/example/lib/barcode_scanner_window.dart
  void _handleCapturedBarcode(BarcodeCapture barcodeCapture) {

    print("captured");

    print("mounted: $mounted");
    print("_controller.value.isInitialized: ${_controller.value.isInitialized}");
    print("_controller.value.isRunning: ${_controller.value.isRunning}");
    print("_controller.value.error: ${_controller.value.error}");
    print("barcodeCapture.barcodes.isNotEmpty: ${barcodeCapture.barcodes.isNotEmpty}");
    print("barcodeCapture.barcodes.firstOrNull: ${barcodeCapture.barcodes.firstOrNull}");
    print("barcodeCapture.barcodes.first.corners.isNotEmpty: ${barcodeCapture.barcodes.first.corners.isNotEmpty}");
    print("!_controller.value.size.isEmpty: ${!_controller.value.size.isEmpty}");
    print("!barcodeCapture.size.isEmpty: ${!barcodeCapture.size.isEmpty}");
    print("barcodeCapture.barcodes.first.rawValue: ${barcodeCapture.barcodes.first.rawValue}");

    if (mounted &&

        /// If the [MobileScannerState] is initialized
        _controller.value.isInitialized &&

        /// If the [MobileScannerState] is running
        _controller.value.isRunning &&

        /// If the [MobileScannerState] currently has no errors
        _controller.value.error == null &&

        /// If at least one result is captured
        barcodeCapture.barcodes.isNotEmpty &&

        /// If an actual barcode was returned
        barcodeCapture.barcodes.firstOrNull != null &&

        /// If the corners of the barcode can all be found (= they are all
        /// situated within the scanner window of the [MobileScanner] being used
        barcodeCapture.barcodes.first.corners.isNotEmpty &&

        /// If the size of the camera output is not empty
        !_controller.value.size.isEmpty &&

        /// If the size of the scanned barcode can be detected
        !barcodeCapture.size.isEmpty) {
      /// As soon as a code has been properly detected, cancel the stream
      /// subscription, such that this will not be called again (otherwise,
      /// experienced that the app called Navigator.pop() multiple times in
      /// sequence).
      _subscription?.cancel();
      _subscription = null;
      Navigator.pop(
        context,
        barcodeCapture.barcodes.first.rawValue,
      );
    }
  }

  /// Adapt the behaviour of the [_controller] and the [_subscription] according
  /// to the current [AppLifecycleState]
  /// (see https://pub.dev/packages/mobile_scanner#usage)
  @override
  void didChangeAppLifecycleState(AppLifecycleState state) {
    /// If the controller is not ready, do not try to start or stop it.
    /// Permission dialogs can trigger lifecycle changes before the controller is ready.
    if (!_controller.value.isInitialized) {
      return;
    }

    switch (state) {
      case AppLifecycleState.detached:
      case AppLifecycleState.hidden:
      case AppLifecycleState.paused:
        return;
      case AppLifecycleState.resumed:

      /// Restart the scanner when the app is resumed.
      /// Don't forget to resume listening to the barcode events.
        _subscription = _controller.barcodes.listen(_handleCapturedBarcode);

        unawaited(_controller.start());
      case AppLifecycleState.inactive:

      /// Stop the scanner when the app is paused.
      /// Also stop the barcode events subscription.
        unawaited(_subscription?.cancel());
        _subscription = null;
        unawaited(_controller.stop());
    }
  }

  @override
  void initState() {
    _controller.stop();

    super.initState();

    /// Start everything related to the scanner upon startup
    /// (see https://pub.dev/packages/mobile_scanner#usage)
    /// Start listening to lifecycle changes.
    WidgetsBinding.instance.addObserver(this);

    /// Start listening to the barcode events.
    _subscription = _controller.barcodes.listen(_handleCapturedBarcode);

    /// Finally, start the scanner itself.
    unawaited(_controller.start());

  }

  /// Dispose of anything related to the scanner state upon dispose
  /// (see https://pub.dev/packages/mobile_scanner#usage)
  @override
  Future<void> dispose() async {
    /// Stop listening to lifecycle changes.
    WidgetsBinding.instance.removeObserver(this);

    /// Stop listening to the barcode events.
    unawaited(_subscription?.cancel());
    _subscription = null;

    /// Dispose the widget itself.
    super.dispose();

    /// Finally, dispose of the controller.
    await _controller.dispose();
  }

  @override
  Widget build(BuildContext context) {
    /// Build a squared Y x Y QR-Code Scanner Window
    final scanWindow = Rect.fromCenter(
      center: MediaQuery.sizeOf(context).center(Offset.zero),
      width: 250,
      height: 250,
    );

    final double spacingOfInstructionLabelFromTheTop =
        MediaQuery.of(context).size.height * 0.1;

    final Widget screenContent = Stack(
      fit: StackFit.expand,
      children: [
        /// QR-Code Scanner
        MobileScanner(
          /// To make sure that the entire screen is covered
          fit: BoxFit.contain,

          /// Controller of the Mobile Scanner, to bind the controller to the
          /// Stream returning the scanned barcodes upon scan
          controller: _controller,

          /// Such that the barcode scanner will only scan codes which intersect
          /// with this rectangle
          scanWindow: scanWindow,

          /// Callback used to generate the content to be shown upon an error
          errorBuilder: (context, exception, _) {
            /// To resolve issue with error saying 'called start() while
            /// already started',
            /// see https://github.com/juliansteenbakker/mobile_scanner/issues/539
            _controller.stop();
            _controller.start();

            return const Text('Error');
          },
        ),

        /// Overlay with the provided [scanWindow] cut out
        _QRCodeScannerWindow(
          scanWindow: scanWindow,
          controller: _controller,
        ),

        /// Label present above the square of the QR-Code Scanner, providing the
        /// instruction to place the code to be scanner within the frame
        Align(
          alignment: Alignment.topCenter,
          child: Padding(
            padding: EdgeInsets.fromLTRB(
              50,
              spacingOfInstructionLabelFromTheTop,
              50,
              spacingOfInstructionLabelFromTheTop / 2,
            ),
            child: Row(
              mainAxisAlignment: MainAxisAlignment.center,
              mainAxisSize: MainAxisSize.max,
              children: [
                /// Use expanded + softWrap: true to wrap text if it exceeds
                /// the screen length
                Expanded(
                  child: Text(
                    AppLocalizations.of(context).pleasePlaceQRCodeWithinSquare,
                    textAlign: TextAlign.center,
                    softWrap: true,
                  ),
                ),
              ],
            ),
          ),
        )
      ],
    );

    return Scaffold(
      appBar: AppBar(
        title: Text(
          AppLocalizations.of(context).qrScannerTitle,
        ),
      ),
      body: screenContent,
    );

  }
}

dummy_window.dart:

part of 'dummy.dart';

/// Painter used to generate the [_QRCodeScannerWindow] widget
class _QRCodeScannerPainter extends CustomPainter {
  /// Rectangle within which the scanning will be active
  final Rect scanWindow;

  const _QRCodeScannerPainter({
    required this.scanWindow,
  });

  /// Method used to paint the overlay, to create a not-entirely opaque screen
  /// overlay with a rectangle cut out in the center of it, according to the
  /// passed [scanWindow]
  @override
  void paint(Canvas canvas, Size size) {
    /// Background Paint, which is simply an overlay that covers the entire
    /// screen
    final backgroundPath = Path()
      ..addRect(
        Rect.largest,
      );

    /// Path to be cut out from the overlay
    final cutoutPath = Path()
      ..addRect(
        scanWindow,
      );

    /// Create the final Paint, corresponding to the semi-transparent overlay
    /// with the passed [scanWindow] cut out, which will finally result in an
    /// overlay screen with the inner rectangle cut out, representing the
    /// scanners' window
    final resultingPath =
    Path.combine(PathOperation.difference, backgroundPath, cutoutPath);

    /// Create the black semi-transparent pain to be applied to
    /// [resultingPath]
    final backgroundPaint = Paint()
      ..color = Colors.black.withOpacity(
        0.75,
      )
      ..style = PaintingStyle.fill
      ..blendMode = BlendMode.dstOut;

    /// Draw the [resultingPath] with the [backgroundPaint] applied onto it by
    /// drawing it into the canvas
    canvas.drawPath(
      resultingPath,
      backgroundPaint,
    );

    /// Create additional paint for the border of the cutout rectangle
    final borderPaint = Paint()
      ..color = Colors.white
      ..style = PaintingStyle.stroke
      ..strokeWidth = 3.0;

    canvas.drawRect(
      scanWindow,
      borderPaint,
    );
  }

  /// A repaint is never required
  @override
  bool shouldRepaint(covariant CustomPainter oldDelegate) {
    return false;
  }
}

/// Scanner Window Overlay used for the QR-Code Scanner Screen, to frame the
/// contents being scanned by the scanner
class _QRCodeScannerWindow extends StatelessWidget {
  /// controller of the [MobileScanner] used for the [QRCodeScannerScreen]
  final MobileScannerController controller;

  /// Rectangle within which the scanning will be active
  final Rect scanWindow;

  const _QRCodeScannerWindow({
    required this.controller,
    required this.scanWindow,
  });

  @override
  Widget build(BuildContext context) {
    return ValueListenableBuilder(
      valueListenable: controller,
      builder: (context, value, child) {
        // Not ready.
        if (!value.isInitialized ||
            !value.isRunning ||
            value.error != null ||
            value.size.isEmpty) {
          return const SizedBox();
        }

        return CustomPaint(
          painter: _QRCodeScannerPainter(
            scanWindow: scanWindow,
          ),
        );
      },
    );
  }
}

Only on physical android devices (both a modern and an older one), the screen is never left. That's why I've added the many print statements in the code, and indeed, for android only, barcoeCapture.size.isEmpty is always true, so the screen is never exit on Android phones. A sample output of all the print statements is:

I/flutter (25114): captured
I/flutter (25114): mounted: true
I/flutter (25114): _controller.value.isInitialized: true
I/flutter (25114): _controller.value.isRunning: true
I/flutter (25114): _controller.value.error: null
I/flutter (25114): barcodeCapture.barcodes.isNotEmpty: true
I/flutter (25114): barcodeCapture.barcodes.firstOrNull: Instance of 'Barcode'
I/flutter (25114): barcodeCapture.barcodes.first.corners.isNotEmpty: true
I/flutter (25114): !_controller.value.size.isEmpty: true
I/flutter (25114): !barcodeCapture.size.isEmpty: false <-- This seems to be the problem
I/flutter (25114): barcodeCapture.barcodes.first.rawValue: properly detected URL

on iOS, no problem at all.

Any idea why this could be happening?

Version we're using: mobile_scanner: ^5.1.1

flutter doctor output:

[✓] Flutter (Channel stable, 3.19.6, on macOS 14.5 23F79 darwin-arm64, locale de-CH)
[✓] Android toolchain - develop for Android devices (Android SDK version 34.0.0)
[✓] Xcode - develop for iOS and macOS (Xcode 15.4)
[✓] Chrome - develop for the web
[✓] Android Studio (version 2023.2)
[✓] IntelliJ IDEA Ultimate Edition (version 2024.1.4)
[✓] Connected device (3 available)
[✓] Network resources

• No issues found!

[navaronbracke]

I will have to investigate why this is happening. Do you have a sample barcode that has a size on iOS ?

[fulstadev]

@navaronbracke Thanks that would be great! For us, it happened for pretty much every classic QR-Code that holds a URL / TOTP-URL in black and white, on a white background (I'm not allowed to share these here). But it seems to happen with any basic QR-Code (I've attached an example with which the above-mentioned happened; works on iOS, size is always empty on android). The value of the QR-Code is the URL https://www.google.com (generated via https://www.qrcode-monkey.com). .

[navaronbracke]

Thanks for the example, I'll investigate this bug!

[fulstadev]

@navaronbracke do you have any news, or know when this could be fixed? Or a possible workaround for the meantime that you could recommend?

[navaronbracke]

I didn't have time to dive into this much yet, although I do see that we pass the raw data from MLKit back to Dart. So perhaps there is no data from MLKit or we do not properly parse the data that we get back. I will only be able to fix the bug if it is the latter situation, though.

[fulstadev]

Alright thanks for the info. Curious to see what's the reason, as this is unfortunately currently preventing our release. If you cannot fix it please let me know too, then I'll probably just omit the size check on Android.

[navaronbracke]

@fulstadev Okay, so it seems we don't pass the bounding box (which is a nullable Rect) to https://github.com/juliansteenbakker/mobile_scanner/blob/master/android/src/main/kotlin/dev/steenbakker/mobile_scanner/MobileScannerUtilities.kt#L29-L37 which is an extension on the Android MLKit Barcode class. We use that extension to get a Map<String, Any?> to pass to Dart. I checked the remaining properties of the MLKit Barcode class on Android and it was the only one we missed.

Should be an easy bugfix, so you can expect this to be released today, once I tested it :)

[fulstadev]

@navaronbracke Excellent News, thank you so much mate!

[navaronbracke]

@fulstadev With the new fix, I will be adding a new size property to Barcode (which is the size of the barcode bounding box). You are going to want to use that from now on.

The BarcodeCapture.size attribute is the size of the capture input image from the camera. I also updated the documentation for that property, to make it less confusing.

[navaronbracke]

@fulstadev If you want to, feel free to do a code review on the linked PR and/or test the fix on your end. (using a git dependency) https://dart.dev/tools/pub/dependencies#git-packages

I'll try to test this today, but this should help you out in the interim.

[navaronbracke]

@fulstadev This is included in version 5.2.2

[navaronbracke]

@fulstadev Small follow-up question, do you want the size and corners on MacOS also? At first I wasn't sure if the result from the Vision API (which we use for MacOS) contained that information. However, the corner points (and by extension the size of the barcode) are available there.

[fulstadev]

@navaronbracke Cheers mate! Sorry was not able to get back on this so far, will implement and test it now! And no thanks, no need for a macOS implementation, at least nor for us.

[navaronbracke]

No worries! I'll be adding the MacOS (and web) size information in another update. For those platforms, we can use the normalized size from the corner points.