Contrast and/or Brightness problem within Firebase_ml_vision OCR recognition

The text recognition from an imported image from the official ´firebase_ml_vision´ -> ´example´ app from the ´stable´ branch is not producing the desired results and:

Recognizes something as Text which does not exist
Does NOT recognize the Text that exist on an image from the gallery

It seems like the problem has to do with contrast and/or brightness of the image. After editing the camera picture from the paper, ml_vision was able to recognize the same like on the picture of the screen.

However, after producing a clear, high-contrast and light picture with the ´image´ plugin from pub.dev, the text could still NOT be recognized.

Is there any solution to increase the recognition chance of paper-based text? Not sure if this is a bug or more a settings or even input issue. Any documentation about restrictions on that?

Steps to Reproduce

Please use the following images and try the stable channel example plugin. Also try to use my modifications within the details section, they aim to improve brightness, contrast, etc. of the image as kind of a solution. But it didn't work, even if the image looks perfect.

Expected results: The same text on the paper is recognized like on the screen.

Actual results: Some or no text was recognized.

Pictures that were imported were taken from: https://letter.greatestview.de/

Result of picture from screen: https://imgur.com/AFx9nLx

Result of picture from printed screen: https://imgur.com/bbOFX7v

Result of edited (with iOS picture editor) picture from printed screen: https://imgur.com/SrMrVyC

Result of edited (with Flutter image plugin) picture from printed screen: https://imgur.com/a/Zpxwvtm

Complete source code from stable channel, with some modifications to manipulate the contrast, brightness, etc. of the imported image. ``` // Copyright 2019 The Chromium Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be // found in the LICENSE file. import 'dart:async'; import 'dart:io'; import 'dart:ui'; import 'package:firebase_ml_vision/firebase_ml_vision.dart'; import 'package:flutter/material.dart'; import 'package:image/image.dart' as img; import 'package:image_picker/image_picker.dart'; import 'package:path_provider/path_provider.dart'; import 'detector_painters.dart'; class PictureScanner extends StatefulWidget { @override State createState() => _PictureScannerState(); } class _PictureScannerState extends State { File _imageFile; Size _imageSize; dynamic _scanResults; Detector _currentDetector = Detector.text; final BarcodeDetector _barcodeDetector = FirebaseVision.instance.barcodeDetector(); final FaceDetector _faceDetector = FirebaseVision.instance.faceDetector(); final ImageLabeler _imageLabeler = FirebaseVision.instance.imageLabeler(); final ImageLabeler _cloudImageLabeler = FirebaseVision.instance.cloudImageLabeler(); final TextRecognizer _recognizer = FirebaseVision.instance.textRecognizer(); final TextRecognizer _cloudRecognizer = FirebaseVision.instance.cloudTextRecognizer(); final DocumentTextRecognizer _cloudDocumentRecognizer = FirebaseVision.instance.cloudDocumentTextRecognizer(); final ImagePicker _picker = ImagePicker(); Image _imageToShow; Future _getAndScanImage() async { setState(() { _imageFile = null; _imageSize = null; }); final PickedFile pickedImage = await _picker.getImage(source: ImageSource.gallery); final File imageFile = File(pickedImage.path); if (imageFile != null) { _getImageSize(imageFile); _scanImage(imageFile); } setState(() { _imageFile = imageFile; }); } Future _getImageSize(File imageFile) async { final Completer completer = Completer(); final Image image = Image.file(imageFile); image.image.resolve(const ImageConfiguration()).addListener( ImageStreamListener((ImageInfo info, bool _) { completer.complete(Size( info.image.width.toDouble(), info.image.height.toDouble(), )); }), ); final Size imageSize = await completer.future; setState(() { _imageSize = imageSize; }); } Future _scanImage(File imageFile) async { setState(() { _scanResults = null; }); final now = DateTime.now(); // Read a jpeg image from file. img.Image image = img.decodeJpg(imageFile.readAsBytesSync()); print('Image read: ${measureTime(now)} seconds'); // Resize the image to a 120x? thumbnail (maintaining the aspect ratio). img.Image thumbnail = img.copyResize(image, width: 1024, height: 768); thumbnail = img.adjustColor(image, saturation: 0.0, contrast: 2.0, brightness: 2.0, gamma: 1.3, exposure: 1.0, blacks: 99); print('Image converted: ${measureTime(now)}seconds'); // Save the thumbnail as a PNG. Directory tempDir = await getTemporaryDirectory(); print('Tempdir: ${tempDir.path}'); final file = File('${tempDir.path}/import.png'); file.writeAsBytesSync(img.encodePng(thumbnail), flush: true); print('File written: ${measureTime(now)} seconds'); print('File exists: ${await file.exists()}'); _imageToShow = Image.file(file); //final FirebaseVisionImage visionImage = FirebaseVisionImage.fromFile(file); final FirebaseVisionImage visionImage = FirebaseVisionImage.fromFile(imageFile); dynamic results; switch (_currentDetector) { case Detector.text: results = await _recognizer.processImage(visionImage); print('Text: ${results.text}'); break; default: print('UNEXPECTED'); return; } print('WANT TO DELETE: ${measureTime(now)} seconds'); //file.delete(); print('File deleted'); print('Duration: ${measureTime(now)} seconds'); setState(() { _scanResults = results; }); } CustomPaint _buildResults(Size imageSize, dynamic results) { CustomPainter painter; switch (_currentDetector) { case Detector.barcode: painter = BarcodeDetectorPainter(_imageSize, results); break; case Detector.face: painter = FaceDetectorPainter(_imageSize, results); break; case Detector.label: painter = LabelDetectorPainter(_imageSize, results); break; case Detector.cloudLabel: painter = LabelDetectorPainter(_imageSize, results); break; case Detector.text: painter = TextDetectorPainter(_imageSize, results); break; case Detector.cloudText: painter = TextDetectorPainter(_imageSize, results); break; case Detector.cloudDocumentText: painter = DocumentTextDetectorPainter(_imageSize, results); break; default: break; } return CustomPaint( painter: painter, ); } int measureTime(DateTime start) { return ((DateTime.now().millisecondsSinceEpoch - start.millisecondsSinceEpoch) / 1000) .round(); } Widget _buildImage() { if (_imageToShow == null) return Center(child: CircularProgressIndicator()); return Container( constraints: const BoxConstraints.expand(), decoration: BoxDecoration( image: DecorationImage( image: _imageToShow.image, fit: BoxFit.fill, ), ), child: _imageSize == null || _scanResults == null ? const Center( child: Text( 'Scanning...', style: TextStyle( color: Colors.green, fontSize: 30.0, ), ), ) : _buildResults(_imageSize, _scanResults), ); } @override Widget build(BuildContext context) { return Scaffold( appBar: AppBar( title: const Text('Picture Scanner'), actions: [ PopupMenuButton( onSelected: (Detector result) { _currentDetector = result; if (_imageFile != null) _scanImage(_imageFile); }, itemBuilder: (BuildContext context) => >[ const PopupMenuItem( child: Text('Detect Barcode'), value: Detector.barcode, ), const PopupMenuItem( child: Text('Detect Face'), value: Detector.face, ), const PopupMenuItem( child: Text('Detect Label'), value: Detector.label, ), const PopupMenuItem( child: Text('Detect Cloud Label'), value: Detector.cloudLabel, ), const PopupMenuItem( child: Text('Detect Text'), value: Detector.text, ), const PopupMenuItem( child: Text('Detect Cloud Text'), value: Detector.cloudText, ), const PopupMenuItem( child: Text('Detect Document Text'), value: Detector.cloudDocumentText, ), ], ), ], ), body: _imageFile == null ? const Center(child: Text('No image selected.')) : _buildImage(), floatingActionButton: FloatingActionButton( onPressed: _getAndScanImage, tooltip: 'Pick Image', child: const Icon(Icons.add_a_photo), ), ); } @override void dispose() { _barcodeDetector.close(); _faceDetector.close(); _imageLabeler.close(); _cloudImageLabeler.close(); _recognizer.close(); _cloudRecognizer.close(); super.dispose(); } } ``` ``` ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Warning ────────────────────────────────────────────────────────────────────────────── Your Flutter application is created using an older version of the Android embedding. It's being deprecated in favor of Android embedding v2. Follow the steps at https://flutter.dev/go/android-project-migration to migrate your project. ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Analyzing example... info • The value of the field '_cloudDocumentRecognizer' isn't used • lib/picture_scanner.dart:36:32 • unused_field 1 issue found. (ran in 10.6s) ``` ``` [✓] Flutter (Channel stable, 1.22.6, on macOS 11.2.1 20D74 darwin-x64, locale de-DE) • Flutter version 1.22.6 at /Users/maximilian/workspaces/flutter • Framework revision 9b2d32b605 (6 weeks ago), 2021-01-22 14:36:39 -0800 • Engine revision 2f0af37152 • Dart version 2.10.5 [✓] Android toolchain - develop for Android devices (Android SDK version 29.0.2) • Android SDK at /Users/maximilian/Library/Android/sdk • Platform android-30, build-tools 29.0.2 • Java binary at: /Applications/Android Studio.app/Contents/jre/jdk/Contents/Home/bin/java • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b3-6915495) • All Android licenses accepted. [✓] Xcode - develop for iOS and macOS (Xcode 12.4) • Xcode at /Applications/Xcode.app/Contents/Developer • Xcode 12.4, Build version 12D4e • CocoaPods version 1.9.3 [!] Android Studio (version 4.1) • Android Studio at /Applications/Android Studio.app/Contents ✗ Flutter plugin not installed; this adds Flutter specific functionality. ✗ Dart plugin not installed; this adds Dart specific functionality. • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b3-6915495) [✓] VS Code (version 1.52.1) • VS Code at /Applications/Visual Studio Code.app/Contents • Flutter extension version 3.19.0 [✓] Connected device (1 available) • iPhone von Maximilian (mobile) • .... • ios • iOS 14.4 ! Error: iPad von Maximilian is not connected. Xcode will continue when iPad von Maximilian is connected. (code -13) ! Doctor found issues in 1 category. ``` ``` Dart SDK 2.10.5 Flutter SDK 1.22.6 firebase_ml_vision_example 0.0.0 dependencies: - camera 0.5.8+17 [flutter] - cupertino_icons 0.1.3 - firebase_core 0.4.5 [firebase_core_platform_interface flutter meta firebase_core_web] - firebase_ml_vision 0.9.6+2 [flutter] - flutter 0.0.0 [characters collection meta typed_data vector_math sky_engine] - image 2.1.19 [archive xml meta] - image_picker 0.6.7+22 [flutter flutter_plugin_android_lifecycle image_picker_platform_interface] dev dependencies: - e2e 0.6.3 [flutter flutter_driver flutter_test path] - flutter_driver 0.0.0 [file json_rpc_2 meta path web_socket_channel vm_service_client webdriver flutter flutter_test fuchsia_remote_debug_protocol archive args async boolean_selector characters charcode clock collection convert crypto fake_async matcher platform process pub_semver source_span stack_trace stream_channel string_scanner sync_http term_glyph test_api typed_data vector_math] - flutter_test 0.0.0 [flutter test_api path fake_async clock stack_trace vector_math async boolean_selector characters charcode collection matcher meta source_span stream_channel string_scanner term_glyph typed_data] - path 1.8.0-nullsafety.1 - path_provider 0.5.0+1 [flutter] - pedantic 1.10.0-nullsafety.2 - test 1.16.0-nullsafety.5 [analyzer async boolean_selector coverage http http_multi_server io js node_preamble package_config path pedantic pool shelf shelf_packages_handler shelf_static shelf_web_socket source_span stack_trace stream_channel typed_data web_socket_channel webkit_inspection_protocol yaml test_api test_core] transitive dependencies: - _fe_analyzer_shared 12.0.0 [meta] - analyzer 0.40.6 [_fe_analyzer_shared args cli_util collection convert crypto glob meta package_config path pub_semver source_span watcher yaml] - archive 2.0.13 [crypto args path] - args 1.6.0 - async 2.5.0-nullsafety.1 [collection] - boolean_selector 2.1.0-nullsafety.1 [source_span string_scanner] - characters 1.1.0-nullsafety.3 - charcode 1.2.0-nullsafety.1 - cli_util 0.2.0 [path] - clock 1.1.0-nullsafety.1 - collection 1.15.0-nullsafety.3 - convert 2.1.1 [charcode typed_data] - coverage 0.14.2 [args logging package_config path source_maps stack_trace vm_service] - crypto 2.1.5 [collection convert typed_data] - fake_async 1.2.0-nullsafety.1 [clock collection] - file 6.0.0-nullsafety.2 [meta path] - firebase 7.3.3 [http http_parser js] - firebase_core_platform_interface 1.0.4 [flutter meta plugin_platform_interface quiver] - firebase_core_web 0.1.1+2 [firebase firebase_core_platform_interface flutter flutter_web_plugins meta js] - flutter_plugin_android_lifecycle 1.0.11 [flutter] - flutter_web_plugins 0.0.0 [flutter characters collection meta typed_data vector_math] - fuchsia_remote_debug_protocol 0.0.0 [json_rpc_2 process web_socket_channel flutter_test flutter_driver archive args async boolean_selector charcode clock collection convert crypto fake_async file matcher meta path platform pub_semver source_span stack_trace stream_channel string_scanner sync_http term_glyph test_api typed_data vector_math vm_service_client webdriver] - glob 1.2.0 [async collection node_io path pedantic string_scanner] - http 0.12.2 [http_parser path pedantic] - http_multi_server 2.2.0 [async] - http_parser 3.1.4 [charcode collection source_span string_scanner typed_data] - image_picker_platform_interface 1.1.6 [flutter meta http plugin_platform_interface] - io 0.3.5 [meta path string_scanner] - js 0.6.3-nullsafety.2 - json_rpc_2 2.2.2 [stack_trace stream_channel] - logging 0.11.4 - matcher 0.12.10-nullsafety.1 [stack_trace] - meta 1.3.0-nullsafety.3 - mime 0.9.7 - node_interop 1.2.1 [js] - node_io 1.1.1 [node_interop path] - node_preamble 1.4.13 - package_config 1.9.3 [path charcode] - petitparser 3.1.0 [meta] - platform 3.0.0-nullsafety.2 - plugin_platform_interface 1.0.3 [meta] - pool 1.5.0-nullsafety.2 [async stack_trace] - process 4.0.0-nullsafety.2 [file path platform] - pub_semver 1.4.4 [collection] - quiver 2.1.5 [matcher meta] - shelf 0.7.9 [async collection http_parser path stack_trace stream_channel] - shelf_packages_handler 2.0.1 [path shelf shelf_static] - shelf_static 0.2.9+2 [convert http_parser mime path shelf] - shelf_web_socket 0.2.4+1 [shelf stream_channel web_socket_channel] - sky_engine 0.0.99 - source_map_stack_trace 2.1.0-nullsafety.3 [path stack_trace source_maps] - source_maps 0.10.10-nullsafety.2 [source_span] - source_span 1.8.0-nullsafety.2 [charcode collection path term_glyph] - stack_trace 1.10.0-nullsafety.1 [path] - stream_channel 2.1.0-nullsafety.1 [async] - string_scanner 1.1.0-nullsafety.1 [charcode source_span] - sync_http 0.2.0 - term_glyph 1.2.0-nullsafety.1 - test_api 0.2.19-nullsafety.2 [async boolean_selector collection meta path source_span stack_trace stream_channel string_scanner term_glyph matcher] - test_core 0.3.12-nullsafety.5 [analyzer async args boolean_selector collection coverage glob io meta package_config path pedantic pool source_map_stack_trace source_maps source_span stack_trace stream_channel vm_service yaml matcher test_api] - typed_data 1.3.0-nullsafety.3 [collection] - vector_math 2.1.0-nullsafety.3 - vm_service 5.5.0 [meta] - vm_service_client 0.2.6+2 [async collection json_rpc_2 pub_semver source_span stack_trace stream_channel web_socket_channel] - watcher 0.9.7+15 [async path pedantic] - web_socket_channel 1.1.0 [async crypto stream_channel] - webdriver 2.1.2 [archive matcher path stack_trace sync_http] - webkit_inspection_protocol 0.7.5 [logging] - xml 4.5.1 [collection convert meta petitparser] - yaml 2.2.1 [charcode collection string_scanner source_span] ```

firebase / flutterfire

Contrast and/or Brightness problem within Firebase_ml_vision OCR recognition #5216

Steps to Reproduce