Error while reading content from uri

[gustavo867]

What happened?

Error: No content provider:

Version

@react-native-ml-kit/barcode-scanning: version@react-native-ml-kit/face-detection: version@react-native-ml-kit/identify-languages: version@react-native-ml-kit/image-labeling: version@react-native-ml-kit/text-recognition: version@react-native-ml-kit/translate-text: version

Which ML Kit packages do you use?

• [ ] @react-native-ml-kit/barcode-scanning
• [ ] @react-native-ml-kit/face-detection
• [ ] @react-native-ml-kit/identify-languages
• [ ] @react-native-ml-kit/image-labeling
• [X] @react-native-ml-kit/text-recognition
• [ ] @react-native-ml-kit/translate-text

What platforms are you seeing this issue on?

• [X] Android
• [ ] iOS

System Information

System: OS: Windows 10 10.0.19045 CPU: (12) x64 Intel(R) Core(TM) i5-10400F CPU @ 2.90GHz Memory: 587.97 MB / 15.88 GB Binaries: Node: 16.10.0 - C:\Program Files\nodejs\node.EXE Yarn: 1.22.19 - ~\AppData\Roaming\npm\yarn.CMD npm: 7.24.0 - C:\Program Files\nodejs\npm.CMD Watchman: Not Found SDKs: Android SDK: API Levels: 31, 33 Build Tools: 30.0.3, 31.0.0, 33.0.0 System Images: android-31 | Google Play Intel x86 Atom_64, android-33 | Google APIs Intel x86 Atom_64 Android NDK: Not Found Windows SDK: Not Found IDEs: Android Studio: AI-213.7172.25.2113.9123335 Visual Studio: Not Found Languages: Java: 11.0.17 - /c/Program Files/Eclipse Adoptium/jdk-11.0.17.8-hotspot/bin/javac npmPackages: @react-native-community/cli: Not Found react: 18.1.0 => 18.1.0 react-native: 0.70.5 => 0.70.5 react-native-windows: Not Found npmGlobalPackages: react-native: Not Found

Steps to Reproduce

install react native mlkit and try to recognize text in android emulator

async function recognizeText() { try { const result = await TextRecognition.recognize( "https://developers.google.com/static/ml-kit/vision/text-recognition/images/Wege_der_parlamentarischen_Demokratie.jpg" );

  console.log("A", JSON.stringify(result, null, 2));

  for (let block of result.blocks) {
    console.log("Block text:", block.text);
    console.log("Block frame:", block.frame);

    for (let line of block.lines) {
      console.log("Line text:", line.text);
      console.log("Line frame:", line.frame);
    }
  }

  // resultFromFile.map(item => {
  //   item.lines
  // });

  setText(result.text);
} catch (err) {
  console.log("Error", err);
}

}

[andrewjb123]

reviewing the code for android:

Uri uri = Uri.parse(url); InputImage image;

try { image = InputImage.fromFilePath(reactContext, uri);

here is the documentation for fromFilePath:

https://developers.google.com/android/reference/com/google/mlkit/vision/common/InputImage

its expecting a local file path rather than a url.

if you use

const image = await ImagePicker.openPicker({ mediaType: 'photo' });

const result = await TextRecognition.recognize(image.path);

does it work?

if yes, you will need to download and persist the input image to disk, and then provide the on disk path to the file.

alternatively android implementation will need to be upgraded to detect if the url is from the web (http / https), download the file to bytes and then use fromByteBuffer rather than fromFilePath

[andrewjb123]

I've added a commit for upgrading all modules to allow for url downloading in android:

https://github.com/a7medev/react-native-ml-kit/pull/16/commits/5d34937334f7cdb0f8034ef53a774b3ce335f3f1

here is the commit, hopefully it will be merged in to main project and this should fix your problem.

[a7medev]

@andrewjb123 Thanks for the contributions and support 🙏🏼 I'll review the PR you've added once I have time. Please mark it as a draft if you are still adding stuff in it and mark it as ready for review once ready. Also, can we separate each change in a separate PR so we don't mix things up in one giant PR? Like, custom image labeling in a PR, text translation support in another PR and URL download support in another PR? That would make the review process faster and will allow the features that are ready to be merged to get merged quickly without relying on irrelevant changes. Again, thanks for your time and contributions, I really appreciate it 🙏🏼

[andrewjb123]

Hi 🙏, no problem, I’ll seperate them out into seperate PR’s if that makes things easier, bare with me whilst I do that.

thanks for releasing what you have done so far, it’s been immensely helpful for me, it’s been a good way to learn react native adapters, objective c and Java with a purpose!

[andrewjb123]

@a7medev ok that should be done now as 3 separate PR's

[gustavo867]

reviewing the code for android:

Uri uri = Uri.parse(url); InputImage image;

try { image = InputImage.fromFilePath(reactContext, uri);

here is the documentation for fromFilePath:

https://developers.google.com/android/reference/com/google/mlkit/vision/common/InputImage

its expecting a local file path rather than a url.

if you use

const image = await ImagePicker.openPicker({ mediaType: 'photo' });

const result = await TextRecognition.recognize(image.path);

does it work?

if yes, you will need to download and persist the input image to disk, and then provide the on disk path to the file.

alternatively android implementation will need to be upgraded to detect if the url is from the web (http / https), download the file to bytes and then use fromByteBuffer rather than fromFilePath

I tested with localURI and all went fine, thanks

[andrewjb123]

No problem, for reference there is this open pull request to allow for Android url downloading in the future

https://github.com/a7medev/react-native-ml-kit/pull/18

[gustavo867]

No problem, for reference there is this open pull request to allow for Android url downloading in the future

18

It is possible to add only one language support? I want to recognize only english texts, but don't have an option for that.

[gustavo867]

No problem, for reference there is this open pull request to allow for Android url downloading in the future

18

It is possible to add only one language support? I want to recognize only english texts, but don't have an option for that.

Is a great feature, i guess, but i don't know if google support this

[andrewjb123]

No problem, for reference there is this open pull request to allow for Android url downloading in the future

18

It is possible to add only one language support? I want to recognize only english texts, but don't have an option for that.

I’ll have a look at the documentation to see if it’s possible - even if it doesn’t support it through their text recognition api, you could theoretically combine the output from text recognition, and put the output text through identify language module, and discard any results where the language identification isn’t English.

https://github.com/a7medev/react-native-ml-kit/tree/main/identify-languages

here is the identify language module

The alternative but a little more complicated is to expand text recognition module to use a custom model which you train with only English corpus text.

[gustavo867]

No problem, for reference there is this open pull request to allow for Android url downloading in the future

18

It is possible to add only one language support? I want to recognize only english texts, but don't have an option for that.

I’ll have a look at the documentation to see if it’s possible - even if it doesn’t support it through their text recognition api, you could theoretically combine the output from text recognition, and put the output text through identify language module, and discard any results where the language identification isn’t English.

https://github.com/a7medev/react-native-ml-kit/tree/main/identify-languages

here is the identify language module

The alternative but a little more complicated is to expand text recognition module to use a custom model which you train with only English corpus text.

Thanks, is gonna work for my case

[andrewjb123]

No problem 👍

[gustavo867]

I having an error in iOS too, don't recognize texts coming from camera image (only in portrait, when is landscape works fine).
You know if really only recognize landscape images in IOS? or is a bug.

[andrewjb123]

What is the error you receive?

Successful detection in ai models is largely determined by the training data, if the training data is provided in portrait then the input should also be in portrait otherwise detection won’t be successful.

that being said you can look at the meta data of an image and determine what position the photograph was taken in and then provide a rotation on the input image.

I have seen some of the methods on mlkit allow to provide a rotation in degrees for the image, so maybe that meta data lookup and rotation could be provided via the module - this would require a module change and an npm release.

to do this without a module update:

rotate the image to what is the correct input which has successful detection, from the camera you can detect which mode it’s in and perform the rotation of the image you provide.

To translate a portrait image to landscape you will probably need to rotate 90 degrees clockwise

[andrewjb123]

Love the problem solvin keep em coming 😉

[andrewjb123]

This bug can probably be closed

[gustavo867]

This bug can probably be closed

Oh sorry, I completely forgot about this issues