search

xamarin / GooglePlayServicesComponents

Other
315 stars 148 forks source link

Exception in MLKit TextRecognition.GetClient Bundled. Attempt to invoke virtual method com.google.mlkit.vision.text.internal.zzm.zza #891

Closed mv4wd closed 1 month ago

mv4wd commented 1 month ago

Describe your Issue

This is an old Xamarin application where I would like to add MLKit text decoding (bundled model, as to not download it from PlayService, usually not available in the installation). I've added the Xamarin packages and can succesfully compile, but when I run the app and try to obtain the TextRecognizer, I get the following exception

using Xamarin.Google.MLKit.Vision.Text;
using Xamarin.Google.MLKit.Vision.Text.Latin;

ITextRecognizer  t = TextRecognition.GetClient(Latin.TextRecognizerOptions.DefaultOptions);

Exception throw:

Attempt to invoke virtual method 'com.google.mlkit.vision.text.TextRecognizer com.google.mlkit.vision.text.internal.zzm.zza(com.google.mlkit.vision.text.TextRecognizerOptionsInterface)' on a null object reference

One Issue that I see on the console, is that when I call MlKit.Initialize, I get these messages:

[ComponentDiscovery] Application info not found.
[ComponentDiscovery] Could not retrieve metadata, returning empty list of registrars.

By reading the google docs, it seems like that, to use the bundled Text model, a key element is the gradle configuration

dependencies {
  // To recognize Latin script
  implementation 'com.google.mlkit:text-recognition:16.0.0'
}

But I see no way to mimic this in VS. Is a Mono working sample for bundled com.google.mlkit.vision.text.TextRecognizer available?

Thanks. Marco

Operating System & Version

Android 8.1.0

Google Play Services Version

N/A

Relevant information

   <PackageReference Include="Xamarin.Google.MLKit.Common">
      <Version>116.0.0</Version>
    </PackageReference>
    <PackageReference Include="Xamarin.Google.MLKit.TextRecognition">
      <Version>116.0.0.1</Version>
    </PackageReference>
    <PackageReference Include="Xamarin.Google.MLKit.TextRecognition.Bundled.Common">
      <Version>116.0.0.1</Version>
    </PackageReference>
   <PackageReference Include="Xamarin.Forms" Version="5.0.0.2012" />
     <PackageReference Include="Xamarin.Essentials">
      <Version>1.6.1</Version>
    </PackageReference>

Build settings (tools)

<AndroidDexTool>d8</AndroidDexTool>
<AndroidLinkTool>r8</AndroidLinkTool>
<AndroidUseAapt2>true</AndroidUseAapt2>
<AndroidLinkMode>None</AndroidLinkMode>

Include any relevant Exception Stack traces, build logs, adb logs:

  at Java.Interop.JniEnvironment+StaticMethods.CallStaticObjectMethod (Java.Interop.JniObjectReference type, Java.Interop.JniMethodInfo method, Java.Interop.JniArgumentValue* args) [0x00068] in /Users/runner/work/1/s/xamarin-android/external/Java.Interop/src/Java.Interop/obj/Release/JniEnvironment.g.cs:12890 
  at Java.Interop.JniPeerMembers+JniStaticMethods.InvokeObjectMethod (System.String encodedMember, Java.Interop.JniArgumentValue* parameters) [0x00008] in /Users/runner/work/1/s/xamarin-android/external/Java.Interop/src/Java.Interop/Java.Interop/JniPeerMembers.JniStaticMethods.cs:151 
  at Xamarin.Google.MLKit.Vision.Text.TextRecognition.GetClient (Xamarin.Google.MLKit.Vision.Text.ITextRecognizerOptionsInterface options) [0x0002c] in D:\a\_work\1\s\generated\com.google.android.gms.play-services-mlkit-text-recognition-common\obj\Release\monoandroid12.0\generated\src\Xamarin.Google.MLKit.Vision.Text.TextRecognition.cs:57 
  at CameraAIScanner.AITextEstractor.TryExtractText (System.String title, System.String message, Android.Graphics.Bitmap theImage) [0x0000c] 
  --- End of managed Java.Lang.NullPointerException stack trace ---
java.lang.NullPointerException: Attempt to invoke virtual method 'com.google.mlkit.vision.text.TextRecognizer com.google.mlkit.vision.text.internal.zzm.zza(com.google.mlkit.vision.text.TextRecognizerOptionsInterface)' on a null object reference
    at com.google.mlkit.vision.text.TextRecognition.getClient(com.google.android.gms:play-services-mlkit-text-recognition-common@@19.0.0:1)
    at mono.java.lang.RunnableImplementor.n_run(Native Method)
    at mono.java.lang.RunnableImplementor.run(RunnableImplementor.java:31)
    at android.os.Handler.handleCallback(Handler.java:790)
    at android.os.Handler.dispatchMessage(Handler.java:99)
    at android.os.Looper.loop(Looper.java:164)
    at android.app.ActivityThread.main(ActivityThread.java:6528)
    at java.lang.reflect.Method.invoke(Native Method)
    at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:438)
    at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:807)
moljac commented 1 month ago

Xamarin Classic -> Xamarin.Forms is out of support.

Besides why using olde nugets:

https://www.nuget.org/packages/Xamarin.Google.MLKit.Common/118.10.0.3

https://www.nuget.org/packages/Xamarin.Forms/5.0.0.2662

https://www.nuget.org/packages/Xamarin.Essentials/1.8.1

??

Even if you would provide repro sample I am afraid I/we would not be able to test it and investigate.

mv4wd commented 1 month ago

Unfortunately this is a legacy app that has a lot of dependencies and would require quite an effort to upgrade. Anyway, I've been able to pass the GetClient exception by manually adding in the Android manifest the following lines that I've extracted from a 'modern' test app.

<provider android:name="com.google.mlkit.common.internal.MlKitInitProvider" android:authorities="com.TesySoftware.TLogClient.mlkitinitprovider" android:exported="false" android:initOrder="99" />
<service android:name="com.google.mlkit.common.internal.MlKitComponentDiscoveryService" android:directBootAware="true" android:exported="false" tools:targetApi="n">
      <meta-data android:name="com.google.firebase.components:com.google.mlkit.common.internal.CommonComponentRegistrar" android:value="com.google.firebase.components.ComponentRegistrar" />
      <meta-data android:name="com.google.firebase.components:com.google.mlkit.vision.common.internal.VisionCommonRegistrar" android:value="com.google.firebase.components.ComponentRegistrar" />
      <meta-data android:name="com.google.firebase.components:com.google.mlkit.vision.text.internal.TextRegistrar" android:value="com.google.firebase.components.ComponentRegistrar" />
</service>

Adding this lines also a avoids the need to call MlKit.Initialize

moljac commented 1 month ago

Unfortunately this is a legacy app that has a lot of dependencies and would require quite an effort to upgrade.

It will be even more effort not to upgrade.

Anyway, I've been able to pass the GetClient exception by manually adding in the Android manifest the following lines that I've extracted from a 'modern' test app.

Does that solve the issue?

If yes. I guess this is solution/workaround for now, but please do migrate to MAUI and newer netX.0 targets (8, 9).

<provider android:name="com.google.mlkit.common.internal.MlKitInitProvider" android:authorities="com.TesySoftware.TLogClient.mlkitinitprovider" android:exported="false" android:initOrder="99" />
<service android:name="com.google.mlkit.common.internal.MlKitComponentDiscoveryService" android:directBootAware="true" android:exported="false" tools:targetApi="n">
      <meta-data android:name="com.google.firebase.components:com.google.mlkit.common.internal.CommonComponentRegistrar" android:value="com.google.firebase.components.ComponentRegistrar" />
      <meta-data android:name="com.google.firebase.components:com.google.mlkit.vision.common.internal.VisionCommonRegistrar" android:value="com.google.firebase.components.ComponentRegistrar" />
      <meta-data android:name="com.google.firebase.components:com.google.mlkit.vision.text.internal.TextRegistrar" android:value="com.google.firebase.components.ComponentRegistrar" />
</service>

Adding this lines also a avoids the need to call MlKit.Initialize

Thanks for letting us know. Appreciated.

mv4wd commented 1 month ago

I've also added these two sections in the manifest, and I'm able to analyze and extract Text items from the image succesfully. This is just a spare-time proof of concept, If a 'sponsor' will be interested the migration to MAUI is on the to-do list. Unfotunately I'm also dealing with old hardware used in the factories, with outdated Android, and any change is always a possible source of issues.

<service android:name="com.google.android.datatransport.runtime.backends.TransportBackendDiscovery" android:exported="false">
   <meta-data android:name="backend:com.google.android.datatransport.cct.CctBackendFactory" android:value="cct" />
</service>
<uses-library android:name="com.google.mlkit.text-recognition" android:required="false" />
moljac commented 1 month ago

@mv4wd Thanks for the feedback.

I've also added these two sections in the manifest,

Thanks for the feedback.

and I'm able to analyze and extract Text items from the image succesfully.

Great. Glad to hear that.

This is just a spare-time proof of concept, If a 'sponsor' will be interested the migration to MAUI is on the to-do list.

Do you have minimal repro sample?

Regarding MAUI: I need to see state of MLKit on iOS.

Unfotunately I'm also dealing with old hardware used in the factories, with outdated Android, and any change is always a possible source of issues.

I have similar issues with my app[s] (not living in US or western Europe).