xamarin / GooglePlayServicesComponents

Other
314 stars 146 forks source link

Xamarin.Google.MLKit.Vision.Objects.Defaults.ObjectDetectorOptions.Builder has no required public methods from original Java class #434

Open mmarinchenko opened 3 years ago

mmarinchenko commented 3 years ago

Xamarin.Android Version:

11.1.0.26 (Visual Studio Community 2019 v16.8.5)

Operating System & Version:

Windows 10 Home Single Language 20H2 v19042.804

Google Play Services Version

117.6.0 with AndroidX dependencies

Relevant information

Packages used:

    <Reference Include="Mono.Android" />
    <Reference Include="System" />
    <PackageReference Include="Xamarin.Google.Android.Material">
      <Version>1.3.0</Version>
    </PackageReference>
    <PackageReference Include="Xamarin.Google.MLKit.ObjectDetection">
      <Version>116.2.0</Version>
    </PackageReference>

Google ML Kit Object Detection Guide with Kotlin and Java samples:

Related Google ML Kit docs:

Minimal Repro Code Sample

Get the following Java code from Google ML Kit Object Detection Guide (link provided above):

// Multiple object detection in static images
ObjectDetectorOptions options =
        new ObjectDetectorOptions.Builder()
                .setDetectorMode(ObjectDetectorOptions.SINGLE_IMAGE_MODE)
                .enableMultipleObjects()
                .enableClassification()  // Optional
                .build();

and write it in C#:

using Xamarin.Google.MLKit.Vision.Objects.Defaults;
<...>
// Multiple object detection in static images
var options = new ObjectDetectorOptions.Builder()
        .SetDetectorMode(ObjectDetectorOptions.SingleImageMode)
        .EnableMultipleObjects()
        .EnableClassification()  // Optional
        .Build();

This will not compile because all Builder's methods are missing, except Build().

mmarinchenko commented 3 years ago

I tried to overcome this by implementing a subclass of ObjectDetectorOptionsBase which has its own nested protected internal abstract class Builder with required methods.

Result looks odd... I had to implement 2 nested classes for that:

DetectorOptions.cs (click to expand) ```csharp public sealed class DetectorOptions : ObjectDetectorOptionsBase // Java.Lang.Object type { private DetectorOptions(BaseBuilder baseBuilder) : base(baseBuilder) { } private sealed class BaseBuilder : ObjectDetectorOptionsBase.Builder // Java.Lang.Object type { private readonly Builder _builder; public BaseBuilder(Builder builder) => _builder = builder; protected override ObjectDetectorOptionsBase Build() => _builder.Build(); } new public sealed class Builder // pure managed type, hides base.Builder { private readonly BaseBuilder _baseBuilder; private readonly DetectorOptions _detectorOptions; public Builder() { _baseBuilder = new BaseBuilder(this); _detectorOptions = new DetectorOptions(_baseBuilder); } public Builder SetDetectorMode(int detectorMode) { _baseBuilder.SetDetectorMode(detectorMode); return this; } public Builder SetExecutor(IExecutor executor) { _baseBuilder.SetExecutor(executor); return this; } public Builder EnableClassification() { _baseBuilder.EnableClassification(); return this; } public Builder EnableMultipleObjects() { _baseBuilder.EnableMultipleObjects(); return this; } public DetectorOptions Build() => _detectorOptions; } } ```

So now I can write this in C# and it compiles fine:

var options = new DetectorOptions.Builder()
        .SetDetectorMode(ObjectDetectorOptionsBase.SingleImageMode)
        .EnableMultipleObjects()
        .EnableClassification()  // Optional
        .Build();

But unfortunately this doesn't compile in Java due to DetectorOptions.BaseBuilder is translated into a separate public class which does not have access to the nested protected ObjectDetectorOptionsBase.Builder:

Output (click to expand) ``` 1> 2 errors 1>...\DetectorOptions_BaseBuilder.java(5,67): javac.exe error JAVAC0000: error: Builder has protected access in ObjectDetectorOptionsBase 1>...\DetectorOptions_BaseBuilder.java(5,67): javac.exe error JAVAC0000: extends com.google.mlkit.vision.objects.ObjectDetectorOptionsBase.Builder 1>...\DetectorOptions_BaseBuilder.java(5,67): javac.exe error JAVAC0000: 1>...\DetectorOptions_BaseBuilder.java(22,7): javac.exe error JAVAC0000: error: cannot find symbol 1>...\DetectorOptions_BaseBuilder.java(22,7): javac.exe error JAVAC0000: if (getClass () == DetectorOptions_BaseBuilder.class) 1>...\DetectorOptions_BaseBuilder.java(22,7): javac.exe error JAVAC0000: symbol: method getClass() 1>...\DetectorOptions_BaseBuilder.java(22,7): javac.exe error JAVAC0000: location: class DetectorOptions_BaseBuilder ```
DetectorOptions_BaseBuilder.java (click to expand) ```java package crc641d15e8c705d2fdbc; public class DetectorOptions_BaseBuilder extends com.google.mlkit.vision.objects.ObjectDetectorOptionsBase.Builder // first compilation error implements mono.android.IGCUserPeer { /** @hide */ public static final String __md_methods; static { __md_methods = "n_build:()Lcom/google/mlkit/vision/objects/ObjectDetectorOptionsBase;:GetBuildHandler\n" + ""; mono.android.Runtime.register ("MSSLink.CameraApp.Detection.DetectorOptions+BaseBuilder, MSSLink.CameraApp", DetectorOptions_BaseBuilder.class, __md_methods); } public DetectorOptions_BaseBuilder () { super (); if (getClass () == DetectorOptions_BaseBuilder.class) // second compilation error mono.android.TypeManager.Activate ("MSSLink.CameraApp.Detection.DetectorOptions+BaseBuilder, MSSLink.CameraApp", "", this, new java.lang.Object[] { }); } public com.google.mlkit.vision.objects.ObjectDetectorOptionsBase build () { return n_build (); } private native com.google.mlkit.vision.objects.ObjectDetectorOptionsBase n_build (); private java.util.ArrayList refList; public void monodroidAddReference (java.lang.Object obj) { if (refList == null) refList = new java.util.ArrayList (); refList.add (obj); } public void monodroidClearReferences () { if (refList != null) refList.clear (); } } ```

Is there a way to translate a nested Java.Lang.Object type written in C# to a nested class in Java while preserving the access modifiers?

AlexDevMobile commented 3 years ago

@mmarinchenko Hi, did you find a solution for this?

mmarinchenko commented 3 years ago

@AlerzDev Hi! I have 2 issues here:

I'm not sure it's right to discuss second issue here. Maybe it should be moved to Xamarin.Android repo. Anyway I don't have a solution for this.


As for original issue I assume the following bindings are incomplete:

Each Transforms/Metadata.xml file in these directories has custom binding rules and the following (or similar) comment:

    <!--
    TODO: possible problems! return type covariance/contravariance (needs investigation)

    removing `build` with return="<...>"
    renaming in Addditions
    -->

but only one of corresponding Additions/***Options.cs files (the image-labeling-automl one) actually implements some logic.

So I'm waiting for a comment from @moljac. Or from someone else who can bring this to light :)

moljac commented 3 years ago

@AlerzDev

Thanks for the feedack

missing methods in some Builder classes (original one)

That happens for various reasons (difference between java and c# idioms, generics implementations,...). Even our tooling and our/my mistakes.

I rushed to publish MLKit and to catch up with google, so I might have made some mistakes.

Furthermore - the quality of bindings cannot be determined w/o samples or API testing (unit test). Are you getting where am I heading? The more minimal sample you guys can provide us - the better.

*Options classes were tough because of inheritance and covariance (return types). I need to dive in and check and am not sure when I'll be ready to do so. Namely I need to do tons of updates in XamarinComponents which fix several bugs and are crucial for new updates of AndroidX and GPS-FB-MLKit

mmarinchenko commented 3 years ago

@moljac

Hi! I guess you answered the wrong person :) Anyway thanks for the explanation.

I spent some time playing with build process lately (PR #437). I'll try to write sample which cover my needs in separate PR.

moljac commented 3 years ago

@mmarinchenko

I'll need to dive deeper into the bindings, but sample would help me a lot.

Bindings are important, but samples are also important to verify the bindings.

I finished tons of updates in XamarinComponents and want to fix something from months ago, before I can make further steps (updates, samples and tooliing)

tomkulaga commented 3 years ago

Did you guys have any success with getting the builder options working?

mmarinchenko commented 3 years ago

@tomkulaga

Sorry, I'm busy with another project right now. @moljac is busy too, I guess.

tomkulaga commented 3 years ago

@mmarinchenko

no problems. Did you every try and compile from source and edit the Additions.cs like the image automl?

I’m trying that but keep getting build errors on latest branch and latest tagged release

mmarinchenko commented 3 years ago

Xamarin.Android v11.2 (VS v16.9.*) has issues related to Kotlin metadata (ex. see https://github.com/xamarin/java.interop/issues/826). But I haven't tried VS v16.10 yet.

tomkulaga commented 3 years ago

I had mode success in 16.10 than 16.11 preview.

It finished the binderate target but bailed on the line.

I opened the generated SLN but it failed on an Android generator step late in the build. I was building MLKIT Obejct Detection and got 10/12 built.

Gave up after that :(

Sent from my iPhone

On 1 Jun 2021, at 4:09 pm, Maxim Marinchenko @.***> wrote:

 Xamarin.Android v11.2 (VS v16.9.*) has issues related to Kotlin metadata (ex. see xamarin/java.interop#826). But I haven't tried VS v16.10 yet.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

alexgg94 commented 2 years ago

Any update on this?

tomkulaga commented 2 years ago

I was able to build them BUT couldn't fix the original issue, so gave up.

mmarinchenko commented 2 years ago

For those who may hit this issue and can't wait for Google MLKit fix in Xamarin.

ONNX can be used for ML tasks on mobile platforms as an alternative.

Blog post: https://devblogs.microsoft.com/xamarin/machine-learning-in-xamarin-forms-with-onnx-runtime/ Sample: https://github.com/microsoft/onnxruntime-inference-examples/tree/main/mobile/examples/Xamarin

tomkulaga commented 2 years ago

Were you able to get realtime object detection working with Onnx?

mmarinchenko commented 2 years ago

No, I haven't tried it yet.