[Bug] Xamarin.TensorFlow.Lite.Gpu does not define any types

[OutSorcerer]

Trying to use Xamarin.TensorFlow.Lite.Gpu version 2.1.0 like this

var gpuDelegate = new Xamarin.TensorFlow.Lite.GPU.GpuDelegate();

results in

Error   CS0234  The type or namespace name 'GPU' does not exist in the namespace 'Xamarin.TensorFlow.Lite' (are you missing an assembly reference?)

No types can be found in the Xamarin.TensorFlow.Lite.Gpu assembly using Object Browser in Visual studio as well as using a decompiler.

Interpreter from Xamarin.TensorFlow.Lite works fine in the same project. When comparing these two packages in decompiler Xamarin.TensorFlow.Lite has

    public static void RegisterPackages()
    {
      TypeManager.RegisterPackages(new string[2]
      {
        "org/tensorflow/lite",
        "org/tensorflow/lite/nnapi"
      }, new Converter<string, Type>[2]
      {
        new Converter<string, Type>(__TypeRegistrations.lookup_org_tensorflow_lite_package),
        new Converter<string, Type>(__TypeRegistrations.lookup_org_tensorflow_lite_nnapi_package)
      });
    }

but in Xamarin.TensorFlow.Lite.Gpu the corresponding method is empty:

 public static void RegisterPackages()
    {
      TypeManager.RegisterPackages(new string[0], new Converter<string, Type>[0]);
    }

Please check if the bug really exists.

Otherwise, maybe someone would be so kind as to post a small example with GpuDelegate (I was not able to find any examples of Xamarin.TensorFlow.Lite.Gpu on the web).

[moljac]

@OutSorcerer Thanks for the feedback.

api.xml

<api>
  <package name="org.tensorflow.lite.gpu" jni-name="org/tensorflow/lite/gpu">
    <class abstract="false" deprecated="not deprecated" extends="java.lang.Object" extends-generic-aware="java.lang.Object" jni-extends="Ljava/lang/Object;" final="true" name="GpuDelegate.Options" static="true" visibility="public" jni-signature="Lorg/tensorflow/lite/gpu/GpuDelegate$Options;">
      <constructor deprecated="not deprecated" final="false" name="GpuDelegate.Options" jni-signature="()V" bridge="false" static="false" type="org.tensorflow.lite.gpu.GpuDelegate.Options" synthetic="false" visibility="public">
      </constructor>
      <method abstract="false" deprecated="not deprecated" final="false" name="setInferencePreference" jni-signature="(I)Lorg/tensorflow/lite/gpu/GpuDelegate$Options;" bridge="false" native="false" return="org.tensorflow.lite.gpu.GpuDelegate.Options" jni-return="Lorg/tensorflow/lite/gpu/GpuDelegate$Options;" static="false" synchronized="false" synthetic="false" visibility="public">
        <parameter name="preference" type="int" jni-type="I">
        </parameter>
      </method>
      <method abstract="false" deprecated="not deprecated" final="false" name="setPrecisionLossAllowed" jni-signature="(Z)Lorg/tensorflow/lite/gpu/GpuDelegate$Options;" bridge="false" native="false" return="org.tensorflow.lite.gpu.GpuDelegate.Options" jni-return="Lorg/tensorflow/lite/gpu/GpuDelegate$Options;" static="false" synchronized="false" synthetic="false" visibility="public">
        <parameter name="precisionLossAllowed" type="boolean" jni-type="Z">
        </parameter>
      </method>
      <field deprecated="not deprecated" final="true" name="INFERENCE_PREFERENCE_FAST_SINGLE_ANSWER" jni-signature="I" static="true" transient="false" type="int" type-generic-aware="int" value="0" visibility="public" volatile="false">
      </field>
      <field deprecated="not deprecated" final="true" name="INFERENCE_PREFERENCE_SUSTAINED_SPEED" jni-signature="I" static="true" transient="false" type="int" type-generic-aware="int" value="1" visibility="public" volatile="false">
      </field>
    </class>
  </package>
</api>

So GpuDelegate should have been generated.

MCW - not generated!?!?! I need to see why.

content of aar:

tree ./externals/tensorflow-lite-gpu/
./externals/tensorflow-lite-gpu/
├── AndroidManifest.xml
├── R.txt
├── classes.jar
├── headers
│   └── tensorflow
│       └── lite
│           └── delegates
│               └── gpu
│                   └── delegate.h
├── jni
│   ├── arm64-v8a
│   │   └── libtensorflowlite_gpu_jni.so
│   ├── armeabi-v7a
│   │   └── libtensorflowlite_gpu_jni.so
│   ├── x86
│   │   └── libtensorflowlite_gpu_jni.so
│   └── x86_64
│       └── libtensorflowlite_gpu_jni.so
├── proguard.txt
└── res

So, native libs. Makes sense, but some c# should have been generated.

Decompiled classes.jar

java \
    -jar $HOME/Downloads/procyon-decompiler-0.5.36.jar \
    -jar ./externals/tensorflow-lite-gpu/classes.jar  \
    -o ./externals/tensorflow-lite-gpu-decompiled/

Gpu.Delegate.java:

// 
// Decompiled by Procyon v0.5.36
// 

package org.tensorflow.lite.gpu;

import java.io.Closeable;
import org.tensorflow.lite.Delegate;

public class GpuDelegate implements Delegate, Closeable
{
    private static final long INVALID_DELEGATE_HANDLE = 0L;
    private static final String TFLITE_GPU_LIB = "tensorflowlite_gpu_jni";
    private long delegateHandle;

    public GpuDelegate(final Options options) {
        this.delegateHandle = createDelegate(options.precisionLossAllowed, options.inferencePreference);
    }

    public GpuDelegate() {
        this(new Options());
    }

    public long getNativeHandle() {
        return this.delegateHandle;
    }

    public void close() {
        if (this.delegateHandle != 0L) {
            deleteDelegate(this.delegateHandle);
            this.delegateHandle = 0L;
        }
    }

    private static native long createDelegate(final boolean p0, final int p1);

    private static native void deleteDelegate(final long p0);

    static {
        System.loadLibrary("tensorflowlite_gpu_jni");
    }

    public static final class Options
    {
        public static final int INFERENCE_PREFERENCE_FAST_SINGLE_ANSWER = 0;
        public static final int INFERENCE_PREFERENCE_SUSTAINED_SPEED = 1;
        boolean precisionLossAllowed;
        int inferencePreference;

        public Options() {
            this.precisionLossAllowed = true;
            this.inferencePreference = 0;
        }

        public Options setPrecisionLossAllowed(final boolean precisionLossAllowed) {
            this.precisionLossAllowed = precisionLossAllowed;
            return this;
        }

        public Options setInferencePreference(final int preference) {
            this.inferencePreference = preference;
            return this;
        }
    }
}

import java.io.Closeable; - should be in Android (Mono.Android.dll).

https://docs.microsoft.com/en-us/dotnet/api/java.io.icloseable?view=xamarin-android-sdk-9

Confirmed.

import org.tensorflow.lite.Delegate; - needs reference to TensorFlow.Lite.Delegate??

OK. IDelegate was generated.

Reference might be wrong.

<Project Sdk="MSBuild.Sdk.Extras/2.0.54">

    <PropertyGroup>
        <AssemblyName>Xamarin.TensorFlow.Lite.Gpu</AssemblyName>

        <MonoAndroidResourcePrefix>Resources</MonoAndroidResourcePrefix>
        <MonoAndroidAssetsPrefix>Assets</MonoAndroidAssetsPrefix>
        <AndroidUseLatestPlatformSdk>False</AndroidUseLatestPlatformSdk>
        <AndroidUseIntermediateDesignerFile>True</AndroidUseIntermediateDesignerFile>
        <AndroidResgenFile>Resources\Resource.designer.cs</AndroidResgenFile>

        <TargetFrameworks>MonoAndroid81</TargetFrameworks>
        <AndroidCodegenTarget>XAJavaInterop1</AndroidCodegenTarget>
    </PropertyGroup>

    <PropertyGroup>
        <IsBindingProject>true</IsBindingProject>
        <AndroidClassParser>class-parse</AndroidClassParser>
    </PropertyGroup>

    <PropertyGroup>
        <!-- 
        nuget packaging
        -->
        <GeneratePackageOnBuild>true</GeneratePackageOnBuild>
        <PackageId>Xamarin.TensorFlow.Lite.Gpu</PackageId>
        <PackageVersion>2.1.0</PackageVersion>
        <Title>Xamarin.TensorFlow.Lite.Gpu</Title>
        <PackageDescription>
            Bindings for Google's TensorFlow Lite GPU package (Google Play Services dependency)
        </PackageDescription>
        <Owners>Microsoft</Owners>
        <Authors>Microsoft</Authors>
        <Copyright>© Microsoft Corporation. All rights reserved.</Copyright>
        <PackageLicenseUrl>https://go.microsoft.com/fwlink/?linkid=2013315</PackageLicenseUrl>
        <PackageProjectUrl>https://go.microsoft.com/fwlink/?linkid=2013420</PackageProjectUrl>
        <RepositoryUrl>https://github.com/tensorflow/tensorflow/</RepositoryUrl>
        <PackageTags>xamarin, android, bindings, google, tensorflow, lite, gpu</PackageTags>
        <PackageRequireLicenseAcceptance>true</PackageRequireLicenseAcceptance>
    </PropertyGroup>

    <ItemGroup>
        <TransformFile Include="transforms\*.xml" />
    </ItemGroup>

    <ItemGroup>
        <LibraryProjectZip Include="..\..\externals\tensorflow-lite-gpu.aar">
            <Link>Jars\tensorflow-lite-gpu.aar</Link>
        </LibraryProjectZip>
    </ItemGroup>

</Project>

Confirmed: Reference to TesnorFlow.Lite project is missing.

[moljac]

@OutSorcerer Can you provide minimal repro sample as zip and attach it here, please?

Any sample helps us a lot for testing etc.

[moljac]

Reminder message to myself: check package to namespace mapping Matadata.xml. Something is not working!

[moljac]

Solved: BuildAction was None

[moljac]

Bumps to do:

1.14.0 -> 1.14.0.1
1.15.0 -> 1.15.0.1
2.0.0  -> 2.0.0.1
2.1.0  -> 2.1.0.1

Will not be fixed (no GPU and too old):

1.12.0 -> 1.12.0.1

[moljac]

Fixed in PRs https://github.com/xamarin/XamarinComponents/pull/877 https://github.com/xamarin/XamarinComponents/pull/878 https://github.com/xamarin/XamarinComponents/pull/879 https://github.com/xamarin/XamarinComponents/pull/880

[moljac]

@OutSorcerer

published nugets

https://www.nuget.org/packages/Xamarin.TensorFlow.Lite/ https://www.nuget.org/packages/Xamarin.TensorFlow.Lite.Gpu/

Please report if the fix works and if yes - close the issue.

And again, sample would be appreciated (please)

[OutSorcerer]

@moljac, thank you very much for the fixes!

As you requested here is the code that reproduces this bug: https://github.com/OutSorcerer/MushroomDetector/blob/master/MushroomDetector.Android/TensorflowClassifier.cs#L26

I can also confirm that updating packages Xamarin.TensorFlow.Lite and Xamarin.TensorFlow.Lite.Gpu to 2.1.0.1 resolves the issue.

[moljac]

Thanks @OutSorcerer

I will add sample to the repo. I hope this is OK with you.

[moljac]

Ups. Seems like I will not. It is more that the repro sample.