Crash on Android using exported models from latests azure custom vision update

[AlejandroRuiz]

Hey the plugin work fine with old exported models but since the last updates the models are throwing the next error

  at Xam.Plugins.OnDeviceCustomVision.ImageClassifierImplementation.ClassifyImage (System.IO.Stream imageStream) [0x00138] in <51f1c3984df84204baf85f264ebf2f6e>:0 
  --- End of managed Java.Lang.IllegalArgumentException stack trace ---
java.lang.IllegalArgumentException: Input to reshape is a tensor with 97556 values, but the requested shape requires a multiple of 90944
     [[Node: block0_0_reshape0 = Reshape[T=DT_FLOAT, Tshape=DT_INT32, _device="/job:localhost/replica:0/task:0/device:CPU:0"](block0_0_concat, block0_0_reshape0/shape)]]

[jpd21122012]

Hey the plugin work fine with old exported models but since the last updates the models are throwing the next error at Xam.Plugins.OnDeviceCustomVision.ImageClassifierImplementation.ClassifyImage (System.IO.Stream imageStream) [0x00138] in <51f1c3984df84204baf85f264ebf2f6e>:0 --- End of managed Java.Lang.IllegalArgumentException stack trace --- java.lang.IllegalArgumentException: Input to reshape is a tensor with 97556 values, but the requested shape requires a multiple of 90944 [[Node: block0_0_reshape0 = Reshape[T=DT_FLOAT, Tshape=DT_INT32, _device="/job:localhost/replica:0/task:0/device:CPU:0"](block0_0_concat, block0_0_reshape0/shape)]]

I have the solution, please send me an email and I'll help you.

[AlejandroRuiz]

@jpd21122012 if is a fix on code you can create a PR to the plugin code or put the solution over here to allow other devs see the solution, thanks

[feliperugai]

I'm getting the same error. Did you guys were able to solve this? @jpd21122012 did you PR that?

[Hisaack]

Getting the same error here please? Any help or some solved it?

[feliperugai]

Sorry for not posting before, I was planning to make a Pull Request with the solution, but I saw that there was already a pull request opened last year and it's not yet aproved or rejected, so I though it wouldn't have the attention. You can solve the issue by getting the model InputSize dynamically and not hardcoded (227). My model for example is 225, and Tensorflow refused to accept another size after the initialization with 227. Here's what I did: In the Android implementation (/Xam.Plugins.OnDeviceCustomVision/ImageClassifier.android.cs), I added this line at the Init(string modelName, string labelsFileName, ModelType modelType) method:

InputSize = Convert.ToInt32(_inferenceInterface.GraphOperation(InputName).Output(0).Shape().Size(1));

It worked like a charm. Hope that can help you out.

[Hisaack]

Since I downloaded the Xam.Plugins.OnDeviceCustomVision on Nuget packages, how can I find Xam.Plugins.OnDeviceCustomVision/ImageClassifier.android.cs class. It is a metadata file that cant be edited? Any help please ?

[AlejandroRuiz]

@Hisaack you need to modify the source code and recompile it then target the new dll to your project

[AlejandroRuiz]

@felrugai thanks for sharing the fix, lets ask @jimbobbennett if is possible to get this added in a new nuget version of the plugin

[jimbobbennett]

@felrugai - can you raise a PR for this?

[feliperugai]

@jimbobbennett Just did a PR but unfortunatelly I couldn't test it, are the .csproj/.sln missing the source files? I mean, they are in the folder structure but I couldn't find them inside the solution, so I just edited the one file. Actually in my project I changed a lot, but now I don't have time to make a PR, later on I'll do it. I don't know if you agree, but I'm using DependencyService instead of your way of getting the singletons.

[jimbobbennett]

It gets worse and worse - they've changed something so if I export a model today using your changes it crashes with a different error... Will investigate...

[jimbobbennett]

Ok - I'm stumped. I've pushed a new preview NuGet package using the changes you made so try that with your models, but it doesn't work with the latest models. Checking the samples and those haven't been updated, but something has changed. I'll raise it with the product teams and see what I can find out.

[feliperugai]

Nice, I'll try my best to help to fix it for new models as I will need to recreate my model soon

[jimbobbennett]

The first issue is the output name appears to have changed - loss is gone, and the example python code has model_outputs. I tried just changing this, but now I get a buffer overflow exception.

I've raised it with the team - https://github.com/Azure-Samples/cognitive-services-android-customvision-sample/issues/24. Hopefully they'll fix that sample and I can duplicate their code.

[Hisaack]

@Jimbobbennett can we test it with latest Nuget Package Preview for the Models we got from Custom Vision? Hope it will solve this error of ..ArgumentException: Input to reshape is a tensor with 97556 values, but the requested shape requires a multiple of 90944

[jimbobbennett]

And I feel dumb - I was testing with the wrong model type.

@Hisaack - the NuGet package is up on NuGet as 2.1.1-alpha. I'll test it and mark it as stable later today.

[feliperugai]

Hahahahaha it's part of the game. Waiting for the stable version today :)

[Hisaack]

Hey guys, did you notice that when you test the model with Android, the accuracy is not correct???? Panicking though :(. How can I Improve my accuracy on Android ?

@ jimbobbennett, @felrugai, @AlejandroRuiz

[Hisaack]

Project type I used on Custom Vision is Classification for the Domain type General (compact).

[jimbobbennett]

Fixed in the latest package - thanks @felrugai for the PR!

[AlejandroRuiz]

thanks @jimbobbennett and @felrugai