TFLite parser: failed to CreateNetworkFromBinaryFile with squeeznet.tflite and inceptionv3.tflite

[vinceab]

Both inceptionv3.tflite and squeeznet.tflite models are floating point models and come from: https://www.tensorflow.org/lite/guide/hosted_models

The following code: armnnTfLiteParser::ITfLiteParserPtr parser = armnnTfLiteParser::ITfLiteParser::Create(); armnn::INetworkPtr network = parser->CreateNetworkFromBinaryFile("squeeznet.tflite");

Leads to the following issue: terminate called after throwing an instance of 'armnn::ParseException' what(): Buffer #32 has 0 bytes. For tensor: [1,1001] expecting: 4004 bytes and 1001 elements. at function CreateConstTensor

Using inceptionv3.tflite instead of squeeznet.tflite does not output any error but CreateNetworkFromBinaryFile function is never ending.

[Colm-in-Arm]

Hi Vincent,

Can you tell me what target environment and which version of ArmNN you were using please. I'll try and recreate on x86_64 with ArmNN master today.

Colm.

[vinceab]

Hi Colm,

I am using armnn 20.05 on a arm 32bits system STM32MP1 dual cortexA7.

Vincent

[Colm-in-Arm]

Hi Vincent,

I was able to parse squeeznet.tflite on x86_64. I didn't encounter the problem in CreateConstTensor.

However, there was an exception around the final reshape layer. It complains about the target shape not being one dimensional. It looks like there's a difference in interpretation between TF and Android. The Android definition defines shape as "1: A 1-D tensor of ANEURALNETWORKS_TENSOR_INT32" while the TF definition is a little looser. Our implementation focuses on the Android definition.

The inceptionv3.tflite network loaded and executed normally.

I'll try again on v7 32bit Raspberry Pi just in case there's a difference.

Colm.

[Colm-in-Arm]

I just spotted we have a plan to fix the difference in shape handling between Android NDK and TF.

[Colm-in-Arm]

Hi Vincent,

I've managed to recreate both problems on v7 32bit. It occurs on both 20.05 and on master. I'll start investigating next week.

Colm.

[Colm-in-Arm]

Hi Vincent,

There's good news and bad...

I've fixed the exception handling in the TfLiteParser so now the squeeznet.tflite fails gracefully. This is the commit on master.

It still means that this model fails because the format of the Reshape parameter is unsupported. There is an open bug that @TeresaARM reported. It is highly likely to be fixed before the 20.08 release.

There's also a solution for inceptionv3.tflite: time. I ran it on a Raspberry Pi 3B+ and it took 7m22.207s. I didn't check the accuracy but it did complete without errors. Of the total time, inference took 99.09% of it. Does the length of execution cause a problem on your system?

Colm.

[vinceab]

Hi Colm,

Thanks for the information. I understand that for squeeznet, a fix will be likely integrated into 20.08 release. It sounds good for me. For inceptionv3.tflite , the 7m22.207s you mention is for one inference? Because inceptionv3.tflite seems to be lock at CreateNetworkFromBinaryFile function. So I understand you suggest me to wait longer right? Or did you also fix something in the CreateNetworkFromBinaryFile parser?

Vincent

[Colm-in-Arm]

Hi Vincent,

Yes 7 minus for 1 inference on CpuRef. I didn't encounter any problem in CreateNetworkFromBinaryFile with inceptionv3.tflite. I just tried it again with the 20.05 release version and it gets past network loading and optimization. The same inference on CpuAcc takes 1.4 seconds.

My environment is: Raspberry Pi 3B+(ArmV7 32). Cross compiled with gcc-arm-linux-gnueabihf/bionic-updates,bionic-security,now 4:7.4.0-1ubuntu2.3 amd64 on x_86_64 Ubuntu 18.04.

Colm.

[vinceab]

Hi Colm,

I confirm that inceptionv3 is running fine. It was actually a RAM issue. I did not have enough RAM with the STM32MP157C-DK2 board (with 512MB of RAM). I move to the STM32MP157C-EV1 (with 1GB of RAM) and the model is running fine.

So only the squeeznet is finaly having trouble.

Thanks Vincent

[james-conroy-arm]

Hi @vinceab ,

Are you still experiencing issues with the SqueezeNet model?

Cheers, James

[vinceab]

The fix mention by colm avoid the application to crash that's a good point but the Squeeznet is still not supported.

[janeil01]

Hi @vinceab,

I just added a patch to master that should fix the issue in the Reshape operation. @Colm-in-Arm mentioned you had a problem with that when running squeezenet. You could give that a go and see if you can execute squeezenet with it :)

Good luck Jan

[vinceab]

Hi Jan,

I have verified the patch and squeeznet is now running successfully.

Thank you Vincent