Poblem with custom model

[PCPJ]

Hello everyone. I have a himax board and I am trying to deploy my own classification model. I am having some problems, hope someone can help me. My model is a simpler version of a mobilenet_v2 for classification. I removed he final layers of the network to make it smaller. I used this colab notebook as a guide line to convert my model to tensorflow lite for micro controllers. The final .cc file with the model data ended up with 368 Kbytes, way less than 1.5 Mega bytes which is the size of the peron detection model in the examples of this repository. I used the person detection example as a guide to deploy my model. I made the following modification:

• Replaced the model_data.cc and model_data.h files with my model files.
• Added the operations the my model uses in the micro mutable op resolver.
• Increased the kTensorArenaSize to the maximum size possible, without the makefile pointing out that the img file become too big.

When I flash this image, using my model, I got the following error message:

-----------------------------------------------------------                  
 Himax WEI Boot loader                                                       
------------------------------------------------------------                 

embARC Build Time: Jan  4 2021, 13:44:14                                     
Compiler Version: Metaware, 4.2.1 Compatible Clang 8.0.1 
Boot loader Version : 1.4.4 (Date:Jan  4 2021)
chip version : 0x8535a1 
cpu speed : 400000000 hz
spi speed : 50000000 hz
wake up evt:4
...secure lib version = 352380df9a347b1187d2361bfcd4455178a1ebcb
1st APPLICATION addr[3]=21000 (main-1966)
Bootloader Done !!!!!!
jump to app FW : 0x10000004
Arena size is too small for all buffers. Needed 14961408 but only 842320 was available.
AllocateTensors() failed

Why tensorflow is telling me that I need all this memory? My model is smaller than the person detection one in the examples. Does someone understand what is happening? I can provide my source files if needed, including the model data file. Any help is welcome, thank you so much.

[bigcat-himax]

Hi, PCPJ: Thanks for taking himax board as a platform to experiment your model, please let me classify the situation first: There will be 2 kinds of size for the model

1. Model data size, which can be found in the model data.cc (ex. in person_detect_model_data.cc, we can see the g_person_detect_model_data_size = 300568 bytes)
2. Tensor arena using size, this is the memory needed when invoke with the model (ex.examples/person_detection/main_functions.cc. tensor_arena[kTensorArenaSize] and int kTensorArenaSize = 136 * 1024 bytes)

As described in your post, "model data ended up with 368 Kbytes", so i think that is the first one, which is the model data size and as log shows "Arena size is too small for all buffers", which is second one, tensor arena size.

Usually, tensor arena size related to the tensor shape in the model, kernel weight/bias parameter and data type(floating/int8...) used in the model.

Thanks.

[PCPJ]

Hello @bigcat-himax. Thank you so much for the quick response. This is my model data size:

const unsigned int classification_model_data_size = 60424;

Do you have any idea where this number came from?

Arena size is too small for all buffers. Needed 14961408 but only 842320 was available.

Where does this number, 14961408, came from?

[bigcat-himax]

Hi, @PCPJ :

Arena size is too small for all buffers. Needed 14961408 but only 842320 was available.

This number 14961408 is the memory bytes size needed by TFLM and calculated while invoke.

Could you help to provide the tensor arena size information in your case? ex. in person_detection/main_functions.cc kTensorArenaSize = 136 * 1024;

and perhaps your model tflite file or .cc files also will help to analyze.

Thanks.

[PCPJ]

Sorry for not replaying earlier, @bigcat-himax. Today, a few hours after my last comment I found the problem. The tensor arena is the memory needed for the model parameters, and also the input tensors. The model data file has all the model parameters, but of course it don't have the input data. In my case the input was set to be a tensor of size 1,3,480,640, way bigger than the board can handle. That's why my model data file is smaller than the person detection example, and the memory needed was bigger. Now I re-created the model file, using a smaller input size, and the tensors was allocated successfully. Thank you so much for your attention. Now I have to retrain my model with this smaller input size. Wish me luck.

[bigcat-himax]

Hi, @PCPJ Thanks for the information, all the best.