Meaning of TF_LITE_STATIC_MEMORY

[Gostas]

I am building an updated version of TFLM for Arduino and wanted to understand the meaning of TF_LITE_STATIC_MEMORY.

The reason is that if I define it, the library won't work correctly - for instance the hello_world example will only output 0. After some debugging I noticed that different versions of some classes like TfLiteNode and TfLiteTensor were used and narrowed it down to the TF_LITE_STATIC_MEMORY define.

In MANIFEST.ini the following files are patched to change TF_LITE_STATIC_MEMORY to ARDUINO:

• src/tensorflow/lite/c/common.h
• src/tensorflow/lite/core/api/flatbuffer_conversions.cpp
• src/tensorflow/lite/kernels/internal/compatibility.h
• src/tensorflow/lite/kernels/internal/types.h
• src/tensorflow/lite/kernels/kernel_util.cpp
• src/tensorflow/lite/kernels/kernel_util.h
• src/tensorflow/lite/micro/compatibility.h

However the define is still used in the following files.

• src/tensorflow/lite/core/macros.h
• src/tensorflow/lite/core/c/common.cpp
• src/tensorflow/lite/core/c/common.h

The Arduino IDE obviously doesn't define TF_LITE_STATIC_MEMORY and it works correctly this way. So is it done on purpose that it is not replaced by ARDUINO in these 3 files?

BTW I found this thread in the Google group:

Nick Kreeger Nov 12, 2020, 10:11:54 a.m. to SIG Micro TL;DR;

If you explicitly have TF_LITE_STATIC_MEMORY defined in a Make or build config, please update that build define to TF_LITE_MICRO.

NOTE: This is not a flag that developers can reply on - it is not explicitly set on certain platforms (e.g. Arduino).

More details: The define TF_LITE_STATIC_MEMORY was originally introduced to work around methods and functionality inside TFLite that would introduce dynamic memory usage (e.g. malloc/free). This functionality was needed for embedded devices that generally do not use dynamic memory allocation. Since the introduction of this flag, TFLite Micro has grown and is the main use case for this flag. Specific memory optimizations and functionality exist in TFLite and TFLite Micro code under this flag. To help clear up the real reason for this flag, the name is being changed to TF_LITE_MICRO.

Thanks, Nick & The TFLite Micro Team

[rascani]

This define is used in code that is common between TFLite and TFLM to eliminate dynamic memory allocation for TFLM. If your platform supports dynamic memory allocation, then it might work, but the tested path for TFLM is always with TF_LITE_STATIC_MEMORY defined.