Build Errors and Improper Inference for Patch-Based Inference

[goats-9]

I am trying to run the inference tutorial on STM32Cube IDE 1.5.0 as documented in the repo. However, on generating code using the script examples/vww_patchbased.py and moving the generated folder codegen to the inference project folder tutorial/TinyEngine_vww_tutorial/Src/TinyEngine, the following errors are seen in the IDE.

Description Resource    Path    Location    Type
make: *** [makefile:88: TTE_demo_mcunet.elf] Error 1    TTE_demo_mcunet         C/C++ Problem
more undefined references to `arm_nn_requantize' follow depthwise_kernel3x3_stride2_inplace_CHW.c   /TTE_demo_mcunet/Src/TinyEngine/codegen/Source  line 139    C/C++ Problem
undefined reference to `arm_nn_mat_mult_kernel_s8_s16_reordered'    convolve_1x1_s8.c   /TTE_demo_mcunet/Src/TinyEngine/src/kernels/int_forward_op  line 69 C/C++ Problem
undefined reference to `arm_nn_requantize'  depthwise_kernel3x3_stride1_inplace_CHW.c   /TTE_demo_mcunet/Src/TinyEngine/codegen/Source  line 110    C/C++ Problem
undefined reference to `arm_nn_requantize'  depthwise_kernel3x3_stride1_inplace_CHW.c   /TTE_demo_mcunet/Src/TinyEngine/codegen/Source  line 116    C/C++ Problem
undefined reference to `arm_nn_requantize'  depthwise_kernel3x3_stride1_inplace_CHW.c   /TTE_demo_mcunet/Src/TinyEngine/codegen/Source  line 139    C/C++ Problem
undefined reference to `arm_nn_requantize'  depthwise_kernel3x3_stride2_inplace_CHW.c   /TTE_demo_mcunet/Src/TinyEngine/codegen/Source  line 110    C/C++ Problem
undefined reference to `arm_nn_requantize'  depthwise_kernel3x3_stride2_inplace_CHW.c   /TTE_demo_mcunet/Src/TinyEngine/codegen/Source  line 116    C/C++ Problem

IDE console output is below.

Do let me know if any other data is required.

[goats-9]

There is a workaround to build the patch-based inference codebase.

Note: The following steps have to be done after cloning the repo and installing the python packages necessary.

1. Copy arm_nn_mat_mult_kernel_s8_s16_reordered.c from the CMSIS library to TinyEngine/src. Execute the below command from the root of the repo.

   cp TinyEngine/third_party/CMSIS/CMSIS/NN/Source/ConvolutionFunctions/arm_nn_mat_mult_kernel_s8_s16_reordered.c TinyEngine/src/kernels/int_forward_op/

2. Modify tinyengine_function.h as shown below.

   
   diff --git a/TinyEngine/include/tinyengine_function.h b/TinyEngine/include/tinyengine_function.h
   index 55f379c..bb4ed0b 100644
   --- a/TinyEngine/include/tinyengine_function.h
   +++ b/TinyEngine/include/tinyengine_function.h
   @@ -17,6 +17,9 @@
   * Target ISA:  ARMv7E-M
   * -------------------------------------------------------------------- */

+#include "arm_nnfunctions.h" +#include "arm_nnsupportfunctions.h" +

include

include

include

However, on running the code on the board, I am unable to get an inference result (using `mcunet-vww1`), with the network always predicting _No Person_ even when a person is displayed to the camera. I am unsure if these changes are the cause of wrong inference.

[rahulvigneswaran]

@goats-9

Copy arm_nn_mat_mult_kernel_s8_s16_reordered.c from the CMSIS library to TinyEngine/src. Execute the below command from the root of the repo.

Simply renaming the function arm_nn_mat_mult_kernel_s8_s16_reordered in convolve_1x1_s8.c to arm_nn_mat_mult_kernel_s8_s16_reordered_8mul does the trick.

[goats-9]

@rahulvigneswaran

Thanks for the reply. We tried renaming the function to both _8mul and oddch prefixes already, but inference results did not improve. We believe the issue may be among the following.

1. Code generation for patch-based inference (in examples/vww_patchbased.py)
2. Models are not pretrained (which is very unlikely looking at the results published by the authors).

Nevertheless, I will add these inputs to the workarounds stated above.

[rahulvigneswaran]

@goats-9

Building upon your insights, we've explored additional strategies, yet encountered persistent issues. Here's a summary:

• Experimented with diverse patch grid configurations.
• Set the 'inplace' parameter to False.

While these adjustments occasionally impact prediction outcomes, the issue of classifying 'person' and 'no person' as the same remains consistent.