Implementing custom post processing for different models

[Darshcg]

Hi team,

Is it possible to add the custom post-processing function in the middleware of the object detection application https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/stm32ai_application_code/object_detection ?

Thanks!

[GRATTINSTM]

Hello @Darshcg,

The model output post processing is done in the app_postprocess_run function in app_postprocess.c. This function calls the post processing function corresponding to the model type. The post processing functions are not accessible as there are part of the post processing library which is a compiled library. Though you can add a custom function to the post processing middleware by creating new source file and header file in the STM32_Objdetect_Postprocess folder. The code in the app_postprocess_run function can be modified to make it call your custom function.

Guillaume

[Darshcg]

Hi @GRATTINSTM,

Thank you for your response. I got the clarity of integrating the custom post-processing in the middleware.

Also, can we have a postprocessing code available for reference? also, I want to use images as input rather than camera input. Do we have any reference demos with image as an input for inference?

Thanks!

[GRATTINSTM]

Hi @Darshcg,

The postprocessing source code may be available in the future. Since we don't know when, I suggest you start the development of your custom post processing function. The post processing is public information, you can find it in the documentation of the model your are using. To use images instead of camera input, you can use GDB to inject data in place of the camera image. To do that you need to:

• rescale your image to the camera image resolution (CAM_RES_WIDTH, CAM_RES_HEIGHT) ;
• use a python script (or another software) to get an RGB565 binary file from the image ;
• in the embedded C code, place a breakpoint on the function Display_CameraPreview ;
• when the breakpoint is hit, use the restore GDB command to inject the RGB565 image file in the buffer App_Config.camera_capture_buffer ;
• run the continue command. The image should appear on the screen with the model output.

Guillaume

[Darshcg]

Hi @GRATTINSTM,

Thank you for your detailed response.

With respect to the post-processing code, I was wondering if I had to take care of any specific HAL API calls, or if it would be just a plain C code inference?

I would follow the steps you had provided for passing the image for inference. Thank you!

[GRATTINSTM]

Hi @Darshcg,

The post processing library is a plain C code, it doesn't call any HAL function.

Guillaume

[Darshcg]

Hi @GRATTINSTM,

Thank you for your reply. I got the clarity regarding integrating a custom post processing code in a middleware.

Thank you!

(Also, I filed a new issue of .elf file not exist during the flashing the code to the board. Would be great if you can assist). Thanks!

[Darshcg]

Hi @GRATTINSTM,

Just wanted to check if there is any specific way of creating a library(.a) for postprocessing? because I created a library out of my custom processing script, but when I am building the project I am getting a error with undefined reference to a function even though all seems to be correct.

Below is the error that I am getting:

**in function 'app_postprocess_run':", "D:/stm32ai-modelzoo/stm32ai_application_code/object_detection/Application/STM32H747I-DISCO/Src/CM7/app_postprocess.c:117: undefined reference to objdetect_custom_pp_process'", 'collect2.exe: error: ld returned 1 exit status', 'make: *** [makefile:69: STM32H747I-DISCO_GettingStarted_ObjectDetection_CM7.elf] Error 1', '"make -j8 all" terminated with exit code 2. Build might be incomplete.'**

Thank you!

[GRATTINSTM]

Hello @Darshcg,

I don't have enough information to help you. I would suggest you not to use a Lib if it is not necessary. You would better keep the source files. Take care to add the header files path to the CubeIDE project in Project -> Properties -> C/C++ Build->Settings -> MCU GCC Compiler -> Include paths, and to add the source files by drag&dropping them in the project folder tree.

Guillaume

[Darshcg]

Hi @GRATTINSTM,

Thank you a lot for your response.

Actually, I am not using the CubeIDE for this part. I am using the deployment reference from the readme and modified few scripts in order to plugin the custom postprocessing and run the stm32ai_main.py script for testing. I have manually added the header file in this path: https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/stm32ai_application_code/object_detection/Middlewares/ST/STM32_Objdetect_Postprocess/Inc.

And I also tried adding the C++ source script in this path https://github.com/STMicroelectronics/stm32ai-modelzoo/tree/main/stm32ai_application_code/object_detection/Middlewares/ST/STM32_Objdetect_Postprocess and then gave the path here for compilation: https://github.com/STMicroelectronics/stm32ai-modelzoo/blob/main/stm32ai_application_code/object_detection/Application/STM32H747I-DISCO/STM32CubeIDE/CM7/.project. But then it also doesn't seem to work actually( I mean the the .o, .su, .d, .cyclo files are not being generated if I manually give the c++ source file path in the .project and eventually getting the undefined reference issue while building.