mit-han-lab / tinyengine

[NeurIPS 2020] MCUNet: Tiny Deep Learning on IoT Devices; [NeurIPS 2021] MCUNetV2: Memory-Efficient Patch-based Inference for Tiny Deep Learning; [NeurIPS 2022] MCUNetV3: On-Device Training Under 256KB Memory
https://mcunet.mit.edu
MIT License
757 stars 127 forks source link

.patch file to build it for other boards #73

Closed senceryucel closed 1 year ago

senceryucel commented 1 year ago

Hey, @meenchen

For person_detection example, the "openmv_person_detection.patch" file handles everything for OpenMV H7 only.

For example, for the section:

diff --git a/src/omv/boards/OPENMV4/omv_boardconfig.h b/src/omv/boards/OPENMV4/omv_boardconfig.h
index 412de472..f7da2c03 100644
--- a/src/omv/boards/OPENMV4/omv_boardconfig.h
+++ b/src/omv/boards/OPENMV4/omv_boardconfig.h
@@ -150,16 +150,18 @@
 // The maximum available fb_alloc memory = FB_ALLOC_SIZE + FB_SIZE - (w*h*bpp).
 #define OMV_FFS_MEMORY          DTCM        // Flash filesystem cache memory
 #define OMV_MAIN_MEMORY         SRAM1       // data, bss and heap memory
+#define OMV_MAIN_MEMORY2        SRAM5       // my memory
 #define OMV_STACK_MEMORY        ITCM        // stack memory
 #define OMV_DMA_MEMORY          SRAM2       // DMA buffers memory.
 #define OMV_FB_MEMORY           AXI_SRAM    // Framebuffer, fb_alloc
 #define OMV_JPEG_MEMORY         SRAM3       // JPEG buffer memory.
 #define OMV_VOSPI_MEMORY        SRAM4       // VoSPI buffer memory.

-#define OMV_FB_SIZE             (400K)      // FB memory: header + VGA/GS image
-#define OMV_FB_ALLOC_SIZE       (100K)      // minimum fb alloc size
+#define OMV_FB_SIZE             (100K)      // defualt: 400 FB memory: header + VGA/GS image
+#define OMV_FB_ALLOC_SIZE       (50K)      // default: 100 minimum fb alloc size
 #define OMV_STACK_SIZE          (64K)
-#define OMV_HEAP_SIZE           (236K)
+#define OMV_HEAP_SIZE           (136K)
+// #define OMV_HEAP_SIZE           (236K)

 #define OMV_LINE_BUF_SIZE       (3 * 1024)  // Image line buffer round(640 * 2BPP * 2 buffers).
 #define OMV_MSC_BUF_SIZE        (2K)        // USB MSC bot data

This is the copied version of OPENMV4. That is why changing the "OPENMV4" parts with "OPENMV4P" is not enough to successfully build it. Further changes required since the "OPENMV4P/omv_boardconfig.h" is completely different than "OPENMV4/omv_boardconfig.h".

Potential changes are tough to guess, so it would be excellent to have some information about how to modify this .patch file. Besides, if there exists any further changes needed, I would be appreciated if you can also mention them.

Thanks in advance.

meenchen commented 1 year ago

Hi @senceryucel,

That section defines the memory allocation of the openmv cam. Since our demo does not require high resolution so we can reduce the size of framebuffer, but I am not sure if this statement holds for the OPENMV4P.

Anyway, I create a new patch for OPEMV4P https://github.com/mit-han-lab/tinyengine/pull/74/files#diff-84b614231826accb212d46fb3b52e41f72a7a2a6767bd30b964f1684c569027d. I am able to compile with TARGET=OPENMV4P after applying that patch. Unfortunately, I don't have that board so I cannot test it out.

Feel free to give it a try and let me know if there is any other issues.

senceryucel commented 1 year ago

Hi, @meenchen

Thanks for all you have done, really appreciate it.

I've tried it with the patch you have created. It produced some other errors, that can be seen below:

src/winc.c: In function 'wifi_callback_sta':
src/winc.c:315:21: error: dereferencing type-punned pointer will break strict-aliasing rules [-Werror=strict-aliasing]
                     ipconfig.u32StaticIP = *((uint32_t*)ifconfig.ip_addr);
                     ^~~~~~~~
src/winc.c:316:21: error: dereferencing type-punned pointer will break strict-aliasing rules [-Werror=strict-aliasing]
                     ipconfig.u32SubnetMask = *((uint32_t*)ifconfig.subnet_addr);
                     ^~~~~~~~
src/winc.c:317:21: error: dereferencing type-punned pointer will break strict-aliasing rules [-Werror=strict-aliasing]
                     ipconfig.u32Gateway = *((uint32_t*)ifconfig.gateway_addr);
                     ^~~~~~~~
src/winc.c:318:21: error: dereferencing type-punned pointer will break strict-aliasing rules [-Werror=strict-aliasing]
                     ipconfig.u32DNS = *((uint32_t*)ifconfig.dns_addr);
                     ^~~~~~~~
CC src/m2m_ssl.c
CC src/flexible_flash.c
CC src/nmi2c.c
CC src/m2m_crypto.c
CC src/nmuart.c
cc1: all warnings being treated as errors
Makefile:19: recipe for target '/home/senceryucel/Desktop/tinyengine/examples/openmv_person_detection/openmv/src/build/drivers/winc1500/src/winc.o' failed
make[1]: *** [/home/senceryucel/Desktop/tinyengine/examples/openmv_person_detection/openmv/src/build/drivers/winc1500/src/winc.o] Error 1
make[1]: Leaving directory '/home/senceryucel/Desktop/tinyengine/examples/openmv_person_detection/openmv/src/drivers/winc1500'
omv/ports/stm32/omv_portconfig.mk:593: recipe for target 'FIRMWARE_OBJS' failed
make: *** [FIRMWARE_OBJS] Error 2
make: Leaving directory '/home/senceryucel/Desktop/tinyengine/examples/openmv_person_detection/openmv/src'
meenchen commented 1 year ago

Hi @senceryucel,

Did you apply openmv_person_detection.patch before the new patch? The new patch is based on the old patch, so you may encounter some errors if only applying the new patch.

Note: This error should be fixed by removing the -Werror from -CFLAGS in src/omv/ports/stm32/omv_portconfig.mk(inside the old patch: https://github.com/mit-han-lab/tinyengine/blob/d412e116593a2b26c8504ccfaadd727a770a4219/examples/openmv_person_detection/openmv_person_detection.patch#L534)