search

mit-han-lab / tinyengine

[NeurIPS 2020] MCUNet: Tiny Deep Learning on IoT Devices; [NeurIPS 2021] MCUNetV2: Memory-Efficient Patch-based Inference for Tiny Deep Learning; [NeurIPS 2022] MCUNetV3: On-Device Training Under 256KB Memory
https://mcunet.mit.edu
MIT License
792 stars 130 forks source link

patch inference #7

Closed hqulxw123 closed 1 year ago

hqulxw123 commented 2 years ago

thanks for your excellent jobs! it's very useful for me to porting ai model on low power edge devices. i'm very interesting about the patch inference method, but i can't find anymore information about the patch method in the codebase. will you provide this code?

tonylins commented 2 years ago

Hi, the patch-based inference feature has not been added to the repo yet. Please stay tuned! Thanks.

hqulxw123 commented 2 years ago

Hi, the patch-based inference feature has not been added to the repo yet. Please stay tuned! Thanks.

I'm very interested about the patch-based inference. can you point out the principle?

rodonguyen commented 1 year ago

I'm very interested about the patch-based inference. can you point out the principle?

Hi, the team's paper would be a great start to understanding the patch-based method. If you haven't read it, consider having a look at https://arxiv.org/abs/2110.15352

meenchen commented 1 year ago

Hi, we have released the patch-based inference feature. Feel free to check out the tutorial and the code generation script for more details.

EricWu-ZL commented 1 year ago

Hi, we have released the patch-based inference feature. Feel free to check out the tutorial and the code generation script for more details.

Hi, @meenchen. Thanks for your great jobs. When I use patch-based inferencing tutorial, the c file 'patchpadding_convolve_s8_kernel3_inputch3_stride2.c' in the int_forward_op, the function 'get_kernel_buffer' is used inside, and this function has an undefined error. May I ask where this function is defined or maybe I've done something wrong? I would appreciate if you could provide some help. image

**Error message** arm-none-eabi-g++ -o "TTE_demo_mcunet.elf" @"objects.list" -mcpu=cortex-m7 -T"../STM32F746NGHx_FLASH.ld" --specs=nosys.specs -Wl,-Map="TTE_demo_mcunet.map" -Wl,--gc-sections -static -mfpu=fpv5-sp-d16 -mfloat-abi=hard -mthumb -Wl,--start-group -lc -lm -lstdc++ -lsupc++ -Wl,--end-group c:\st\stm32cubeide_1.5.0\stm32cubeide\plugins\com.st.stm32cube.ide.mcu.externaltools.gnu-tools-for-stm32.7-2018-q2-update.win32_1.5.0.202011040924\tools\arm-none-eabi\bin\ld.exe:../STM32F746NGHx_FLASH.ld:163: warning: memory region DTCMRAM' not declared Src/TinyEngine/src/kernels/int_only/patchpadding_convolve_s8_kernel3_inputch3_stride2.o: In functionpatchpadding_convolve_s8_kernel3_inputch3_stride2': C:/ST/super_light_weight_detector/TinyEngine_vww_tutorial/Debug/../Src/TinyEngine/src/kernels/int_only/patchpadding_convolve_s8_kernel3_inputch3_stride2.c:52: undefined reference to `get_kernel_buffer' collect2.exe: error: ld returned 1 exit status make: *** [makefile:86: TTE_demo_mcunet.elf] Error 1 "make -j8 all" terminated with exit code 2. Build might be incomplete.

meenchen commented 1 year ago

Hi @EricWu-ZL,

Thanks for reaching out! I have merged the fix https://github.com/mit-han-lab/tinyengine/pull/36. Please update the codebase and try again. Feel free to let me know if there is any other question.

EricWu-ZL commented 1 year ago

Hi @EricWu-ZL,

Thanks for reaching out! I have merged the fix #36. Please update the codebase and try again. Feel free to let me know if there is any other question.

It worked! Thank you for the support.

meenchen commented 1 year ago

Close this issue as the patch inference is supported. Feel free to reopen if needed.