MichelBerg
commented
5 months ago I got an other version of MobilenetV2 actually converting using the TVM tool. But after flashing my ESP32S3-EYE i get the following error:
ELF file SHA256: f5fcc5d1cf125178
Rebooting...
ESP-ROM:esp32s3-20210327
Build:Mar 27 2021
rst:0x3 (RTC_SW_SYS_RST),boot:0x2a (SPI_FAST_FLASH_BOOT)
Saved PC:0x40375945
0x40375945: esp_restart_noos_dig at /home/bergm/esp_v5.0/esp-idf/components/esp_system/esp_system.c:64 (discriminator 1)
SPIWP:0xee
mode:DIO, clock div:1
load:0x3fce3810,len:0x1860
load:0x403c9700,len:0xd7c
load:0x403cc700,len:0x2e5c
entry 0x403c9934
I (25) boot: ESP-IDF v5.0.4-384-ge42bcf2e88 2nd stage bootloader
I (26) boot: compile time 19:13:40
I (27) boot: Multicore bootloader
I (29) boot: chip revision: v0.1
I (33) qio_mode: Enabling default flash chip QIO
I (38) boot.esp32s3: Boot SPI Speed : 80MHz
I (43) boot.esp32s3: SPI Mode : QIO
I (48) boot.esp32s3: SPI Flash Size : 8MB
I (52) boot: Enabling RNG early entropy source...
I (58) boot: Partition Table:
I (61) boot: ## Label Usage Type ST Offset Length
I (69) boot: 0 factory factory app 00 00 00010000 003c0000
I (76) boot: 1 nvs WiFi data 01 02 003d0000 00004000
I (83) boot: 2 fr Unknown data 01 06 003e0000 00020000
I (91) boot: End of partition table
I (95) esp_image: segment 0: paddr=00010020 vaddr=3c070020 size=d9864h (890980) map
I (239) esp_image: segment 1: paddr=000e988c vaddr=3fc92e00 size=02bc4h ( 11204) load
I (241) esp_image: segment 2: paddr=000ec458 vaddr=40374000 size=03bc0h ( 15296) load
I (247) esp_image: segment 3: paddr=000f0020 vaddr=42000020 size=61b54h (400212) map
I (313) esp_image: segment 4: paddr=00151b7c vaddr=40377bc0 size=0b1f8h ( 45560) load
I (329) boot: Loaded app from partition at offset 0x10000
I (330) boot: Disabling RNG early entropy source...
I (330) cpu_start: Multicore app
I (333) octal_psram: vendor id : 0x0d (AP)
I (338) octal_psram: dev id : 0x02 (generation 3)
I (344) octal_psram: density : 0x03 (64 Mbit)
I (350) octal_psram: good-die : 0x01 (Pass)
I (355) octal_psram: Latency : 0x01 (Fixed)
I (360) octal_psram: VCC : 0x01 (3V)
I (365) octal_psram: SRF : 0x01 (Fast Refresh)
I (371) octal_psram: BurstType : 0x01 (Hybrid Wrap)
I (377) octal_psram: BurstLen : 0x01 (32 Byte)
I (382) octal_psram: Readlatency : 0x02 (10 cycles@Fixed)
I (388) octal_psram: DriveStrength: 0x00 (1/1)
I (394) MSPI Timing: PSRAM timing tuning index: 5
I (399) esp_psram: Found 8MB PSRAM device
I (404) esp_psram: Speed: 80MHz
I (408) cpu_start: Pro cpu up.
I (411) cpu_start: Starting app cpu, entry point is 0x40375424
0x40375424: call_start_cpu1 at /home/bergm/esp_v5.0/esp-idf/components/esp_system/port/cpu_start.c:147
I (0) cpu_start: App cpu up.
I (739) esp_psram: SPI SRAM memory test OK
I (750) cpu_start: Pro cpu start user code
I (750) cpu_start: cpu freq: 240000000 Hz
I (751) cpu_start: Application information:
I (752) cpu_start: Project name: esp32-template-project
I (757) cpu_start: App version: 1444d92-dirty
I (758) cpu_start: Compile time: Jan 30 2024 19:13:34
I (760) cpu_start: ELF file SHA256: f5fcc5d1cf125178...
I (761) cpu_start: ESP-IDF: v5.0.4-384-ge42bcf2e88
I (763) cpu_start: Min chip rev: v0.0
I (764) cpu_start: Max chip rev: v0.99
I (765) cpu_start: Chip rev: v0.1
I (767) heap_init: Initializing. RAM available for dynamic allocation:
I (771) heap_init: At 3FC97B00 len 00051C10 (327 KiB): DRAM
I (774) heap_init: At 3FCE9710 len 00005724 (21 KiB): STACK/DRAM
I (777) heap_init: At 600FE010 len 00001FD8 (7 KiB): RTCRAM
I (781) esp_psram: Adding pool of 8048K of PSRAM memory to heap allocator
I (784) spi_flash: detected chip: generic
I (784) spi_flash: flash io: qio
I (787) app_start: Starting scheduler on CPU0
I (788) main_task: Started on CPU0
I (788) app_start: Starting scheduler on CPU1
I (788) esp_psram: Reserving pool of 32K of internal memory for DMA/internal allocations
I (798) main_task: Calling app_main()
Guru Meditation Error: Core 0 panic'ed (LoadProhibited). Exception was unhandled.
Core 0 register dump:
PC : 0x4201fd3c PS : 0x00060d30 A0 : 0x8200bd42 A1 : 0x3fca5360
0x4201fd3c: tie728_s8_conv2d_per_channel_hwcn_bias_relu_loop at /home/gansichen/Workspace/delete/aigit/master/dl/components/dl/base/isa/tie728/dl_tie728_s8_conv2d.S:2028
A2 : 0x3c150000 A3 : 0x3c162000 A4 : 0x3fca5bb8 A5 : 0x3c132a20
A6 : 0x00000001 A7 : 0x00000002 A8 : 0x00000001 A9 : 0x00000000
A10 : 0x000005c0 A11 : 0x00000600 A12 : 0x00000200 A13 : 0x3c133cc0
A14 : 0x00000000 A15 : 0x0000000b SAR : 0x0000000b EXCCAUSE: 0x0000001c
EXCVADDR: 0x00000000 LBEG : 0x4201fb9c LEND : 0x4201fbdf LCOUNT : 0x00000000
0x4201fb9c: tie728_s8_conv2d_per_channel_hwcn_bias_relu_loop at /home/gansichen/Workspace/delete/aigit/master/dl/components/dl/base/isa/tie728/dl_tie728_s8_conv2d.S:2024
0x4201fbdf: tie728_s8_conv2d_per_channel_hwcn_bias_relu_loop at /home/gansichen/Workspace/delete/aigit/master/dl/components/dl/base/isa/tie728/dl_tie728_s8_conv2d.S:2024
Backtrace: 0x4201fd39:0x3fca5360 0x4200bd3f:0x3fca53e0 0x4200c84b:0x3fca5450 0x4200bb7d:0x3fca5470 0x4200b090:0x3fca5540 0x42008dea:0x3fca5650 0x420086ba:0x3fca5770 0x42007e4d:0x3fca57e0 0x42007dca:0x3fca5800 0x4037a8e5:0x3fca5830
0x4201fd39: tie728_s8_conv2d_per_channel_hwcn_bias_relu_loop at /home/gansichen/Workspace/delete/aigit/master/dl/components/dl/base/isa/tie728/dl_tie728_s8_conv2d.S:2028
0x4200bd3f: void dl::base::conv_operation_shell<signed char, long>(dl::base::ArgsType<signed char>&, void (*)(signed char*, signed char*, void*), void (*)(signed char*, signed char*, void*), void (*)(long*, signed char*, dl::base::ArgsType<signed char> const&), void (*)(long*, signed char*, dl::base::ArgsType<signed char> const&), void (*)(signed char*, long*, dl::base::ArgsType<signed char> const&)) at /home/gansichen/Workspace/delete/aigit/master/dl/components/dl/base/dl_base.hpp:471
0x4200c84b: void dl::base::conv2d<signed char, signed char, signed char, signed char, long>(void*) at /home/gansichen/Workspace/delete/aigit/master/dl/components/dl/base/dl_base_conv2d.cpp:1220
0x4200bb7d: dl::nn::conv2d(dl::Tensor<signed char>&, dl::Tensor<signed char>&, std::vector<int, std::allocator<int> >&, dl::Filter<signed char> const&, int, int, dl::Bias<signed char> const*, dl::Activation<signed char> const*, std::vector<int, std::allocator<int> > const&) at /home/gansichen/Workspace/delete/aigit/master/dl/components/dl/nn/src/dl_nn_conv2d.cpp:72
0x4200b090: esp_dl_nn_conv2d_s8_wrapper at /home/gansichen/Workspace/delete/aigit/master/dl/components/dl/tvm/src/dl_nn_kernel.cpp:39
0x42008dea: tvmgen_default_esp_main_1 at /home/bergm/Desktop/SeedStudioModels/tvm_96x96x3_mobilenetv2_0.35/local_dummy_tvm_inference_project/new_project/components/tvm_model/model/codegen/host/src/default_lib6.c:14
0x420086ba: tvmgen_default___tvm_main__ at /home/bergm/Desktop/SeedStudioModels/tvm_96x96x3_mobilenetv2_0.35/local_dummy_tvm_inference_project/new_project/components/tvm_model/model/codegen/host/src/default_lib1.c:329
0x42007e4d: tvmgen_default_run at /home/bergm/Desktop/SeedStudioModels/tvm_96x96x3_mobilenetv2_0.35/local_dummy_tvm_inference_project/new_project/components/tvm_model/model/codegen/host/src/default_lib0.c:83007
0x42007dca: inference_task at /home/bergm/Desktop/SeedStudioModels/tvm_96x96x3_mobilenetv2_0.35/local_dummy_tvm_inference_project/new_project/main/app_main.c:30
0x4037a8e5: vPortTaskWrapper at /home/bergm/esp_v5.0/esp-idf/components/freertos/FreeRTOS-Kernel/portable/xtensa/port.c:149The model is based on: https://github.com/Seeed-Studio/sscma-model-zoo/blob/main/docs/en/Person_Classification_MobileNetV2_0.35_Rep_96.md where i downloaded the ONNX file and exported it with ESP-DL TVM.
I wonder why this happens even if the TVM export worked.
BlueSkyB
Recently I am working with ESP-DL Quantization Tool and TVM. I am trying to build different models and try to deploy them to the ESP32S3-EYE.
Creating several simple models works in most cases with the quantization tool, but rarely with TVM.
While comparing simple model like this:
My tests are using a static dummy image with the correct shape. Using the Quantization Tool: ~113122 us Using TVM: ~680491 us
Overall the performance of TVM, seams to be very inefficient for me right now. In most of these szenarios with "large" Models, TVM needs a large amount of memory. To fix the overflow in the .dram0.bss segment, I already moved the data to the PSRAM. (using EXT_RAM_BSS_ATTR and the necessary menuconfig ind the esp-idf)
What I want to do is Transfer learning with MobileNetV2. Building and train it with TF and converting it to ONNX isn't a problem. But somehow the conversions given with the ESP-DL Tools (Quant and TVM) is one. I understand that using the quantization tool won't be able to convert MobileNetV2 due to the missing layers, that are not implemented yet (ESP-DL Layers) But using the TVM method leads to errors by using
esp-dl/tools/tvm/export_onnx_model.py.My Transfer learning dummy model looks like this:
Exporting it to ONNX looks like this:
python -m tf2onnx.convert --saved-model tmp_model --output {model_name} --opset {opset}For building a model i use a seperate google colab notebook. The versions I use:
As a test I tried the TVM export by using different opsets (10-15). But the error looks for every version the same.
My colab notebook for the TVM export is based on https://github.com/espressif/esp-dl/issues/139.
The resulting error looks like this:
My goal is to compare these tools from Espressif directly with TF Lite micro or with models that are build with EdgeImpulse. I wonder if somebody tried the same and faced similar issues.
I would be very happy about any kind of advice and support.