search

espressif / esp-dl

Espressif deep-learning library for AIoT applications
MIT License
564 stars 121 forks source link

When will the ESP-DL library support ESP32-P4 (AIV-703) #169

Closed mootterry closed 5 days ago

sun-xiangyu commented 3 months ago

In the new version, we've done significant refactoring and plan to release the source code for optimized operators, automatic computation graph loading, and memory planning.

The beta version, supporting ESP32-P4, will be out before August 30th. The official release, scheduled for September 30th, will support both ESP32-S3 and ESP32-P4, with more documentation.

sun-xiangyu commented 2 months ago

please refer to release/esp32p4. It is not compatible with the master branch, and currently supports only the ESP32-P4. We will be adding more documentation and support ESP32-S3 in the next month.

nicklasb commented 1 month ago

However, the release/esp32p4 does not currently compile, can we fix at least that, we can test it out:

managed_components\espressif__esp-dl\dl\typedef\src\dl_tensor_base.cpp: In function 'int dl::quantize(float, float, float, float)':
managed_components\espressif__esp-dl\dl\typedef\src\dl_tensor_base.cpp:6:18: error: 'dl_esp32p4_round_half_even' was not declared in this scope
    6 |     int output = dl_esp32p4_round_half_even(input * scale);

(also, it is after August 30th. :-) )

sun-xiangyu commented 1 month ago

@nicklasb, Thank you for your feedback. I have updated some code and added CI to ensure that all examples are compilable. Please try again.

sun-xiangyu commented 1 month ago

The new ESP-DL has a different usage from the previous one. The previous ESP-DL has been moved to esp-dl/esp-dl. This folder is a separate component that includes all APIs and operator implementations. The esp-dl/models directory is used to store specific model files and running code separately. The examples directory is for demonstrating how to use ESP-DL and models.

nicklasb commented 1 month ago

@nicklasb, Thank you for your feedback. I have updated some code and added CI to ensure that all examples are compilable. Please try again.

I am afraid I am getting the exact same error:

esp-dl/esp-dl/dl/typedef/src/dl_tensor_base.cpp:6:18: error: 'dl_esp32p4_round_half_even' was not declared in this scope  
    6 |     int output = dl_esp32p4_round_half_even(input * scale);

dl_esp32p4_round_half_even() doesn't seem to be anywhere. I mean literally on all of github and the collective internet..

Please also don't forget to update the documentation, it doesn't map to the structure and contents of the repo.

sun-xiangyu commented 1 month ago

dl_esp32p4_round_half_even: https://github.com/espressif/esp-dl/blob/release/esp32p4/esp-dl/dl/tool/isa/esp32p4/dl_esp32p4_round.S

nicklasb commented 1 month ago

@sun-xiangyu

Ok, now it worked, however the 

    delete detect;
    delete pedestrian;

..causes errors, seems there is some issue with the cleanup:


I (33) boot: ESP-IDF v5.3 2nd stage bootloader
I (33) boot: compile time Sep 14 2024 13:03:22
I (34) boot: Multicore bootloader
I (37) boot: chip revision: v0.1
I (40) qio_mode: Enabling default flash chip QIO
I (45) boot.esp32p4: SPI Speed      : 80MHz
I (50) boot.esp32p4: SPI Mode       : QIO
I (55) boot.esp32p4: SPI Flash Size : 16MB
I (59) boot: Enabling RNG early entropy source...
I (65) boot: Partition Table:
I (68) boot: ## Label            Usage          Type ST Offset   Length
I (76) boot:  0 nvs              WiFi data        01 02 00009000 00006000
I (83) boot:  1 phy_init         RF data          01 01 0000f000 00001000
I (91) boot:  2 factory          factory app      00 00 00010000 007d0000
I (99) boot: End of partition table
I (102) esp_image: segment 0: paddr=00010020 vaddr=480c0020 size=4f05ch (323676) map
I (160) esp_image: segment 1: paddr=0005f084 vaddr=30100000 size=00020h (    32) load
I (161) esp_image: segment 2: paddr=0005f0ac vaddr=30100020 size=0003ch (    60) load
I (166) esp_image: segment 3: paddr=0005f0f0 vaddr=4ff00000 size=00f28h (  3880) load
I (176) esp_image: segment 4: paddr=00060020 vaddr=48000020 size=b3474h (734324) map
I (292) esp_image: segment 5: paddr=0011349c vaddr=4ff00f28 size=104c0h ( 66752) load
I (305) esp_image: segment 6: paddr=00123964 vaddr=4ff11400 size=02d34h ( 11572) load
I (312) boot: Loaded app from partition at offset 0x10000
I (312) boot: Disabling RNG early entropy source...
I (326) hex_psram: vendor id    : 0x0d (AP)
I (326) hex_psram: Latency      : 0x01 (Fixed)
I (326) hex_psram: DriveStr.    : 0x00 (25 Ohm)
I (329) hex_psram: dev id       : 0x03 (generation 4)
I (335) hex_psram: density      : 0x07 (256 Mbit)
I (341) hex_psram: good-die     : 0x06 (Pass)
I (346) hex_psram: SRF          : 0x02 (Slow Refresh)
I (351) hex_psram: BurstType    : 0x00 ( Wrap)
I (356) hex_psram: BurstLen     : 0x03 (2048 Byte)
I (362) hex_psram: BitMode      : 0x01 (X16 Mode)
I (367) hex_psram: Readlatency  : 0x04 (14 cycles@Fixed)
I (373) hex_psram: DriveStrength: 0x00 (1/1)
I (379) MSPI DQS: tuning success, best phase id is 2
I (571) MSPI DQS: tuning success, best delayline id is 11
I esp_psram: Found 32MB PSRAM device
I esp_psram: Speed: 200MHz
I (572) mmu_psram: flash_drom_paddr_start: 0x10000
I (606) mmu_psram: flash_irom_paddr_start: 0x60000
I (675) hex_psram: psram CS IO is dedicated
I (675) cpu_start: Multicore app
I (1584) esp_psram: SPI SRAM memory test OK
W (1593) clk: esp_perip_clk_init() has not been implemented yet
I (1600) cpu_start: Pro cpu start user code
I (1600) cpu_start: cpu freq: 360000000 Hz
I (1601) app_init: Application information:
I (1603) app_init: Project name:     human_face_detect
I (1609) app_init: App version:      v2.0-36-g9a0d2a6
I (1615) app_init: Compile time:     Sep 14 2024 13:02:48
I (1621) app_init: ELF file SHA256:  6f4af5b36...
0x4ff034b0: cpu_utility_ll_reset_cpu at C:/Espressif/frameworks/esp-idf-v5.3/components/hal/esp32p4/include/hal/cpu_utility_ll.h:23
 (inlined by) esp_cpu_reset at C:/Espressif/frameworks/esp-idf-v5.3/components/esp_hw_support/cpu.c:49

0x4ff03620: esp_cpu_wait_for_intr at C:/Espressif/frameworks/esp-idf-v5.3/components/esp_hw_support/cpu.c:57 (discriminator 1)

I (1626) app_init: ESP-IDF:          v5.3
I (1631) efuse_init: Min chip rev:     v0.1
I (1636) efuse_init: Max chip rev:     v0.99
I (1641) efuse_init: Chip rev:         v0.1
I (1646) heap_init: Initializing. RAM available for dynamic allocation:
I (1653) heap_init: At 4FF17180 len 00023E40 (143 KiB): RAM
I (1659) heap_init: At 4FF3AFC0 len 00004BF0 (18 KiB): RAM
I (1666) heap_init: At 4FF40000 len 00060000 (384 KiB): RAM
I (1672) heap_init: At 3010005C len 00001FA4 (7 KiB): TCM
I (1678) esp_psram: Adding pool of 31360K of PSRAM memory to heap allocator
I (1686) spi_flash: detected chip: generic
I (1690) spi_flash: flash io: qio
I (1695) main_task: Started on CPU0
I (1735) esp_psram: Reserving pool of 32K of internal memory for DMA/internal allocations
I (1735) main_task: Calling app_main()
I (1815) dl::Model: model:torch-jit-export, version:1

I (1815) dl::Model: Conv_0: Conv
I (1815) dl::Model: Conv_2: Conv
I (1815) dl::Model: Conv_4: Conv
I (1825) dl::Model: Conv_5: Conv
I (1825) dl::Model: Conv_7: Conv
I (1835) dl::Model: Conv_8: Conv
I (1835) dl::Model: Conv_10: Conv
I (1835) dl::Model: Conv_26: Conv
I (1845) dl::Model: Conv_28: Conv
I (1845) dl::Model: Conv_29: Conv
I (1855) dl::Model: Conv_32: Conv
I (1855) dl::Model: Conv_48: Conv
I (1855) dl::Model: Conv_31: Conv
I (1865) dl::Model: Conv_34: Conv
I (1865) dl::Model: Conv_50: Conv
I (1875) dl::Model: Conv_35: Conv
I (1875) dl::Model: Conv_38: Conv
I (1875) dl::Model: Conv_66: Conv
I (1885) dl::Model: Conv_37: Conv
I (1885) dl::Model: Conv_40: Conv
I (1895) dl::Model: Conv_68: Conv
I (1895) dl::Model: Conv_41: Conv
I (1895) dl::Model: Conv_69: Conv
I (1905) dl::Model: Conv_72: Conv
I (1905) dl::Model: Conv_43: Conv
I (1915) dl::Model: Conv_71: Conv
I (1915) dl::Model: Conv_74: Conv
I (1915) dl::Model: Concat_44: Concat
I (1925) dl::Model: Conv_75: Conv
I (1925) dl::Model: Conv_78: Conv
I (1935) dl::Model: Conv_45: Conv
I (1935) dl::Model: Conv_46: Conv
I (1935) dl::Model: Conv_47: Conv
I (1945) dl::Model: Conv_77: Conv
I (1945) dl::Model: Conv_80: Conv
I (1955) dl::Model: Conv_81: Conv
I (1955) dl::Model: Conv_83: Conv
I (1955) dl::Model: Concat_84: Concat
I (1965) dl::Model: Conv_85: Conv
I (1965) dl::Model: Conv_86: Conv
I (1975) dl::Model: Conv_87: Conv
I (1975) MemoryManagerGreedy: Maximum mermory size: 202400

I (1985) dl::Model: model:torch-jit-export, version:1

I (1985) dl::Model: Conv_0: Conv
I (1995) dl::Model: PRelu_1: PRelu
I (1995) dl::Model: Conv_2: Conv
I (1995) dl::Model: PRelu_3: PRelu
I (2005) dl::Model: Conv_4: Conv
I (2005) dl::Model: PRelu_5: PRelu
I (2015) dl::Model: Conv_6: Conv
I (2015) dl::Model: Conv_7: Conv
I (2015) dl::Model: PRelu_8: PRelu
I (2025) dl::Model: Conv_9: Conv
I (2025) dl::Model: Conv_10: Conv
I (2035) dl::Model: PRelu_11: PRelu
I (2035) dl::Model: Conv_12: Conv
I (2035) dl::Model: PRelu_13: PRelu
I (2045) dl::Model: Conv_14: Conv
I (2045) dl::Model: Conv_15: Conv
I (2055) dl::Model: PRelu_16: PRelu
I (2055) dl::Model: Conv_17: Conv
I (2055) dl::Model: PRelu_18: PRelu
I (2065) dl::Model: Conv_19: Conv
I (2065) dl::Model: Conv_20: Conv
I (2075) dl::Model: Conv_21: Conv
I (2075) dl::Model: Conv_22: Conv
I (2085) MemoryManagerGreedy: Maximum mermory size: 49232

I (2115) human_face_detect: [score: 0.999980, x1: 104, y1: 141, x2: 158, y2: 194]

I (2115) human_face_detect: left_eye: [100, 156], left_mouth: [123, 186], nose: [100, 175], right_eye: [119, 200], right_mouth: [97, 202]]

Guru Meditation Error: Core  0 panic'ed (Store access fault). Exception was unhandled.

Stack dump detected
Core  0 register dump:
MEPC    : 0x4ff0e6f8  RA      : 0x4ff0e9c0  SP      : 0x4ff19510  GP      : 0x4ff11c00
0x4ff0e6f8: remove_free_block at C:/Espressif/frameworks/esp-idf-v5.3/components/heap/tlsf/tlsf.c:333
 (inlined by) block_remove at C:/Espressif/frameworks/esp-idf-v5.3/components/heap/tlsf/tlsf.c:380
 (inlined by) block_merge_next at C:/Espressif/frameworks/esp-idf-v5.3/components/heap/tlsf/tlsf.c:486
 (inlined by) tlsf_free at C:/Espressif/frameworks/esp-idf-v5.3/components/heap/tlsf/tlsf.c:1204
0x4ff0e9c0: tlsf_fls at C:/Espressif/frameworks/esp-idf-v5.3/components/heap/tlsf/tlsf.c:75 (discriminator 1)
 (inlined by) mapping_insert at C:/Espressif/frameworks/esp-idf-v5.3/components/heap/tlsf/tlsf.c:274 (discriminator 1)
 (inlined by) block_remove at C:/Espressif/frameworks/esp-idf-v5.3/components/heap/tlsf/tlsf.c:379 (discriminator 1)
 (inlined by) block_merge_next at C:/Espressif/frameworks/esp-idf-v5.3/components/heap/tlsf/tlsf.c:486 (discriminator 1)
 (inlined by) tlsf_free at C:/Espressif/frameworks/esp-idf-v5.3/components/heap/tlsf/tlsf.c:1204 (discriminator 1)

TP      : 0x4ff19660  T0      : 0x4fc19b4c  T1      : 0x4ff0fc28  T2      : 0x00000200
0x4ff0fc28: block_absorb at C:/Espressif/frameworks/esp-idf-v5.3/components/heap/tlsf/tlsf.c:456
 (inlined by) block_merge_next at C:/Espressif/frameworks/esp-idf-v5.3/components/heap/tlsf/tlsf.c:487
 (inlined by) block_trim_used at C:/Espressif/frameworks/esp-idf-v5.3/components/heap/tlsf/tlsf.c:516
 (inlined by) tlsf_realloc at C:/Espressif/frameworks/esp-idf-v5.3/components/heap/tlsf/tlsf.c:1278

S0/FP   : 0x4819d30c  S1      : 0x48110014  A0      : 0x00000000  A1      : 0x48110084
A2      : 0x00000019  A3      : 0x48110014  A4      : 0x00000001  A5      : 0x555d4157
A6      : 0x00000001  A7      : 0x0000000a  S2      : 0x481813f8  S3      : 0x48181400
S4      : 0x4819d30c  S5      : 0x8555455c  S6      : 0x480c5508  S7      : 0x00000000
S8      : 0x00000000  S9      : 0x00000000  S10     : 0x00000000  S11     : 0x00000000
T3      : 0x00000000  T4      : 0x00000000  T5      : 0x00000000  T6      : 0x481ab018
MSTATUS : 0x00011880  MTVEC   : 0x4ff00003  MCAUSE  : 0x00000007  MTVAL   : 0x555d415f
0x4ff00003: _vector_table at ??:?

MHARTID : 0x00000000

Failed to run gdb_panic_server.py script: Command '['riscv32-esp-elf-gdb', '--batch', '-n', 'C:\\XXXXXX\\Projects\\esp-dl\\examples\\human_face_detect\\build\\human_face_detect.elf', '-ex', 'target remote | "C:\\Espressif\\python_env\\idf5.3_py3.11_env\\Scripts\\python.exe" -m esp_idf_panic_decoder --target esp32p4 "C:\\XXXXX\\AppData\\Local\\Temp\\tmp4ix2mfi5"', '-ex', 'bt']' returned non-zero exit status 1.
b"\r\nwarning: could not convert 'main' from the host encoding (CP1252) to UTF-32.\r\nThis normally should not happen, please file a bug report.\r\nerror starting child process 'C:\\Espressif\\python_env\\idf5.3_py3.11_env\\Scripts\\python.exe -m esp_idf_panic_decoder --target esp32p4 C:\\XXXXX\\AppData\\Local\\Temp\\tmp4ix2mfi5': CreateProcess: No such file or directory\r\nNo stack.\r\n"

Stack memory:
4ff19510: 0x480c5508 0x48110c54 0x4ff228d0 0x00000843 0x480c5508 0x4819d250 0x48110000 0x4ff0e424
4ff19530: 0x480c5508 0x4ff23e5c 0x4ff23e98 0x4ff00f0a 0x4ff23468 0x4ff2342c 0x4ff23e04 0x48011a48
4ff19550: 0x480c5508 0x00000011 0x4ff29fe4 0x480139c0 0x480c5508 0x4ff29958 0x4ff29fe4 0x48013ad2
4ff19570: 0x4ff2975c 0x4ff195c0 0x4ff29958 0x4800b1b8 0x00000000 0x4ff29958 0x4ff228d0 0x4800b216
4ff19590: 0x480c5508 0x00000000 0x4ff195c0 0x4800afda 0x000000ba 0x00000064 0x000000af 0x00000077
4ff195b0: 0x000000c8 0x00000061 0x000000ca 0x00000000 0x4ff25668 0x4ff25668 0x00000001 0x4ff298a0
4ff195d0: 0x4ff298ac 0x4ff298ac 0x4ff195e0 0x00000003 0x000000f0 0x00000140 0x00000003 0x480b1812
4ff195f0: 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x480c4000 0x480b1816
4ff19610: 0x00000000 0x00001388 0x00000003 0x00000000 0x00000000 0x00000000 0x00000000 0x4ff095fc
4ff19630: 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
4ff19650: 0x00000000 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0x00000007 0x4ff18688 0x4ff186a0 0x00000000
4ff19670: 0x4ff18688 0x00000000 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0x00000154 0x4ff18f90
4ff19690: 0x00000000 0x4ff14364 0x4ff14364 0x4ff1968c 0x4ff1435c 0x00000018 0x4ff2beb4 0x4ff2beb4
4ff196b0: 0x4ff1968c 0x00000000 0x00000001 0x4ff186a0 0x6e69616d 0x00000000 0x00000000 0x00000000
4ff196d0: 0x00000000 0x4ff19680 0x00000001 0x00000000 0x00000000 0x00000000 0x00000000 0x4ff17730
4ff196f0: 0x4ff17798 0x4ff17800 0x00000000 0x00000000 0x00000001 0x00000000 0x00000000 0x4ff298a0
4ff19710: 0x4fc0aa82 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
4ff19730: 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
4ff19750: 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
4ff19770: 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
0x00000000 0x00000000 0x00000000
4ff197b0: 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000
4ff197d0: 0x00000000 0x00000000 0x00000000 0x00000000 0x00000600 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5
4ff197f0: 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5
4ff19810: 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5
4ff19830: 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5
4ff19850: 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5
4ff19870: 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5
4ff19890: 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5
4ff198b0: 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5
4ff198d0: 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5
4ff198f0: 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5 0xa5a5a5a5