matteocarnelos / microflow-rs

A robust and efficient TinyML inference engine.
Apache License 2.0
70 stars 4 forks source link

ESP32 stalls when i run person detect example #2

Open salsasteve opened 1 month ago

salsasteve commented 1 month ago
salsasteve in 🌐 thinkcentre0 in microflow-rs/examples/esp32 on  main [!] is 📦 v0.0.0 via 🦀 v1.79.0-nightly took 8s
❯ cargo run --example person_detect --release
    Finished `release` profile [optimized] target(s) in 0.09s
     Running `espflash flash --monitor target/xtensa-esp32-none-elf/release/examples/person_detect`
[2024-07-30T23:00:17Z INFO ] Serial port: '/dev/ttyUSB0'
[2024-07-30T23:00:17Z INFO ] Connecting...
[2024-07-30T23:00:17Z INFO ] Using flash stub
Chip type:         esp32 (revision v1.0)
Crystal frequency: 40 MHz
Flash size:        4MB
Features:          WiFi, BT, Dual Core, 240MHz, Coding Scheme None
MAC address:       fc:f5:c4:28:e9:54
App/part. size:    450,032/4,128,768 bytes, 10.90%
[00:00:01] [========================================]      17/17      0x1000
Commands:
    CTRL+R    Reset chip
    CTRL+C    Exit

ets Jun  8 2016 00:22:57

rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0030,len:7104
load:0x40078000,len:15576
load:0x40080400,len:4
ho 8 tail 4 room 4
load:0x40080404,len:3876
entry 0x4008064c
0x4008064c - handle_interrupts
    at ??:??
I (30) boot: ESP-IDF v5.1-beta1-378-gea5e0ff298-dirt 2nd stage bootloader
I (30) boot: compile time Jun  7 2023 07:48:23
I (33) boot: Multicore bootloader
I (37) boot: chip revision: v1.0
I (40) boot.esp32: SPI Speed      : 40MHz
I (45) boot.esp32: SPI Mode       : DIO
I (50) boot.esp32: SPI Flash Size : 4MB
I (54) boot: Enabling RNG early entropy source...
I (60) boot: Partition Table:
I (63) boot: ## Label            Usage          Type ST Offset   Length
I (70) boot:  0 nvs              WiFi data        01 02 00009000 00006000
I (78) boot:  1 phy_init         RF data          01 01 0000f000 00001000
I (85) boot:  2 factory          factory app      00 00 00010000 003f0000
I (93) boot: End of partition table
I (97) esp_image: segment 0: paddr=00010020 vaddr=3f400020 size=3d640h (251456) map
I (196) esp_image: segment 1: paddr=0004d668 vaddr=3ffb0000 size=00004h (     4) load
I (197) esp_image: segment 2: paddr=0004d674 vaddr=40080000 size=00fach (  4012) load
I (204) esp_image: segment 3: paddr=0004e628 vaddr=00000000 size=019f0h (  6640)
I (213) esp_image: segment 4: paddr=00050020 vaddr=400d0020 size=2dda8h (187816) map
I (287) boot: Loaded app from partition at offset 0x10000
I (287) boot: Disabling RNG early entropy source...

this is where it stalls

speech on the other hand works fine

salsasteve in 🌐 thinkcentre0 in microflow-rs/examples/esp32 on  main [!] is 📦 v0.0.0 via 🦀 v1.79.0-nightly took 34m13s
❯ cargo run --example speech --release
    Finished `release` profile [optimized] target(s) in 0.13s
     Running `espflash flash --monitor target/xtensa-esp32-none-elf/release/examples/speech`
[2024-07-30T23:35:02Z INFO ] Serial port: '/dev/ttyUSB0'
[2024-07-30T23:35:02Z INFO ] Connecting...
[2024-07-30T23:35:03Z INFO ] Using flash stub
Chip type:         esp32 (revision v1.0)
Crystal frequency: 40 MHz
Flash size:        4MB
Features:          WiFi, BT, Dual Core, 240MHz, Coding Scheme None
MAC address:       fc:f5:c4:28:e9:54
App/part. size:    112,096/4,128,768 bytes, 2.71%
[2024-07-30T23:35:04Z INFO ] Segment at address '0x1000' has not changed, skipping write
[2024-07-30T23:35:04Z INFO ] Segment at address '0x8000' has not changed, skipping write
[00:00:05] [========================================]      57/57      0x10000                                                                              [2024-07-30T23:35:10Z INFO ] Flashing has completed!
Commands:
    CTRL+R    Reset chip
    CTRL+C    Exit

ets Jun  8 2016 00:22:57

rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0030,len:7104
load:0x40078000,len:15576
load:0x40080400,len:4
ho 8 tail 4 room 4
load:0x40080404,len:3876
entry 0x4008064c
0x4008064c - handle_interrupts
    at ??:??
I (30) boot: ESP-IDF v5.1-beta1-378-gea5e0ff298-dirt 2nd stage bootloader
I (30) boot: compile time Jun  7 2023 07:48:23
I (32) boot: Multicore bootloader
I (36) boot: chip revision: v1.0
I (40) boot.esp32: SPI Speed      : 40MHz
I (45) boot.esp32: SPI Mode       : DIO
I (49) boot.esp32: SPI Flash Size : 4MB
I (54) boot: Enabling RNG early entropy source...
I (59) boot: Partition Table:
I (63) boot: ## Label            Usage          Type ST Offset   Length
I (70) boot:  0 nvs              WiFi data        01 02 00009000 00006000
I (78) boot:  1 phy_init         RF data          01 01 0000f000 00001000
I (85) boot:  2 factory          factory app      00 00 00010000 003f0000
I (93) boot: End of partition table
I (97) esp_image: segment 0: paddr=00010020 vaddr=3f400020 size=07ca4h ( 31908) map
I (117) esp_image: segment 1: paddr=00017ccc vaddr=3ffb0000 size=00004h (     4) load
I (117) esp_image: segment 2: paddr=00017cd8 vaddr=40080000 size=00fach (  4012) load
I (126) esp_image: segment 3: paddr=00018c8c vaddr=00000000 size=0738ch ( 29580)
I (143) esp_image: segment 4: paddr=00020020 vaddr=400d0020 size=0b594h ( 46484) map
I (161) boot: Loaded app from partition at offset 0x10000
I (161) boot: Disabling RNG early entropy source...

Input sample: 'yes.wav'
Prediction: 0.0% silence, 0.0% unknown, 99.6% yes, 0.0% no
Outcome: YES
Execution time: 177960 us

Input sample: 'no.wav'
Prediction: 0.0% silence, 5.5% unknown, 0.0% yes, 94.5% no
Outcome: NO
Execution time: 175907 us

I get a similar result when I run sine.

I'm looking into the code but i don't see too much of a differences between speech.rs and person_detect.rs examples

matteocarnelos commented 1 month ago

Hey @salsasteve, thanks for reporting the issue!

It looks like the ESP32 runs out of memory. However, I couldn't reproduce the issue on my ESP32-WROOM-32 module. Which ESP32 module are you using?

salsasteve commented 1 month ago

I'm using a esp32-wroom-32 module(esp32 devkit v1) as well. Wondering why I would be running out of memory. Maybe its the way I'm programming it??

its definitely crashing here let person_predicted = PersonDetect::predict_quantized(features::PERSON);

I'm also getting

salsasteve in 🌐 thinkcentre0 in microflow-rs/examples/esp32 on  HEAD (600ebca) [!] is 📦 v0.0.0 via 🦀 v1.79.0-nightly
❯ cargo r --example person_detect --release
   Compiling examples-esp32 v0.0.0 (/home/salsasteve/Documents/solo-gits/microflow-rs/examples/esp32)
error[E0061]: this function takes 1 argument but 2 arguments were supplied
   --> examples/person_detect.rs:43:19
    |
43  |     let mut rtc = Rtc::new(peripherals.LPWR, None);
    |                   ^^^^^^^^                 ------
    |                                            | |
    |                                            | unexpected argument of type `Option<_>`
    |                                            help: remove the extra argument
    |
note: associated function defined here
   --> /home/salsasteve/.cargo/git/checkouts/esp-hal-42ec44e8c6943228/3049f28/esp-hal/src/rtc_cntl/mod.rs:196:12
    |
196 |     pub fn new(rtc_cntl: impl Peripheral<P = crate::peripherals::LPWR> + 'd) -> Self {
    |            ^^^

For more information about this error, try `rustc --explain E0061`.
error: could not compile `examples-esp32` (example "person_detect") due to 1 previous error

when I use the v0.1.3 release directly. I'm thinking there is a lib version problem

matteocarnelos commented 1 month ago

Ok it's probably something related to esp-hal. They introduce breaking changes quite often and the examples need to be updated accordingly.

I'll have a look at this as soon as I'll be back from vacation.

salsasteve commented 1 month ago

put in a simple pull request to fix the 2nd issue I mentioned. unexpected argument of type Option<_>

First issue still remains a mystery

matteocarnelos commented 2 weeks ago

I updated esp-hal to the latest 0.19.0 version and I could reproduce the issue on the person detect example. However, I couldn't find any cause yet. This issue needs further investigation.