search

espressif / esp-idf

Espressif IoT Development Framework. Official development framework for Espressif SoCs.
Apache License 2.0
13.34k stars 7.2k forks source link

Latest release/v5.1 hints crashes on mac m1, maybe others (IDFGH-10075) #11351

Closed someburner closed 1 year ago

someburner commented 1 year ago

Answers checklist.

IDF version.

v5.1 head

Operating System used.

macOS

How did you build your project?

Command line with idf.py

If you are using Windows, please specify command line type.

None

Development Kit.

custom

Power Supply used.

External 5V

What is the expected behavior?

idf.py monitor to work without crashing

What is the actual behavior?

Crashes immediately unless no hints specified.

Steps to reproduce.

idf.py monitor

Debug Logs.

this is all I get if hints are enabled.

--- idf_monitor on /dev/cu.usbserial-11301 500000 ---
--- Quit: Ctrl+] | Menu: Ctrl+T | Help: Ctrl+T followed by Ctrl+H ---
xx<xxx<xxx<x<x<0x40081158: call_start_cpu1 at /COMPONENT_ESP_SYSTEM_DIR/port/cpu_start.c:154

More Information.

Manually connecting a serial monitor works fine. It's just idf.py monitor that stops outputting after a pointer is seen. Was previously on an earlier release/v5.1 sha: bb9200acec7dd60e9adb4a381e5400dcd5024534

Using --no-reset also works fine. Seems like it might be from 83a682c but I have not tried reverting that yet.

igrr commented 1 year ago

I can say that this has happened to me once as well. I recall that I could get it back working by doing a reset command in that terminal and running idf.py monitor again.

someburner commented 1 year ago

Just tried on linux machine too. Reset did not help. I'm not ruling out this being somewhat specific to my setup, but when earlier rev works perfectly fine, it feels like a regression.

someburner commented 1 year ago

Two different terminals up doing the same exact thing on same chip, latest v5.1 exhibits this, earlier v5.1 does not. Also tried on ESP Prog, same behavior. Although in the case of ESP Prog it just stops with:

0x40081178: call_start_cpu1 at /COMPONENT_ESP_SYSTEM_DIR/port/cpu_start.c:166

Warning: checksum mismatch between flashed and built applications. Checksum of built application is 32f7303a2c3b9028fcb021d12aba38e94d93f33e5022c6c5ffdb05c133469be8

Other window shows what I'd expect

I (516) cpu_start: Pro cpu up.
I (517) cpu_start: Starting app cpu, entry point is 0x40081178
0x40081178: call_start_cpu1 at /COMPONENT_ESP_SYSTEM_DIR/port/cpu_start.c:166

I (0) cpu_start: App cpu up.
I (525) cpu_start: Pro cpu start user code
I (525) cpu_start: cpu freq: 240000000 Hz
I (525) cpu_start: Application information:
I (525) cpu_start: Project name:     app_main
I (527) cpu_start: App version:      7b424e0
I (528) cpu_start: Compile time:      
I (529) cpu_start: ELF file SHA256:  62f29d2d0774f870...
Warning: checksum mismatch between flashed and built applications. Checksum of built application is 32f7303a2c3b9028fcb021d12aba38e94d93f33e5022c6c5ffdb05c133469be8
peterdragun commented 1 year ago

Hi @someburner, thanks for reporting. I am trying to reproduce the issue but with no luck. Can you please provide some additional info, like what chip are you using? Are you getting the same error also on some applications from the examples folder? If not, can you please provide some example code where I can reproduce the issue? Thanks in advance

someburner commented 1 year ago

ESP32-Wroom-32e 16MB chip. Flash encryption enabled in release mode, but ROM uart still enabled for flashing encrypted binaries. Did you try with uart at 500000?

I can try commenting out most of my application and see if it still happens.

someburner commented 1 year ago

Tried different bauds and also commenting out pretty much all application code, so app main is just a basic while loop that prints a test message. The failure mode is the same every time, appears something gets corrupted in the log output of the bootloader. Although in this case the bootloader has not changed, and also flashing a newly built bootloader has the same behavior.

If nobody can reproduce it must be something different in my situation. I do flash the chip using baud == 921600 but that seems like it shouldnt be relevant as the issue persists without a reflash.

someburner commented 1 year ago

sdkconfig.txt

someburner commented 1 year ago

Still not ruling out it being an issue with my build somehow, but reverting: https://github.com/espressif/esp-idf/commit/49718b20a5745c090d625dbfe3a177337eb38adf

completely fixes the issue.

someburner commented 1 year ago

I think I misunderstood what hints are.

@igrr

passing --no-hints and things work fine. I cleared the hints list and it still has problems. I don't understand why hints should be enabled for monitor? It appears hints.yml is entirely related to compiler output. Is this for idf.py flash monitor? In my case I execute scripts separately, since I need to pre-encrypt binaries, and then run monitor after.

JustinOng commented 1 year ago

I have a similar issue on Ubuntu 20.04, using an ESP-PROG targeting a ESP32-WROOM-32E on a custom board. Development is done in the release-v5.1 Docker container through VSCs Dev Containers.

When running idf.py flash monitor, after flashing and idf_monitor launches, the output sometimes stops at the same point:

...
Wrote 173584 bytes (96565 compressed) at 0x00010000 in 2.6 seconds (effective 542.5 kbit/s)...
Hash of data verified.
Compressed 3072 bytes to 103...
Writing at 0x00008000... (100 %)
Wrote 3072 bytes (103 compressed) at 0x00008000 in 0.1 seconds (effective 383.6 kbit/s)...
Hash of data verified.

Leaving...
Hard resetting via RTS pin...
Executing action: monitor
Running idf_monitor in directory /workspaces/esp-idf-11351-repro
Executing "/opt/esp/python_env/idf5.1_py3.8_env/bin/python /opt/esp/idf/tools/idf_monitor.py -p /dev/ttyUSB1 -b 115200 --toolchain-prefix xtensa-esp32-elf- --target esp32 --revision 0 /workspaces/esp-idf-11351-repro/build/cyphal-distance-sensor.elf -m '/opt/esp/python_env/idf5.1_py3.8_env/bin/python' '/opt/esp/idf/tools/idf.py'"...
--- idf_monitor on /dev/ttyUSB1 115200 ---
--- Quit: Ctrl+] | Menu: Ctrl+T | Help: Ctrl+T followed by Ctrl+H ---
x0x40080400: _init at ??:?

0x400810d4: call_start_cpu1 at /opt/esp/idf/components/esp_system/port/cpu_start.c:154
# Reset button on ESP-PROG physically pressed a few times here
0x400810d4: call_start_cpu1 at /opt/esp/idf/components/esp_system/port/cpu_start.c:154

0x40080400: _init at ??:?

0x400810d4: call_start_cpu1 at /opt/esp/idf/components/esp_system/port/cpu_start.c:154

0x40080400: _init at ??:?

0x400810d4: call_start_cpu1 at /opt/esp/idf/components/esp_system/port/cpu_start.c:154

Full log

The issue seems very intermittent - it takes quite a few tries to cause the error. I've attached a zip file of the project, though its worth noting I coudn't reproduce the issue with the getting-started/hello-world example.

esp-idf-11351-repro.zip