search

atanisoft / ESP32CommandStation

An ESP32 based DCC Command Station with integrated OpenLCB (LCC) --- NOTE: this project is not under active development.
https://atanisoft.github.io/ESP32CommandStation/
GNU General Public License v3.0
90 stars 34 forks source link

Intermittent crash in LwIP #64

Closed atanisoft closed 2 years ago

atanisoft commented 3 years ago 
Guru Meditation Error: Core  1 panic'ed (LoadProhibited). Exception was unhandled.

Core  1 register dump:
PC      : 0x4015395c  PS      : 0x00060c30  A0      : 0x801547ba  A1      : 0x3ffd95c0  
A2      : 0x3ffe9b00  A3      : 0x00000000  A4      : 0x0000013f  A5      : 0x00000002  
A6      : 0x00000000  A7      : 0x00000001  A8      : 0x3ffbf308  A9      : 0x3ffd9570  
A10     : 0x80142dff  A11     : 0x00000050  A12     : 0x3ffee314  A13     : 0x00000000  
A14     : 0x00000494  A15     : 0x000005a0  SAR     : 0x00000019  EXCCAUSE: 0x0000001c  
EXCVADDR: 0x80142e03  LBEG    : 0x4000c2e0  LEND    : 0x4000c2f6  LCOUNT  : 0x00000000  

Backtrace:0x40153959:0x3ffd95c0 0x401547b7:0x3ffd95e0 0x4014390d:0x3ffd9600
0x40153959: lwip_netconn_do_writemore at esp-idf-master/components/lwip/lwip/src/api/api_msg.c:1788
0x401547b7: lwip_netconn_do_write at esp-idf-master/components/lwip/lwip/src/api/api_msg.c:1910
0x4014390d: tcpip_thread_handle_msg at esp-idf-master/components/lwip/lwip/src/api/tcpip.c:208
 (inlined by) tcpip_thread at esp-idf-master/components/lwip/lwip/src/api/tcpip.c:154

ELF file SHA256: 88f540fa6f16d625
TrainzLuvr commented 3 years ago

Another one, from the ESP32olcbhub:

Incoming connection from 192.168.4.2, fd 51.
[TWAI] RX:0 (pending:0,overrun:0,discard:0) TX:34 (pending:0,suc:34,fail:0) bus (arb-err:0,err:0,state:Running)
GCHubPort: Shutting down gridconnect port -1. (0x3ffdb338)
Incoming connection from 192.168.4.2, fd 51.
Guru Meditation Error: Core  1 panic'ed (LoadProhibited). Exception was unhandled.

Core  1 register dump:
PC      : 0x40127361  PS      : 0x00060830  A0      : 0x80117290  A1      : 0x3ffc97b0  
A2      : 0x3ffbd848  A3      : 0x31753263  A4      : 0x0000ffff  A5      : 0x0000ffff  
A6      : 0x3ffdb0f0  A7      : 0x3ffdae30  A8      : 0x00080000  A9      : 0x3ffc9790  
A10     : 0x3ffdb0f0  A11     : 0x0000059b  A12     : 0x3ffc89d8  A13     : 0x00000000  
A14     : 0x3ffc8a14  A15     : 0x00000000  SAR     : 0x00000010  EXCCAUSE: 0x0000001c  
EXCVADDR: 0x00080008  LBEG    : 0x4000c2e0  LEND    : 0x4000c2f6  LCOUNT  : 0xffffffff  

Backtrace:0x4012735e:0x3ffc97b0 0x4011728d:0x3ffc97d0

0:  0x4011728d: tcpip_thread_handle_msg at C:/espressif/esp-idf/components/lwip/lwip/src/api/tcpip.c:208
  \-> inlined by: tcpip_thread at C:/espressif/esp-idf/components/lwip/lwip/src/api/tcpip.c:154
atanisoft commented 3 years ago 
Guru Meditation Error: Core  1 panic'ed (LoadProhibited). Exception was unhandled.

Core  1 register dump:
PC      : 0x4014927e  PS      : 0x00060a30  A0      : 0x8014a7b5  A1      : 0x3ffc6640  
A2      : 0x00000001  A3      : 0x00000004  A4      : 0x00000032  A5      : 0x3ffb5aac  
A6      : 0x00000001  A7      : 0x3ffb5adc  A8      : 0x80137b19  A9      : 0x3ffc6600  
A10     : 0x00000001  A11     : 0x00007210  A12     : 0x00060a20  A13     : 0x00000000  
A14     : 0x3ffc3350  A15     : 0x00000000  SAR     : 0x00000010  EXCCAUSE: 0x0000001c  
EXCVADDR: 0x00000001  LBEG    : 0x4000c2e0  LEND    : 0x4000c2f6  LCOUNT  : 0xffffffff  

Backtrace:0x4014927b:0x3ffc6640 0x4014a7b2:0x3ffc6660 0x4014b0aa:0x3ffc6690 0x401399b1:0x3ffc66b0

0x4014927b: sys_sem_signal at esp-idf/components/lwip/port/esp32/freertos/sys_arch.c:150
0x4014a7b2: lwip_netconn_do_close_internal at esp-idf/components/lwip/lwip/src/api/api_msg.c:1119 (discriminator 3)
0x4014b0aa: lwip_netconn_do_delconn at esp-idf/components/lwip/lwip/src/api/api_msg.c:1230 (discriminator 2)
0x401399b1: tcpip_thread_handle_msg at esp-idf/components/lwip/lwip/src/api/tcpip.c:208
 (inlined by) tcpip_thread at esp-idf/components/lwip/lwip/src/api/tcpip.c:154
atanisoft commented 3 years ago 
assertion "tcpip_thread: invalid message" failed: file "esp-idf/components/lwip/lwip/src/api/tcpip.c", line 213, function: tcpip_thread_handle_msg

abort() was called at PC 0x40193fe3 on core 1
0x40193fe3: __assert_func at /builds/idf/crosstool-NG/.build/xtensa-esp32-elf/src/newlib/newlib/libc/stdlib/assert.c:62 (discriminator 8)

Backtrace:0x400871e2:0x3ffc7880 0x4008797d:0x3ffc78a0 0x4008b9ed:0x3ffc78c0 0x40193fe3:0x3ffc7940 0x401373e0:0x3ffc7970
0x400871e2: panic_abort at esp-idf/components/esp_system/panic.c:341
0x4008797d: esp_system_abort at esp-idf/components/esp_system/system_api.c:106
0x4008b9ed: abort at esp-idf/components/newlib/abort.c:46
0x40193fe3: __assert_func at /builds/idf/crosstool-NG/.build/xtensa-esp32-elf/src/newlib/newlib/libc/stdlib/assert.c:62 (discriminator 8)
0x401373e0: tcpip_thread_handle_msg at esp-idf/components/lwip/lwip/src/api/tcpip.c:213
 (inlined by) tcpip_thread at esp-idf/components/lwip/lwip/src/api/tcpip.c:154
atanisoft commented 3 years ago 
Guru Meditation Error: Core  1 panic'ed (LoadProhibited). Exception was unhandled.

Core  1 register dump:
PC      : 0x40146ee2  PS      : 0x00060c30  A0      : 0x8014914e  A1      : 0x3ffc7950  
A2      : 0x00000008  A3      : 0x000016cf  A4      : 0x3ffc6cb8  A5      : 0x3ffc6cc0  
A6      : 0x00000001  A7      : 0x3ffc6c94  A8      : 0x8013ab6d  A9      : 0x3ffc7930  
A10     : 0x00000000  A11     : 0x3ffc799c  A12     : 0x3f421308  A13     : 0x00000000  
A14     : 0xfffffff3  A15     : 0x00000020  SAR     : 0x00000010  EXCCAUSE: 0x0000001c  
EXCVADDR: 0x00000008  LBEG    : 0x4000c2e0  LEND    : 0x4000c2f6  LCOUNT  : 0xffffffff  

Backtrace:0x40146edf:0x3ffc7950 0x4014914b:0x3ffc7970 0x40137615:0x3ffc7990
0x40146edf: sys_sem_signal at esp-idf/components/lwip/port/esp32/freertos/sys_arch.c:150
0x4014914b: lwip_netconn_do_recv at esp-idf/components/lwip/lwip/src/api/api_msg.c:1675
0x40137615: tcpip_thread_handle_msg at esp-idf/components/lwip/lwip/src/api/tcpip.c:208
 (inlined by) tcpip_thread at esp-idf/components/lwip/lwip/src/api/tcpip.c:154
TrainzLuvr commented 3 years ago

Looks like you are hitting all the "right" keys. 🤣

atanisoft commented 3 years ago

I'll been dumping the "interesting" backtraces here that I'm finding from pulling the UWT-100 CDI via Esp32OlcbHub.

As part of this, I've been digging through OSSelectWakeup and I believe that there may be a bug in the VFS usage in that code. I've got a few local updates which I'm working through testing locally with the first use-case of load the CDI reliably and once that is reliable I'll move on to hooking up a CS on the bus and have the UWT-100 download the FDI from the CS.

atanisoft commented 2 years ago

With https://github.com/espressif/esp-idf/commit/bf8ceae78039280d1eeb9a5a3d1ac7dbaf02ff5c included on the IDF v4.4 branch this issue no longer reproduces!

atanisoft commented 2 years ago

IDF v4.4 branch now contains the VFS bug fix!