search

Spark-Concepts / xPro-V5

xPro-V5 CNC Motion Control System Documentation and User Information
42 stars 19 forks source link

XPro V5 stuck in a boot loop when trying to connect to a Raspberry Pi Zero 2 #187

Closed synman closed 1 year ago

synman commented 1 year ago

I am the developer of an Octoprint plugin that provides a fully functional GRBL emulation / support layer for it (https://github.com/synman/Octoprint-Bettergrblsupport/wiki).

I suspect there is something between the XPro V5 and Rasbian (Octoprint typically runs on a Raspberry Pi) that is triggering the XPro to enter a boot loop upon USB connection to the Pi.

After a bit of controlled rebooting... tends require the XPro to be rebooted / power toggled and then the pi rebooted afterwards, I'm able to (painfully) get a connection reestablished.

One thing I noticed is SPI_FAST_FLASH_BOOT showing up in the boot loop log output (shown below). Is it possible the XPro is getting put into a bootloader mode? If so, is there a jumper or something via software I can configure on it to not allow it to enter the bootloader and proceed with a clean boot?

Any help is greatly appreciated. I can provide whatever further details you require to assist with resolution.

Recv: <Idle|MPos:-5.000,-5.000,-5.000|FS:0,0|WCO:-405.000,-405.000,-55.000>
Recv: ok
Connection closed, closing down monitor
Changing monitoring state from "Operational" to "Offline"
Changing monitoring state from "Offline" to "Opening serial connection"
Connecting to port /dev/ttyUSB0, baudrate 115200
Changing monitoring state from "Opening serial connection" to "Connecting"
Connected to: Serial<id=0x69c4ab30, open=True>(port='/dev/ttyUSB0', baudrate=115200, bytesize=8, parity='N', stopbits=1, timeout=10.0, xonxoff=False, rtscts=False, dsrdtr=False), starting monitor
Send: 

 ?
Send: 

 ?
Recv: )
Recv: configsip: 0, SPIWP:0xee
Recv: clk_dòets Jun  8 2016 00:22:57
Recv: 
Recv: rst:0x1 (POWERON_RESET),boot:0x17 (SPI_FAST_FLASH_BOOT)
Recv: configsip: 0, SPIWP:0xee
Recv: clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
Recv: mode:DIO, clock div:1
Recv: load:0x3fff0018,len:4
Recv: load:0x3fff001c,len:1216
Recv: ho 0 tail 12 room 4
Recv: load:0x40078000,len:9720
Recv: ho 0 tail 12 room 4
Recv: load:0x40080400,len:6352
Recv: entry 0x400806b8
Recv: 
Recv: [MSG:Grbl_ESP32 Ver 1.3a Date 20220621]
Recv: [MSG:Compiled with ESP32 SDK:v3.2.3]
Recv: [MSG:Using machine:CNC_xPRO_V5_XYYZ_NC]
Send: ?
Send: ?
Send: ?
Recv: [MSG:Axis count 3]
Send: ?
Recv: [MSG:RMT Steps]
Send: ?
Recv: [MSG:Door switch on pin GPIO(16)]
Send: ?
Recv: [MSG:Macro Pin 1 GPIO(13)]
Send: ?
Recv: [MSG:Macro Pin 2 GPIO(0)]
Send: ?
Recv: [MSG:Init Motors]
Send: ?
Recv: <Idle|MPos:0.000,0.000,0.000|FS:0,0|Pn:XYZ|WCO:0.000,0.000,0.000>
Recv: Guru Meditation Error: Core  1 panic'ed (LoadProhibited). Exception was unhandled.
Recv: Core 1 register dump:
Recv: PC      : 0x400da9ee  PS      : 0x00060430  A0      : 0x800db033  A1      : 0x3ffd5820
Recv: A2      : 0x00000000  A3      : 0x3ffc6fd4  A4      : 0x00000009  A5      : 0x3ffd5980
Recv: A6      : 0x3ffcaee4  A7      : 0x00000003  A8      : 0x800da9e9  A9      : 0x3ffd5800
Recv: A10     : 0x00000000  A11     : 0x3ffd583c  A12     : 0x000000a0  A13     : 0x00000000
Recv: A14     : 0x00000000  A15     : 0x00000003  SAR     : 0x00000016  EXCCAUSE: 0x0000001c
Recv: EXCVADDR: 0x00000000  LBEG    : 0x400014fd  LEND    : 0x4000150d  LCOUNT  : 0xffffffff
Recv: 
Recv: Backtrace: 0x400da9ee:0x3ffd5820 0x400db030:0x3ffd59a0 0x400db2aa:0x3ffd59c0 0x4008faad:0x3ffd59e0
Recv: 
Recv: Rebooting...
Recv: ets Jun  8 2016 00:22:57
Recv: 
Recv: rst:0xc (SW_CPU_RESET),boot:0x17 (SPI_FAST_FLASH_BOOT)
Recv: configsip: 0, SPIWP:0xee
Recv: clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
Recv: mode:DIO, clock div:1
Recv: load:0x3fff0018,len:4
Recv: load:0x3fff001c,len:1216
Recv: ho 0 tail 12 room 4
Recv: load:0x40078000,len:9720
Recv: ho 0 tail 12 room 4
Recv: load:0x40080400,len:6352
Recv: entry 0x400806b8
Recv: 
Recv: [MSG:Grbl_ESP32 Ver 1.3a Date 20220621]
Send: ?
Recv: [MSG:Compiled with ESP32 SDK:v3.2.3]
Send: ?
Recv: [MSG:Using machine:CNC_xPRO_V5_XYYZ_NC]
Send: ?
Recv: [MSG:Axis count 3]
Send: ?
Recv: [MSG:RMT Steps]
Send: ?
Recv: [MSG:Door switch on pin GPIO(16)]
Send: ?
Recv: [MSG:Macro Pin 1 GPIO(13)]
Send: ?
Recv: [MSG:Macro Pin 2 GPIO(0)]
Send: ?
Recv: [MSG:Init Motors]
Send: ?
Recv: <Idle|MPos:0.000,0.000,0.000|FS:0,0|Pn:XYZ|WCO:0.000,0.000,0.000>
Recv: [MSG:TMCStepper Library Ver. 0x000701]
Recv: [MSG:X  Axis TrinGuru Meditation Error: Core  1 panic'ed (LoadProhibited). Exception was unhandled.
Recv: Core 1 register dump:
Recv: PC      : 0x400da9ee  PS      : 0x00060430  A0      : 0x800db033  A1      : 0x3ffd5820
Recv: A2      : 0x00000000  A3      : 0x3ffc6fd4  A4      : 0x00000009  A5      : 0x3ffd5980
Recv: A6      : 0x3ffcaee4  A7      : 0x00000003  A8      : 0x800da9e9  A9      : 0x3ffd5800
Recv: A10     : 0x00000000  A11     : 0x3ffd583c  A12     : 0x000000a0  A13     : 0x00000000
Recv: A14     : 0x00000000  A15     : 0x00000003  SAR     : 0x00000016  EXCCAUSE: 0x0000001c
Recv: EXCVADDR: 0x00000000  LBEG    : 0x400014fd  LEND    : 0x4000150d  LCOUNT  : 0xffffffff
Recv: 
Recv: Backtrace: 0x400da9ee:0x3ffd5820 0x400db030:0x3ffd59a0 0x400db2aa:0x3ffd59c0 0x4008faad:0x3ffd59e0
Recv: 
Recv: Rebooting...
synman commented 1 year ago

https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-guides/fatal-errors.html#loadprohibited-storeprohibited

Guessing this doesn't have anything to do with an attempted bootloader mode. Maybe RTS/CTS flow control? I did read something about reboots being triggered to enable hardware flow control.

Odd too... EXCVADDR: 0x00000000 is a weird address to be reading. My guess is there's something going sideways above this execution wise.

Spark-Concepts commented 1 year ago

Sunman,

It could possibly be a flow control issue. The xPro uses the CP2102 UART bridge. I'm assuming you've installed the arropriate drivers?

The esp module shouldn't be going into bootloader mode - GPIO 0 is only asserted Low when the recessed switch on the side if the xPro is physically depressed.

I'll have to do some experimentation & report back.

If it turns out to be a flow control issue - we may be able to tweak some setting on the CU2102 uart bridge itself

synman commented 1 year ago

Thanks for the update. It's my understanding the CP2102 drivers are baked into the kernel. All reports indicate it just works.

The pi is enumerating the port just fine as far as I can tell.

Will need some pointers if there is something to tweak on the XPro V5 side of this.

Python on the pi is indicating the port is being opened @ 111520,n,8,1,xon/off=off,cts/rts=off,dtr/dsr=off.

synman commented 1 year ago

I think we can close this, although I learned quite a bit in the process.

The first thing I do when connecting is issue a soft reset (/x18) as a means of handshaking. The XPro V5 then sends its flood of startup messages (I think the key piece here is initializing the TMC stepper driver).

If I immediately start querying the machine (?), it panics and reboots.

What I found is if I sleep for a second after sending that initial handshake before sending anything else, the XPro has enough time to initialize the TMC driver and things go smoothly.

Why do I think it is the TMC driver? Very simple. If I try to connect to XPro with the main 24V power line to the XPro powered off, I do not experience the kernel panic reboot loop -- but the TMC Driver initialization fails.

That was fun.

synman commented 1 year ago

Here's what my comm log looks like now:

Changing monitoring state from "Offline" to "Opening serial connection"
Connecting to port /dev/ttyUSB0, baudrate 115200
Changing monitoring state from "Opening serial connection" to "Connecting"
Connected to: Serial<id=0x7f702a2a30, open=True>(port='/dev/ttyUSB0', baudrate=115200, bytesize=8, parity='N', stopbits=1, timeout=10.0, xonxoff=False, rtscts=False, dsrdtr=False), starting monitor
Send: 

\x18
Recv: ,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wøets Jun  8 2016 00:22:57
Recv: 
Recv: rst:0x1 (POWERON_RESET),boot:0x17 (SPI_FAST_FLASH_BOOT)
Recv: configsip: 0, SPIWP:0xee
Recv: clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
Recv: mode:DIO, clock div:1
Recv: load:0x3fff0018,len:4
Recv: load:0x3fff001c,len:1044
Recv: load:0x40078000,len:10124
Recv: load:0x40080400,len:5856
Recv: entry 0x400806a8
Recv: 
Recv: [MSG:Grbl_ESP32 Ver 1.3a Date 20221127]
Recv: [MSG:Compiled with ESP32 SDK:v4.4.2]
Recv: [MSG:Using machine:CNC_xPRO_V5_XYYZ_NC]
Recv: [MSG:Axis count 3]
Recv: [MSG:RMT Steps]
Recv: [MSG:Door switch on pin GPIO(16)]
Recv: [MSG:Macro Pin 1 GPIO(13)]
Recv: [MSG:Macro Pin 2 GPIO(0)]
Recv: [MSG:Init Motors]
Recv: [MSG:TMCStepper Library Ver. 0x000703]
Recv: [MSG:X  Axis Trinamic TMC5160 Step:GPIO(12) Dir:GPIO(14) CS:GPIO(17) Disable:None Index:1 R:0.050 Limits(-505.000,0.000)]
Recv: [MSG:X  Axis Trinamic driver test passed]
Recv: [MSG:Y  Axis Trinamic TMC5160 Step:GPIO(27) Dir:GPIO(26) CS:GPIO(17) Disable:None Index:2 R:0.050 Limits(-505.000,0.000)]
Recv: [MSG:Y  Axis Trinamic driver test passed]
Recv: [MSG:Y2 Axis Trinamic TMC5160 Step:GPIO(33) Dir:GPIO(32) CS:GPIO(17) Disable:None Index:3 R:0.050 Limits(-505.000,0.000)]
Recv: [MSG:Y2 Axis Trinamic driver test passed]
Recv: [MSG:Z  Axis Trinamic TMC5160 Step:GPIO(15) Dir:GPIO(2) CS:GPIO(17) Disable:None Index:4 R:0.050 Limits(-105.000,0.000)]
Recv: [MSG:Z  Axis Trinamic driver test passed]
Recv: [MSG:Initializing RS485 VFD spindle]
Recv: [MSG:Using Auto-RTS]
Recv: [MSG:VFD RS485  Tx:GPIO(4) Rx:GPIO(25) RTS:None]
Recv: 
Recv: [MSG:Client Started]
Recv: [MSG:Connecting SHELLWLAN]
Recv: [MSG:Connecting.]
Recv: [MSG:VFD: VFD settings read: RPM Range(0, 24000)]]
Recv: [MSG:Connecting..]
Recv: [MSG:Connected with 10.151.51.23]
Recv: [MSG:Start mDNS with hostname:http://CNC_xPRO_V5.local/]
Recv: [MSG:SSDP Started]
Recv: [MSG:HTTP Started]
Recv: [MSG:TELNET Started 23]
Recv: [MSG:Mist coolant on pin GPIO(21)]
Recv: [MSG:X  Axis limit switch on pin GPIO(35)]
Recv: [MSG:Y  Axis limit switch on pin GPIO(34)]
Recv: [MSG:Z  Axis limit switch on pin GPIO(39)]
Recv: [MSG:Probe on pin GPIO(22)]
Recv: 
Recv: Grbl 1.3a ['$' for help]
Changing monitoring state from "Connecting" to "Operational"
Recv: [MSG:'$H'|'$X' to unlock]
Send: $+
Recv: $0=4
Recv: $1=255
Recv: $2=0
Recv: $3=3
Recv: $4=0
Recv: $5=1
Recv: $6=1
Recv: $10=1
Recv: $11=0.010
Recv: $12=0.002
Recv: $13=0
Recv: $20=1
Recv: $21=0
Recv: $22=1
Recv: $23=0
Recv: $24=200.000
Recv: $25=2000.000
Recv: $26=1000.000
Recv: $27=5.000
Recv: $30=24000.000
Recv: $31=0.000
Recv: $32=0
Recv: $N1=
Recv: $N0=
Recv: $33=5000.000
Recv: $34=0.000
Recv: $35=0.000
Recv: $36=100.000
Recv: $100=200.000
Recv: $101=200.000
Recv: $102=200.000
Recv: $103=200.000
Recv: $104=100.000
Recv: $105=100.000
Recv: $110=3000.000
Recv: $111=3000.000
Recv: $112=450.000
Recv: $113=1000.000
Recv: $114=1000.000
Recv: $115=1000.000
Recv: $120=300.000
Recv: $121=300.000
Recv: $122=300.000
Recv: $123=200.000
Recv: $124=200.000
Recv: $125=200.000
Recv: $130=505.000
Recv: $131=505.000
Recv: $132=105.000
Recv: $133=300.000
Recv: $134=300.000
Recv: $135=300.000
Recv: $140=2.121
Recv: $141=2.121
Recv: $142=2.121
Recv: $143=1.800
Recv: $144=0.800
Recv: $145=0.800
Recv: $150=1.000
Recv: $151=1.000
Recv: $152=1.000
Recv: $153=0.250
Recv: $154=0.400
Recv: $155=0.400
Recv: $160=8
Recv: $161=8
Recv: $162=8
Recv: $163=16
Recv: $164=16
Recv: $165=16
Recv: $170=16
Recv: $171=16
Recv: $172=16
Recv: $173=16
Recv: $174=16
Recv: $175=16
Recv: ok
Send: ?