[BUG] PID AutoTune + Meatpack = Reboot

[skl111]

Did you test the latest bugfix-2.0.x code?

Yes, and the problem still exists.

Bug Description

[BUG] Running PID Autotune with OctoPrint Meatpack Plugin and defined in Marlin causes a reboot.

Bug Timeline

09-19-2021

Expected behavior

Expected Behavior is for PID Autotune for hotend or bed when used with #define MEATPACK_ON_SERIAL_PORT_2 to not cause a crash.

Actual behavior

When running a PID autotune using OctoPrint with the Meatpack Plugin, it causes the printer board to reboot.

Steps to Reproduce

Using Octoprint with the Meatpack Plugin installed and Meatpack enabled in the firmware, run PID Autotune. Printer board will reboot a in a minute or 2, Happens with Hotend and Bed PID Autotuning .

Version of Marlin Firmware

bugfix-2.0.x as of 09-18-2021

Printer model

Ender 5 Pro

Electronics

BigTreeTech SKR Mini E3 2.0

Add-ons

No response

Bed Leveling

No response

Your Slicer

Cura

Host Software

OctoPrint

Additional information & file uploads

09_18_2021_Meatpack_Crash.zip

A similar issue was raised on the Meatpack Plugin Tracker and closed, Issue was brought up when this crash occurred and was reproduceable on 2 different machines, bug was opened here after not seeing anything similar.

https://github.com/scottmudge/OctoPrint-MeatPack/issues/20#issuecomment-923878680

[ellensp]

I wonder is this is actually the watchdog timer being tripped. As a test comment out #define USE_WATCHDOG See if it now completes PID Autotune

[skl111]

I wonder is this is actually the watchdog timer being tripped. As a test comment out #define USE_WATCHDOG See if it now completes PID Autotune

I did and it got stuck until I turned it off.

2021-09-27 05:04:57,792 - Recv: T:108.82 /0.00 B:22.77 /0.00 @:127 B@:0 2021-09-27 05:04:59,792 - Recv: T:113.16 /0.00 B:22.89 /0.00 @:127 B@:0 2021-09-27 05:05:20,749 - Unexpected error while reading serial port, please consult octoprint

[GhostlyCrowd]

I had filed that bug on the meatpack repo.

It turned out to be related to the fact I had several gigabytes of gcode files on the PI's SD card, the issue went away after I cleaned house. I never figured out if it was the size/count of files or an offending single one. but the problem was resolved by cleaning the sd and has not returned since. although I do keep less on the card now.

I went so far to replace the pi and the sd card and still had the same issue until the above mentioned fix. I didn't bother filing a bug report on marlin because as far as i know marlin cannot see the pi's sd card so It was assumed it was not related.

[skl111]

I had filed that bug on the meatpack repo.

It turned out to be related to the fact I had several gigabytes of STL files on the SD card, the issue went away after I cleaned house. I never figured out if it was the size/count of stls or an offending single stl. but the problem was resolved by cleaning the sd and has not returned since. although I do keep less on the card now.

It happens without and SD card inserted as well

[GhostlyCrowd]

you are using a USB drive to boot your pi then? or are you running octoprint on a x86 machine or something?

I'm talking about the PI's sd card not one presented to marlin.

[skl111]

Ah I was referenced the sd card in the sd slot of the printer board.

Octopi 32gb sd card Pi4 4gb.

[GhostlyCrowd]

if you backup your octopi instance with the gcode files, then delete all of them from your octopi instance, run a pid tune with meatpack on and it works then you and I both have/had the exact same issues and I have no idea why. I also am unsure who's bug this is to find but I was under the understanding that the only interaction that marlin has with the pi is via serial so why this is effecting marlin only when meatpack is enabled and you have a bunch of gcode stored on your pi.

I asked around, the Meatpack plugin dev couldn't figure how his plugin could cause this and the octoprint guys who i talked to said it was not a octoprint core bug.... Its a head scratcher.

[skl111]

I found a different way to reproduce both it happening and not happening with the meatpack option enabled in marlin, which leads me to think it might be a USB to Serial communications issue where something gets sent to marlin and it crashes as it sends something back across the wire, but also I have noticed that if you do it 1-3 times in a row the PI itself crashes.

I ran PID Autotune via the LCD with the Pi connected and the issue happened. I ran PID Autotune via the LCD with the Pi disconnected and the issue did not happen.

[skl111]

To add to this, I stopped the octoprint service and ran PID autotune from the LCD and it did not exhibit the issue, which means this is a communication issue due to octoprint with meatpack & marlin with meatpack having a communication breakdown and then 1 of if not both crashing.

[GhostlyCrowd]

To add to this, I stopped the octoprint service and ran PID autotune from the LCD and it did not exhibit the issue, which means this is a communication issue due to octoprint with meatpack & marlin with meatpack having a communication breakdown and then 1 of if not both crashing.

curious, did you see the pi's resources run away with htop as well when it was happening? When I had this issue, the octoprint service would use 175%+ of the cpu.

Back when i had this issue we were still on the Maple HAL for the skr mini series now were on a different HAL and its still happening.

[skl111]

Haven't checked, but I suspect that is happening when the pi goes offline after a few times of the printer rebooting.

[thinkyhead]

If you send the M303 from the OctoPrint terminal does it continuously print messages during the tuning process? And then, does it complete successfully when started in that way? If you initiate tuning from the LCD while the OctoPrint terminal is open is there no difference? i.e., Do you still see the status messages in that case?

[skl111]

Yes, both LCD & terminal I captured that in the log. Here it is the excerpt.

2021-09-22 02:48:01,472 - Recv:  T:23.09 /0.00 B:24.45 /0.00 @:0 B@:0
2021-09-22 02:48:02,071 - Send: M303 E0 C5 S230
2021-09-22 02:48:02,094 - Recv: echo:M303 E0 C5 S230
2021-09-22 02:48:02,108 - Recv: PID Autotune start
2021-09-22 02:48:02,151 - Recv:  T:23.40 /0.00 B:24.53 /0.00 @:127 B@:0
. . .
2021-09-22 02:48:56,082 - Recv:  T:132.37 /0.00 B:24.53 /0.00 @:127 B@:0
2021-09-22 02:49:00,282 - Unexpected error while reading serial port, please consult octoprint.log for details: SerialException: 'device reports readiness to read but returned no data (device disconnected or multiple access on port?)' @ comm.py:_readline:3860

[skl111]

FYI to add to this I was working on updating my firmware with a patch that not in bugfix and using the latest bugfix as of 11-06-2021 that is for a different issue and I started running a PID bed tune to test that patch and I forgot to disable the meatpack plugin and the printer rebooted and octoprint server went offline and the raspberry pi lost network wireless network connectivity like a DOS caused by the meatpack protocol.

[thinkyhead]

I wonder if @scottmudge or others with more Meatpack experience have encountered this and know a workaround.