[BUG]New thermal model renders MK3 unusable

pstimpel commented 1 year ago

MK3 - stock version 3.12.0

USB/Octoprint

Thermal anomaly shows up during normal print, temperatures look ok, cables tested, and cannot disable that annoying beep

To Reproduce Honestly, I don't know, but it happens at every single print I do

Expected behavior

allow me to disable those warnings on my own risk
do not beep if I did set sound to silent in the settings, so stop ignoring my settings
make the model more tolerant (?), since there seems to be no temperature runaway or anomaly based on the temperature logs/graphs in octoprint

G-code standard gcode out of prusaslicer, and it is not specific to a certain sliced model

Btw, prints proceed and do not fail. Just the message on the printers display and loud annoying repeated beeps despite the silent setting. I cannot contribute further, I switched back to 3.11.

wavexx commented 1 year ago

You can disable it by sending M310 S0 followed by M500 to save the setting and this will persist across resets so you can still use 3.12.

Before doing that, however, we'd really like to know why it does this. I guess you followed the self-calibration. Does the print pause, or it just beeps with thermal anomaly?

If it happens at every print, it would be nice if you could record the print data. Since you're already using octoprint it should be pretty easy. Ensure the serial log is active and large enough. Insert:

M310
M155 S1 C3
D70 S1

as the start g-code, then run the print. This will generate copious output. Please save both the serial log and the gcode you sent to the printer, zip it and attach it here.

From this we can see if there is a model discrepancy (for example, the calibration didn't work) or there might be a more subtle hardware issue.

pstimpel commented 1 year ago

Thanks a lot. I tried to find information on those GCodes but failed. Where are they documented?

The print continues when that happens, but the printer keeps beeping in more or less short intervals. And yes, I ran the calibration thingy right after I updated to the new firmware 3.12.

Right now I need to make some progress with my prints. As soon as I find some time I will try to run some tests with 3.12.

Still, such things should be able to be enabled/disabled in the printers menu, BEFORE it becomes a productive firmware. The way this feature was introduced is not OK for me.

wavexx commented 1 year ago

M310 has been added to the reprap wiki:

https://reprap.org/wiki/G-code#M310:_Temperature_model_settings

M500 is standard.

Still, such things should be able to be enabled/disabled in the printers menu, BEFORE it becomes a productive firmware. The way this feature was introduced is not OK for me.

3.12 RC1/RC2 introduced this feature after a lot of testing. Since it can catch hard faults before any damage is done it was enabled by default in the final release.

So far all the issues reported on stock hardware have been true hardware issues, but sometimes it's unfortunately not easy to debug. For example, it can be as simple as a fan which doesn't spin properly anymore, and if that doesn't make any rattling noise it's hard to tell....

pstimpel commented 1 year ago

Thanks for the link.

I agree this is an important feature, and I agree it should become default enabled. But being a pro software dev myself, I know about the differences between RC testing and productive release, very well. No matter how much effort you spend on RC testinmg, there will be issues in production, because the number of different envorinments ist much more broad compared to testers. The need to fiddle with special gcode for customers is somehow not what I would expect from productive software, especially if a feature can "break" a printer and make it useless. Please remember, we talk about Prusa, well known for delivering printers to non-tech guys.

However, thanks for your help. Have a great weekend, @wavexx

fink-at-trmc-dk commented 1 year ago

Same issue here. Luckily I had an octopi connected for the first time so I could send those magic commands via the terminal to disable the themal model. I would be VERY nice to let us enable/disable this via the menu system instead of fiddling with GCodes this way. If I could not disable it I could not print until rolling back to 3.11...

//Fink

wavexx commented 1 year ago

@fink-at-trmc-dk since you have octopi attached, can you log the serial during calibration? It would be useful to understand why it's failing.

While saving, reset the printer then issue the following gcode:

M155 S1 C3
D70 S1
M310 A F0

the above just performs the calibration, doesn't actually enable the model check. Please attach the full log in a zip file here.

If you want to do an extra test, try to actually enable the model (send M310 S1) and do a test print after that without resetting. Does the print complete, or does it still trigger thermal anomaly?

The two issues are slightly different. For @pstimpel calibration succeeds, but printing fails. Self-calibration could be an issue.

For your case @fink-at-trmc-dk the printer doesn't actually pass the hard-coded self-test values. It can be a degraded heater or thermistor.

fink-at-trmc-dk commented 1 year ago

Hi again, I would like to help but I have already downgraded to 3.11 in order for me to print agian. Sorry :-) BR Fink

On Sun, Jan 15, 2023 at 1:37 PM wavexx @.***> wrote:

@fink-at-trmc-dk https://github.com/fink-at-trmc-dk since you have octopi attached, can you log the serial during calibration? It would be useful to understand why it's failing.

While saving, reset the printer then issue the following gcode:

M155 S1 C3 D70 S1 M310 A F0

the above just performs the calibration, doesn't actually enable the model check. Please attach the full log in a zip file here.

If you want to do an extra test, try to actually enable the model (send M310 S1) and do a test print after that without resetting. Does the print complete, or does it still trigger thermal anomaly?

The two issues are slightly different. For @pstimpel https://github.com/pstimpel calibration succeeds, but printing fails. Self-calibration could be an issue.

For your case @fink-at-trmc-dk https://github.com/fink-at-trmc-dk the printer doesn't actually pass the hard-coded self-test values. It can be a degraded heater or thermistor.

— Reply to this email directly, view it on GitHub https://github.com/prusa3d/Prusa-Firmware/issues/3908#issuecomment-1383140512, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQ6NZQEINITQKSJQKDQLLGTWSPVSFANCNFSM6AAAAAAT3G5U34 . You are receiving this because you were mentioned.Message ID: @.***>

ackerthehacker2 commented 1 year ago

I am seeing somewhat of the same issue ...during initial bed calibration, I am getting a TA ...after 2-3 times, an emergency stop is called. Looking at the temp, maybe it increases 2 degs ...

tmc2130_home_enter(axes_mask=0x01) T:229.9 /230.0 B:85.4 /85.0 T0:229.9 /230.0 @:31 B@:29 P:0.0 A:38.7 echo:busy: processing TM: error |-3.712797|>1.200000 LCD status changed TM: error triggered! LCD status changed ok N100 M117 ETA 1/30 11.34AM109 echo:enqueing "G1 E-1.000 F2700" // action:paused Error:Printer stopped due to errors. Supervision required. M112 N101 M11233 N102 M104 T0 S034 N103 M140 S0103 N0 M110 N0125 start N0 M110 N0125 echo: 3.12.1-5686 echo: Last Updated: Jan 20 2023 10:31:23 | Author: (none, default config) Compiled: Jan 20 2023 echo: Free Memory: 2595 PlannerBufferBytes: 1792 echo:Stored settings retrieved adc_init CrashDetect ENABLED! FSensor ENABLED Sending 0xFF echo:SD card ok ok N0 M110 N0125 ok N0 M110 N0125 ok N1 M115*39 FIRMWARE_NAME:Prusa-Firmware 3.12.1 based on Marlin FIRMWARE_URL:https://github.com/prusa3d/Prusa-Firmware PROTOCOL_VERSION:1.0 MACHINE_TYPE:Prusa i3 MK3S EXTRUDER_COUNT:1 UUID:00000000-0000-0000-0000-000000000000 Cap:AUTOREPORT_TEMP:1 Cap:AUTOREPORT_FANS:1 Cap:AUTOREPORT_POSITION:1 Cap:EXTENDED_M20:1 Cap:PRUSAMMU2:1 ok M20 Begin file list BIN-V3~1.GCO 18367460 PRUSA~3.GCO 409063 WHISTL~1.GCO 857259 PRUSAB~1.GCO 1825245 PRUSA_~2.GCO 318319 TRICER~1.GCO 17667627 VASE_0~1.GCO 7626315 End file list ok M117 IP 192.168.1.211 LCD status changed ok M117 192.168.1.211 LCD status changed ok M155 S2 ok M104 S215 ok M140 S60 ok T:185.0 /215.0 B:79.9 /60.0 T0:185.0 /215.0 @:127 B@:0 P:0.0 A:38.2 T:184.5 /215.0 B:79.6 /60.0 T0:184.5 /215.0 @:127 B@:0 P:0.0 A:38.2 T:185.5 /215.0 B:79.3 /60.0 T0:185.5 /215.0 @:127 B@:0 P:0.0 A:38.2 T:188.6 /215.0 B:78.9 /60.0 T0:188.6 /215.0 @:127 B@:0 P:0.0 A:38.2 T:192.0 /215.0 B:78.7 /60.0 T0:192.0 /215.0 @:127 B@:0 P:0.0 A:38.2 T:195.8 /215.0 B:78.3 /60.0 T0:195.8 /215.0 @:106 B@:0 P:0.0 A:38.2 T:200.0 /215.0 B:78.0 /60.0 T0:200.0 /215.0 @:74 B@:0 P:0.0 A:38.2 T:204.3 /215.0 B:77.8 /60.0 T0:204.3 /215.0 @:37 B@:0 P:0.0 A:38.2 D1 Unknown D code: D1 ok T:208.2 /215.0 B:77.4 /60.0 T0:208.2 /215.0 @:11 B@:0 P:0.0 A:38.2 T:210.6 /215.0 B:77.1 /60.0 T0:210.6 /215.0 @:2 B@:0 P:0.0 A:38.2 T:211.8 /215.0 B:76.8 /60.0 T0:211.8 /215.0 @:12 B@:0 P:0.0 A:38.2 T:212.5 /215.0 B:76.6 /60.0 T0:212.5 /215.0 @:20 B@:0 P:0.0 A:38.1 T:212.1 /215.0 B:76.3 /60.0 T0:212.1 /215.0 @:41 B@:0 P:0.0 A:38.1 MMU not responding - DISABLED T:211.6 /215.0 B:76.0 /60.0 T0:211.6 /215.0 @:61 B@:0 P:0.0 A:38.1 T:211.6 /215.0 B:75.8 /60.0 T0:211.6 /215.0 @:68 B@:0 P:0.0 A:38.1 T:212.4 /215.0 B:75.5 /60.0 T0:212.4 /215.0 @:62 B@:0 P:0.0 A:38.0 T:213.3 /215.0 B:75.2 /60.0 T0:213.3 /215.0 @:53 B@:0 P:0.0 A:38.0 T:214.8 /215.0 B:75.0 /60.0 T0:214.8 /215.0 @:36 B@:0 P:0.0 A:38.0 T:215.8 /215.0 B:74.7 /60.0 T0:215.8 /215.0 @:26 B@:0 P:0.0 A:38.0 T:216.8 /215.0 B:74.4 /60.0 T0:216.8 /215.0 @:18 B@:0 P:0.0 A:37.9 T:217.5 /215.0 B:74.1 /60.0 T0:217.5 /215.0 @:13 B@:0 P:0.0 A:37.9 T:217.4 /215.0 B:73.9 /60.0 T0:217.4 /215.0 @:18 B@:0 P:0.0 A:38.0 T:217.4 /215.0 B:73.7 /60.0 T0:217.4 /215.0 @:21 B@:0 P:0.0 A:38.0 T:217.6 /215.0 B:73.5 /60.0 T0:217.6 /215.0 @:20 B@:0 P:0.0 A:38.0 T:216.8 /215.0 B:73.2 /60.0 T0:216.8 /215.0 @:31 B@:0 P:0.0 A:37.9 T:216.6 /215.0 B:73.0 /60.0 T0:216.6 /215.0 @:33 B@:0 P:0.0 A:37.9 T:216.7 /215.0 B:72.7 /60.0 T0:216.7 /215.0 @:31 B@:0 P:0.0 A:37.9 T:216.7 /215.0 B:72.5 /60.0 T0:216.7 /215.0 @:29 B@:0 P:0.0 A:37.9 T:216.6 /215.0 B:72.3 /60.0 T0:216.6 /215.0 @:29 B@:0 P:0.0 A:38.0 T:216.7 /215.0 B:72.1 /60.0 T0:216.7 /215.0 @:26 B@:0 P:0.0 A:37.9 T:216.6 /215.0 B:71.9 /60.0 T0:216.6 /215.0 @:27 B@:0 P:0.0 A:37.9 T:216.7 /215.0 B:71.6 /60.0 T0:216.7 /215.0 @:25 B@:0 P:0.0 A:37.9 T:216.7 /215.0 B:71.4 /60.0 T0:216.7 /215.0 @:23 B@:0 P:0.0 A:37.8 T:216.6 /215.0 B:71.2 /60.0 T0:216.6 /215.0 @:24 B@:0 P:0.0 A:37.9 T:216.5 /215.0 B:70.9 /60.0 T0:216.5 /215.0 @:24 B@:0 P:0.0 A:37.8 T:216.7 /215.0 B:70.8 /60.0 T0:216.7 /215.0 @:19 B@:0 P:0.0 A:37.8 T:216.2 /215.0 B:70.6 /60.0 T0:216.2 /215.0 @:26 B@:0 P:0.0 A:37.8 T:215.9 /215.0 B:70.3 /60.0 T0:215.9 /215.0 @:29 B@:0 P:0.0 A:37.7

francisdb commented 1 year ago

How do you log this information?

3d-gussner commented 1 year ago

@francisdb Read https://github.com/prusa3d/Prusa-Firmware/issues/3908#issuecomment-1382720540 :wink: and https://help.prusa3d.com/article/thermal-model-calibration_382488 The help article doesn't have line feeds, please enter one gcode per line

M310 ;report current temp model settings
M155 S1 C3 ;enable advanced temp and fan logging
D70 I1 ;enable temp model debug logging
M310 A F0 ;run TM calibration without checking default values non E3d v6 nozzle
M310 ;report new temp model settings
M500 ;when TM cal was successful.

replace line with M310 A0 F1 ;run TM calibration with checking default values E3d v6 hotend

fink-at-trmc-dk commented 1 year ago

Hi again, This time I'm more observant on the temperatures and installed OctoPrint. It seems to me that there indeed is temperature anomaly if I look at the plot from OctoPrint. I do not know if it has been there all the time but it is a factory assembled un-modyfied unit. The question is: What do I do in order to find the fault? Where do I start?

Inputs are all welcome and I expect to roll back to 3.12.1 when the temperatures curve look nice again...

BR Fink

wavexx commented 1 year ago

Yup, this looks pretty wrong. The cable is usually the first source for this sort of problem. See my suggestions here: https://github.com/prusa3d/Prusa-Firmware/issues/3817#issuecomment-1347209233

We're pretty aware this is annoying, but both heater, thermistors and cabling is subject to a lot of failures due to mechanical stress and this is why the model is being pushed on by default. It traps these problems waaay before you can usually notice. There are edge cases for sure, but we're going one-by-one on them.

Rasmus-Fink commented 1 year ago

Been fiddling a bit. Rotated the sensor 180 degrees to see if that made a difference and concluded a new sensor must be ordered. The picture now looks like this:

Thanks for the pointer so far. Will report back when sensor has been replaced and firmware upgraded to 3.12.1.... BR Fink

kompressor2279 commented 1 year ago

Thermal Modeling has been a very large nuisance for me as well. I recently upgraded my mk3 to the mk3s+ and ever since I get TM errors that result in estops. I've done the PID tuning, and the TM calibration that end successfully but prints will seemingly at random will trigger a TM error.

I cannot have a continuous beeping noise, ever. I have a small child in the home and if it happened when they are napping it would not be great to say the least. I don't care if it shuts the printer off, the beep must be adjustable/optional though.
The settings that trigger TM errors are just way too tight. They need to be opened up to be useful.

Some prints work fine, others will trigger this, usually during the first layer, if it can make it beyond that its fine. I use octoprint and my temps look normal, within +-2C, no crazy jumps.

I've had to disable this feature to be able to print.

EDIT: Some background info: The hotend was a kit that I assembled from all new parts. I am using a silicone sock the machine is in an enclosure.

ackerthehacker2 commented 1 year ago

I did the same and disabled it …and ended up with my hotted smoking and melting some parts .. I am a convert …it pays to enable it ...

On Feb 9, 2023, at 3:35 PM, kompressor2279 @.***> wrote:

Thermal Modeling has been a very large nuisance for me as well. I recently upgraded my mk3 to the mk3s+ and ever since I get TM errors that result in estops. I've done the PID tuning, and the TM calibration that end successfully but prints will seemingly at random will trigger a TM error.

I cannot have a continuous beeping noise, ever. I have a small child in the home and if it happened when they are napping it would not be great to say the least. I don't care if it shuts the printer off, the beep must be adjustable/optional though. The settings that trigger TM errors are just way too tight. They need to be opened up to be useful. Some prints work fine, others will trigger this, usually during the first layer, if it can make it beyond that its fine. I use octoprint and my temps look normal, within +-2C, no crazy jumps.

I've had to disable this feature to be able to print.

— Reply to this email directly, view it on GitHub https://github.com/prusa3d/Prusa-Firmware/issues/3908#issuecomment-1424869901, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJHJFLYVGIFROKBY32FXA5DWWVPLDANCNFSM6AAAAAAT3G5U34. You are receiving this because you commented.

kompressor2279 commented 1 year ago

That's a bummer for sure. Wouldn't the thermal runaway protection catch it? Or is it also disabled when disabling TM?

knightsljx commented 1 year ago

there should just be an option to turn it OFF in the menu. it literally limits your printer to the stock 24V 40W heater. Prusa didn't even bother working with their OWN OEM partner E3D to ensure their Revo works. Goes to show the level of thought that went into this 'feature'.

arekm commented 1 year ago

This option doesn't exist in menus because prusa doesn't want for people to disable TM easily. There is a reasoning for that - better safety for example.

There is a way to disable it via g-code commands which is fine for advanced users like revo users.

TM feature is primarily for prusa stock hardware users. Advanced users are perfectly capable of dealing with g-code etc.

And if you track prusa firmware repository commits you will see that they are adding support for other hardware, too even if they don't have to since that's for non standard components.

knightsljx commented 1 year ago

That argument would be fine if the feature was complete. But it's not. It literally only accepts the stock configuration of 40W heaters, which goes against the principle of an open system. If Prusa wants to release fast and take in feedback, then proper warning under documentations and ability to disable the mode easily should be available to users.

Prusa-Support commented 1 year ago

The following may not be relevant for all of the participants of this discussion, but probably for some. The recently released FW 3.12.2 version improves the Thermal Model compatibility over serial connection and with some third-party hardware.

Please give it a try.

Michele Moramarco Prusa Research

github-actions[bot] commented 1 year ago

This issue has been flagged as stale because it has been open for 60 days with no activity. The issue will be closed in 7 days unless someone removes the "stale" label or adds a comment.

github-actions[bot] commented 10 months ago

This issue has been closed due to lack of recent activity.

prusa3d / Prusa-Firmware

[BUG]New thermal model renders MK3 unusable #3908