TMC 2208 drivers are disabled while printing

[Nume1977]

I have just installed klipper branch 20180220a on my Anycubic kossel delta with trigorilla board + RPi3b and a few lines after the print has started one of the stepper motors turns off (i can move it by hand)! I have had this problem on X axis or Y axis (Z has not yet shown this situation).

Here is the log (from start to emergency stop) klippy.zip

[cosminr86]

@KevinOConnor, thanks for the suggestion. I disabled stealthchop and lowered the current of the z axis and I had a couple of successful prints (30h one). I will try with the stealtchop set to 7 as you advise, and I will let you know if it is ok.

Thank you!

[Artem-B]

I may have figured out how to make TMC2208 drivers to behave a bit better. The post a bit long. TL;DR; version is -- disable PWM_AUTOGRAD and set PWM_GRAD to a value slightly higher than PWM_GRAD_AUTO.

---

I've hooked up current sensor to TMC2208 on my printer. Looking at the waveform with an oscilloscope showed that with fast enough acceleration (3000) from a stand-still, the stepper fails to drive full current amplitude into the motor and I sometimes can hear tell-tale knocks of skipped steps. The waveform looks very similar to this one taken from the video posted by Trinamic:

Trinamic recommends reducing acceleration and it indeed helps. In my case with acceleration reduced to <1000 the driver can reach configured current with no problem and the steps are no longer lost.

Going slower is a known solution, but that got me thinking. The driver had no problem driving full current amplitude while the motor was going fast, which is the hardest part due to increased back EMF, so the failure to drive full current at the beginning was likely due to the automatic tuning mechanism. AFAICT from reading the datasheet this is mostly needed to deal with changing driving voltage and changed load. Neither is common in 3d printer. Motor power supply is typically fixed at 12v or 24v and the carriage mechanical characteristics also do not change much (modulo weight increase due to filament deposition).

PWM_GRAD parameter looks like the primary suspect -- it's responsible for how fast the stepper scales output voltage in relation to motor speed. E.g. if PWM_GRADis too low, then the stepper may not be able to compensate for back-EMF and the current will not reach the level we need. The datasheet of TMC22xx seems to have less info about the process, but the Figure 6.4 in TMC2130 datasheet does show that decreased current during acceleration is the result of too small PWM_GRAD:

So, I've disabled PWM_AUTOGRAD and set PWM_GRAD to a value slightly higher than the one the driver determined as reported by PWM_GRAD_AUTO field in PWM_AUTO register. I've also set PWM_OFS to match that of PWM_OFS_AUTO. That does two things. First, explicitly setting PWM_GRAD and PWM_OFS allows the driver to start from already-tuned settings for the motor, so it should work better after power-on before the automatic tuning cycle is finished. Second is that by disabling gradient autotuning, we can tell the driver to scale the driving voltage faster with increased speed. We may overshoot the current limit once we reach the steady state, but that's less of a problem than skipped steps.

In the end, I've added following settings to each or the [tmc2208 ...] sections:

driver_PWM_AUTOGRAD: False
driver_PWM_GRAD: GG
driver_PWM_OFS: OO

where GG is decimal value of the upper 16 bits of PWM_AUTO register and OO is the value of the low 16 bits of PWM_AUTO as reported by DUMP_TMC command for the given stepper after it got a chance to move and idle for some time (e.g. after homing the axis or extruding some filament).

EDIT: @derdigge explained the steps in the post below :

Send: dump_tmc stepper=extruder
...
Recv: // PWM_AUTO: 000e0024

The value which is converted to dez 000e = 14 and 0024 = 36

so in config:

driver_PWM_AUTOGRAD: False
driver_PWM_GRAD: 14
driver_PWM_OFS: 36

I've eventually picked PWM_GRAD values for each motor that allow me to drive them with acceleration set as high as 3000 without any missing steps.

One issue with this approach is that it's probably going to be tricky to figure out correct PWM_GRAD settings without checking the current waveform with an oscilloscope. Without that, one should probably start with the values in PWM_AUTO, bump PWM_GRAD slowly until missing steps are gone and watch out for motors getting too hot (in case too high PWM_GRAD results in a current too high for the motors).

[ghost]

@Artem-B

Thanks for that detailed explaination. Unfotunatly your settings wont work for me. Maybe others noticed that one also. Can you please specify the correct config?

Error: Unable to parse option 'driver_PWM_GRAD' in section 'tmc2208 extruder'

`[tmc2208 extruder]

uart_pin: ar37

tx_pin: ar35

microsteps: 16

interpolate: True

run_current: 1.0

sense_resistor: 0.110

stealthchop_threshold: 500

driver_IHOLDDELAY: 8

driver_TPOWERDOWN: 20

driver_BLANK_TIME_SELECT: 2

driver_TOFF: 3

driver_HEND: 0

driver_HSTRT: 5

driver_PWM_AUTOGRAD: False

driver_PWM_AUTOSCALE: True

driver_PWM_LIM: 12

driver_PWM_REG: 8

driver_PWM_FREQ: 1

driver_PWM_GRAD: GG

driver_PWM_OFS: OO`

[Artem-B]

GG and OO are supposed to be the decimal values for the gradient and offset correspondingly, extracted from the appropriate fields of the PWM_AUTO register. Note that the register values are reported as hexadecimal numbers, but klipper config wants to see decimal values, so you'll need to convert the values.

[rodjerhell]

каким образом узнать необходимые значения ? какой алгоритм для этого ?

[ghost]

EDIT:

With a help of a friend i thing i got it:

my output :

Recv: // PWM_AUTO: 000e0024

which is converted to dez 000e = 14 and 0024 = 36

so in config:

driver_PWM_AUTOGRAD: False driver_PWM_GRAD: 14 driver_PWM_OFS: 36

Is This correct?

@Artem-B Sorry but i understand half what your explaination post tells us. As it looks you´re the only person who solved this. A Hex to dez converter dont know GG or OO, thats why. Can you please for me and maybe others try to help how this is done? Theese seem to be obvious things for you. At least for me it is not.

Lets take a look at my output:

Send: dump_tmc stepper=extruder
Recv: // GCONF:          000001c0
Recv: // GSTAT:          00000001
Recv: // IFCNT:          00000042
Recv: // OTP_READ:       0000000c
Recv: // IOIN:           20000141
Recv: // FACTORY_CONF:   0000000c
Recv: // TSTEP:          000fffff
Recv: // MSCNT:          000003e8
Recv: // MSCURACT:       00f401dc
Recv: // CHOPCONF:       04030053
Recv: // DRV_STATUS:     c0130000
Recv: // PWMCONF:        c80d0e24
Recv: // PWM_SCALE:      00000016
Recv: // PWM_AUTO:       000e0024

So my PWM_AUTO Value is 000e0024 which is 917540 in decimal regarding to excels hex to dez converter. This confuses me in total as it is not even close to anything i saw here and in default configs.

Whats wrong with my brain? please help me.

[lorf]

@rodjerhell @derdigge I made a patch that will decode some TMC2208 registers to decimal values, You can temporarily use it while tuning your drivers: https://github.com/lorf/klipper/tree/tmc2208-decode

Send: dump_tmc stepper=stepper_a
Recv: // GCONF:          0x000001c4
Recv: // GSTAT:          0x00000001
Recv: // - uv_cp (undervoltage): 0
Recv: // - drv_err (error shutdown): 0
Recv: // IFCNT:          0x00000006
Recv: // OTP_READ:       0x0000000d
Recv: // IOIN:           0x20000148
Recv: // - VERSION: 0x20
Recv: // FACTORY_CONF:   0x0000000d
Recv: // TSTEP:          0x000fffff
Recv: // MSCNT:          0x000000b4
Recv: // MSCURACT:       0x006e00dd
Recv: // CHOPCONF:       0x13030053
Recv: // DRV_STATUS:     0x80130040
Recv: // - stst (standstill): 1
Recv: // - stealth (stealthChop): 0
Recv: // - CS_ACTUAL (actual current scale): 19
Recv: // - t157 (157 C reached): 0
Recv: // - t150 (150 C reached): 0
Recv: // - t143 (143 C reached): 0
Recv: // - t120 (120 C reached): 0
Recv: // - olb (open load B): 0
Recv: // - ola (open load A): 1
Recv: // - s2vsb (low side short B): 0
Recv: // - s2vsa (low side short A): 0
Recv: // - s2gb (short to ground B): 0
Recv: // - s2ga (short to ground A): 0
Recv: // - ot (overtemperature): 0
Recv: // - otpw (overtemperature warning): 0
Recv: // PWMCONF:        0xc80d0e24
Recv: // PWM_SCALE:      0x00000016
Recv: // - PWM_SCALE_SUM: 22
Recv: // - PWM_SCALE_AUTO: 0
Recv: // PWM_AUTO:       0x000e0024
Recv: // - PWM_OFS_AUTO: 36
Recv: // - PWM_GRAD_AUTO: 14
Recv: ok

[Artem-B]

Recv: // PWM_AUTO: 000e0024

which is converted to dez 000e = 14 and 0024 = 36

so in config:

driver_PWM_AUTOGRAD: False driver_PWM_GRAD: 14 driver_PWM_OFS: 36

Is This correct?

Yes, that's exactly what needs to be done. I could've done better job explaining these steps. Sorry about that. :-( Also, thank you for posting the explanations. I'll update my post with these details.

[rodjerhell]

I had problems with turning off the printer, I can not figure out the reason. how to find the problem? 2208 driver klippy.log

[ghost]

@Artem-B Nothing to be sorry about. Not all people share thyre knowledge in detail. There is one thing i still did not get. When do i have to take the PWM_AUTO values? Mine are different everytime i try to pick them up....

[Artem-B]

The values should be valid after the driver went through the automatic tuning process (Chapter 6.1. in the TMC2208 datasheet)

Basically, the sequence looks roughly like this:

• enable the motor, so the full configured current runs through the coil.
• stand still for 130ms or more. [Note: you may want to set hold_current current to be the same as run_current so this phase is not accidentally done with the reduced current. By default the driver switches to hold_current after ~ 0.4s of idle time (driver_TPOWERDOWN).]
• move motor at a reasonably high speed for few hundreds of full steps (~few motor rotations)

The values adjust to the properties of the particular motor, so they may be somewhat different even for the motors of the same model. They may also adjust depending on external factors like motor power supply (fluctuating voltage will affect the gradient, IIUIC). In any case, the values you get are just a starting point for further tuning. The idea is to get the automatically determined parameters that proved to be somewhat suboptimal, and then manually adjust them so the driver works in situations that autotuning can't cope with.

[ghost]

@lorf I tested your patch today. DRV status can help people a many. But its not complete so far. I am sure that this would be a candidate for a pull request here. I do currently decode it in excel.

[lorf]

@derdigge OK, filed pr #1156

[killvearn]

I'm having this issue with my TMC2208 too in standalone mode (I don't have pins on the board to use UART). I changed the extruder driver with an A4988 and the problem on E stepper is gone, but still on some prints and at the same spot the Y axes stops working while the rest of the axes keeps going. I was thinking, given that with marlin the problem isn't showing, maybe was addressed using the TMC2208_STANDALONE parameter in configuration.h. Here's the difference between the A4988 and the TMC2208_STANDALONE mode: A4988: MINIMUM_STEPPER_DIR_DELAY 200 - MINIMUM_STEPPER_PULSE 1 - MAXIMUM_STEPPER_RATE 500000 TMC2208_STANDALONE: MINIMUM_STEPPER_DIR_DELAY 20 - MINIMUM_STEPPER_PULSE 0 - MAXIMUM_STEPPER_RATE 400000 Is there a way I can make those changes in Klipper so I can test if the problem goes away?

[mattthebaker]

I've read through I believe the entire issue. Other than suggestions to try, it doesn't seem that any tests have been done with interpolation disabled. I've been reading the tmc2208 datasheet, and the "microplyer" aka interpolater most certainly uses delta t to the previous step when calculating the time base for interpolation.

It interpolates the time in between of two step impulses at the step input based on the last step interval. This way, from 2 microsteps (128 microstep to 256 microstep interpolation) up to 256 microsteps (full step input to 256 microsteps) are driven for a single step pulse. The step rate for the interpolated 2 to 256 microsteps is determined by measuring the time interval of the previous step period and dividing it into up to 256 equal parts

Fig 11.2, TMC220x pg 60.

Could the very short step time in the problematic gcode result in very short/fast interpolated microsteps and an associated fast current ramp (enough to trigger the protections)? You would expect it to not be any worse than performing an un-interpolated step. Its not clear if the systems are coupled in more ways than just the internal steps. The StealthChop2 issue might just be cascading from the conditions created by the interpolater.

It is unclear how it is intended to respond to direction changes.

The attention warning might be a significant red flag.

Knowing whether the issue is present without interpolation, and whether it is present when running a higher external microstepping could be useful for determining root cause.

[KevinOConnor]

@mattthebaker - if one has the uart hooked up, my theory is that the problem goes away as long as one sets a reasonable stealthchop_threshold parameter as described in https://github.com/KevinOConnor/klipper/issues/196#issuecomment-447605307 . Specifically, with a reasonable stealthchop_threshold, my theory is that these quick steps on a direction change would result in an instantaneous switch to spreadcycle mode and then back, which should be harmless.

So, I agree that step interpolation may influence the main problem, but once one has uart I think it's possible to avoid the issue without having to change that setting.

-Kevin

[Yannik]

@Artem-B Would you care to make a quick guide how to determine the correct PWM_GRAD with an oscilloscope? I'm currently not sure how to wire this up and what to look for.

[Artem-B]

It's pretty straightforward. I've bought a current sensor, wired IP+/IP- points in a break in the wire going towards one of the coils and connected oscilloscope between GND and VOUT.

Then you can observe the current through the coil during axis movement commands. The absolute value does not matter much. We mostly care about relative current amplitude throughout movement. There are three distinct phases -- acceleration, steady speed, deceleration. Acceleration/deceleration phases are affected by PWM_GRAD. If it's too low, you will see that the current amplitude is noticeably lower than that during constant speed phase. You need to adjust PWM_GRAD so that the current amplitude during acceleration is similar (or, maybe a bit higher) than that during the constant velocity phase. Or you can reduce acceleration which would allow the driver to deliver higher current throughout the acceleration phase.

[lekv]

@mattthebaker - if one has the uart hooked up, my theory is that the problem goes away as long as one sets a reasonable stealthchop_threshold parameter as described in #196 (comment) . Specifically, with a reasonable stealthchop_threshold, my theory is that these quick steps on a direction change would result in an instantaneous switch to spreadcycle mode and then back, which should be harmless.

So, I agree that step interpolation may influence the main problem, but once one has uart I think it's possible to avoid the issue without having to change that setting.

-Kevin

Today I found this not to be the case. After enabling and tuning pressure pdvance and a handful of successful prints, I had the extruder driver shut itself off during a print. I don't have stealthchop_threshold configured for the extruder at all, effectively running it in spreadcycle mode.

[extruder]
max_extrude_only_distance: 100.0
step_pin: ar26
dir_pin: !ar28
enable_pin: !ar24
# Calibrated 97 steps / mm
step_distance: 0.01030927835
nozzle_diameter: 0.400
filament_diameter: 1.75
heater_pin: ar10
sensor_type: EPCOS 100K B57560G104F
sensor_pin: analog13
control: pid
pid_Kp: 21.527
pid_Ki: 1.063
pid_Kd: 108.982
min_temp: 0
max_temp: 250
pressure_advance: 0.5
pressure_advance_lookahead_time: 0.010

[tmc2208 extruder]
uart_pin: ar65
microsteps: 16
# RMS at 100%
run_current: 0.709

I was surprised by this and I haven't had any issues with drivers disabling prior to turning on pressure advance. Skimming through the logfile I didn't see any error message. I'll try disabling interpolation as I'm otherwise out of ideas.

[KevinOConnor]

@lekv - there are a number of reasons that a trinamic driver may turn itself off. If you weren't using "stealthchop" then it's not due to this issue. If you see the failure again, you should run DUMP_TMC on the stepper to see if it is reporting an error (such as an over heating event). If you need further assistance, open a separate issue.

-Kevin

[lekv]

Thanks for the feedback @KevinOConnor. I configured the driver to use StealthChop mode and set stealthchop_threshold: 41. After a handful of successful prints, it shut itself off again. It also reports an error:

Send: dump_tmc stepper=extruder
Recv: // ========== Write-only registers ==========
Recv: // IHOLD_IRUN: 00081616 IHOLD=22 IRUN=22 IHOLDDELAY=8
Recv: // TPWMTHRS:   000000bd TPWMTHRS=189
Recv: // TPOWERDOWN: 00000014 TPOWERDOWN=20
Recv: // ========== Queried registers ==========
Recv: // GCONF:      00000101 I_scale_analog=1(ExtVREF) multistep_filt=1
Recv: // GSTAT:      00000003 reset=1 drv_err=1(ErrorShutdown!)
Recv: // IFCNT:      00000000
Recv: // OTP_READ:   00000010 OTP_FCLKTRIM=16
Recv: // IOIN@TMC220x: 2000034d ENN=1 MS1=1 MS2=1 PDN_UART=1 SEL_A=1(TMC220x) DIR=1 VERSION=0x20
Recv: // FACTORY_CONF: 00000010 FCLKTRIM=16
Recv: // TSTEP:      000fffff TSTEP=1048575
Recv: // MSCNT:      00000138 MSCNT=312
Recv: // MSCURACT:   01ac00e8 CUR_A=232 CUR_B=-84
Recv: // CHOPCONF:   14010053 toff=3 hstrt=5 TBL=2 MRES=4(16usteps) intpol=1
Recv: // DRV_STATUS: c01f0000 CS_ACTUAL=31 stealth=1 stst=1
Recv: // PWMCONF:    c80d0e24 PWM_OFS=36 PWM_GRAD=14 pwm_freq=1 pwm_autoscale=1 pwm_autograd=1 PWM_REG=8 PWM_LIM=12
Recv: // PWM_SCALE:  00000022 PWM_SCALE_SUM=34
Recv: // PWM_AUTO:   001a0022 PWM_OFS_AUTO=34 PWM_GRAD_AUTO=26
Recv: ok

Based on the explanation from Bernhard from Trinamic, I imagine this could happen if the extruder changes velocity to quickly when using Pressure Advance (I didn't see this without PA configured):

I think, the reason for disabling of the TMC2208 driver could be

• a hard stop of the motor in stealthChop (Step frequency goes from a higher value, e.g. > 0.5 RPS to 0)
• an abrupt change of motor velocity (Step frequency goes from a higher value to a low value within a single step).

When using stealthChop, please always make sure, that you use velocity ramping. A hard stop will cut away motor back-EMF at once. As stealthChop is a voltage based chopper, it cannot respond to this at once, like spreadCycle. The result is an overcurrent, and the motor driver goes to overcurrent switch off, until it becomes disabled / enabled again.

To resolve the problem, please use at least a tiny velocity ramping, when hard stopping the motor, e.g. within a few / a few ten microsteps.

A subsequent print of the same .gcode file without any changes succeeded. Does that indicate that my issue is different from this one, e.g. overheating?

[KevinOConnor]

@lekv - we'd need the Klipper log file to assist. Please open a separate issue, as it does not appear related to this one.

-Kevin

[Vorchak]

I am experiencing a similar issue intermittently. I am using Klipper on a Geeetech A20M (GT2560) with TMC2208. When it occurs, the Y stepper stops responding while X and Z continue moving as expected. Moreover, I believe the Y stepper is no longer energized as I am able to push the bed back and forth. The behavior continues until I cancel the print which will issue an M84 disabling the steppers. After this G28 homing works again on all axis. At first I thought it was due to excessive speed or anything else causing the TMC2208 to overheat. However, that cannot be as there is forced cooling and I see that the driver is not hot. After further experimentation, I have found out that simple actions that as rotating the printed object 90 degrees will cause the problem to go away. Please let me know if you need logs and tell me how to get them. I will make sure to collect them the next time it happens. Additional info: the TMC2208 is in legacy mode with 1/16 steps. I believe that uses stealthchop 2 mode. Another evidence that this is not due to overheating is because it does not always happen and it can even print successfully at much higher speeds and over even longer periods of time. When it happens, it is the middle or early stages of the print, after about 20-50 layers. I have also confirmed the driver is not bad by switching it with other axis. The problem seems to be related to the motion or signals on the Y axis.

[jakep82]

Additional info: the TMC2208 is in legacy mode with 1/16 steps. I believe that uses stealthchop 2 mode.

This is a known issue with no solution, and it's been documented. You will continue to have print failures as long as you use these drivers in standalone mode.

https://github.com/KevinOConnor/klipper/blob/master/docs/FAQ.md#my-tmc-motor-driver-turns-off-in-the-middle-of-a-print

[lunohod]

Do you know if the new TMC2209 is also affected by this issue?

[Vorchak]

I am not sure if the TMC2209 has the same issue; however, the odd thing is that I have tested the same print models with both klipper and marlin 2 and I only see this issue with klipper. I would much rather use klipper as it solves all of my other issues with marlin but this particular issue is a blocker. I am using the same parameters with both as far as I know. Also, why only Y axis? It does not seem to happen with X axis.

[Matou25]

Hi,

i use TMC2130 and during delta calibration motors disable randomly, sometime it's ok, sometimes not klippy.zip .... i try do to disable stealthChop, interpolation , but it was the same update 12 oct 19 i had fan near tmc stepper, print and delta calibration work , it's strange because on marlin i can print same gcode file without fan and without issue....

[KevinOConnor]

With the merge of PR #2565 it is thought that Klipper will no longer have an issue with the tmc2208, even if the tmc2208 is in standalone mode.

-Kevin

[KevinOConnor]

Alas, it appears that PR #2565 was defective. I've reverted that PR (as of commit e872290c4). I hope to have a replacement in a week or so.

-Kevin

[KevinOConnor]

FYI, the updated work around in PR #2597 was merged a week ago. So hopefully this issue is now resolved.

-Kevin