[BUG] Ghost Y-Axis Crashes

DFliyerz commented 4 years ago

Printer type - MK3S Printer firmware version- 3.8.1

Describe the bug I'm getting repeated "ghost" y-axis crashes when printing models both from the SD card supplied with the printer and models sliced on PrusaSlicer where at similar heights the printer will detect a y-axis crash for seemingly no reason, try to recover, and usually end up slightly offset. I've attached a picture showing two Benchys, one that recovered and was allowed to print further, and one that was cancelled after the crash. Additionally, to the right is a model I sliced in PrusaSlicer, which also crashed and was cancelled.

To Reproduce I tried reproducing the behavior by printing a 20mm tall tower while watching the printer (the crash usually happens between 10-15mm), but the crash didn't occur. I also checked the belts, pulleys, etc. to see if they might have been causing the issue but they all seemed normal.

Expected behavior There doesn't seem to be a reason for these crashes occuring, and even if they are legitimate crashes it shouldn't be recovering with an offset.

photo_2020-05-03_11-11-05

awenelo commented 4 years ago

Can you check that your linear rods are lubricated? They could be causing to much resistance, causing the Trinamic drivers to think that they have crashed.

DFliyerz commented 4 years ago

This printer was freshly assembled, and the only lubrication was from what the bearings came with. How should I go about lubricating it? I have 3-in-1 oil as well as the lubricant supplied with the kit.

awenelo commented 4 years ago

You can use the lubricant supplied in the kit. This page has more information on lubricating the linear rods.

DFliyerz commented 4 years ago

I lubricated the linear rods as described, and it still crashed while printing the benchy.

awenelo commented 4 years ago

Can you check that all your wires are plugged in fully? Also, how much resistance does it take to move the bed?

DFliyerz commented 4 years ago

They are, and I'm not exactly sure how I could describe moving the bed. It's not easy, but it's not really hard?

floridaservices commented 4 years ago

They are, and I'm not exactly sure how I could describe moving the bed. It's not easy, but it's not really hard?

With the printer turned off can you move the axes with a single finger pressure? It should not take much. What I have done in the past is to loosen the belt by partially unbolting the motor to drop it down and relieve the belt pressure, so there is no motor friction in the mix. After you drop the motor you should be able to easily slide the axes from end to end with no binding at all, and very little noise. Any inconsistency here can cause problems with jamming, calibrations, layer shifts, you name it.

My first thought is your U-bolts are too tight. They are very very easy to overtighten, and what happens then is the bearings will drag, and wear more than they should. The instructions really don't go into all this as much as they should, probably because it's tough to explain how to adjust these by feel to a non technical audience. How I have learned to do it is to tighten up the U to where it just touches the bearing, then tighten a 1/8 turn at a time until I cannot move the bottom of the U by hand. No more. It is plenty tight at that point.

DFliyerz commented 4 years ago

I'll check the tightness of the bearings

DFliyerz commented 4 years ago

I re-did the bearing tightening, but it crashed again on the benchy. I wonder if there's just a way to turn down the crash detection sensitivity, so it doesn't keep detecting false crashes.

floridaservices commented 4 years ago

Try slowing the print down and see how it does. I am not questioning your skills at all but trust me getting these printers right is not as easy as you would think. There's a lot of fine tuning and tweaking and adjustong. You should consider yourself in the debugging stage. Does your print get thru self test and xyz calibration?

DFliyerz commented 4 years ago

It does, and I can print fine with the crash detection turned off. Btw this is with the files that are supplied with the printer, so I'd expect them to be able to get through the whole thing just fine.

DRracer commented 3 years ago

@DFliyerz does your problem persist? We have seen reports of Y-crashes for no obvious reason when printing in high temperatures (bed + enclosure). What is the temperature of your Y-motor? Are all your cables connected correctly?

thadogg commented 3 years ago

Hello, I'm currently facing the exact same issue. Loads of Y-crashes starting from around 15mm height. The printer has been working fine for months. Crashes started today. Here is all I can tell about it:

I had to get to the PTFE tube in the heatbreak in order to remove a small piece of PLA that was preventing the filament from reaching the hotend. I can't see about anything that could have triggered the issue doing so though, but it's best to mention that operation, just in case. it had been several days since I last printed something.
Calibration is OK, belt tension X:271, Y:271.
Bed is moving smoothly and freely but still, I added Prusa lubricant to the rods.
I, however, noticed that the PETG profile I was using was misconfigured : 1st layer temp was 240/85, and other layers temp was 250/90 and not the other way around, so most of the print was done at fairly high temp. I build the Ikea LACK enclosure and only one door was half opened so the temps was around 45°C in the enclosure.
I tried opening both doors as well as the top and lowered the temp to 240/80, but the print running at the moment suffered 16 Y-crashes already, after 45min of printing.
I was running the previous firmware when I started facing the issue today. I upgraded to the latest one but the issue is still there.

Since I opened everything, Y-crashed seem to be a bit less frequent, and my MK3S have not yet stopped with the interactive menu asking to resume or abort the print. It just does a re-homing and moves on with the print. So maybe the temperature has something to do with it.

Unfortunately, I don't have anything to mesure the motor temperature. Is there a way to retrieve it through software ?

Edit: The printer eventually stopped, asking to resume or abort. I added a video of the problem.

Y-crash.zip

VVlasy commented 3 years ago

I also have had this issue since I got the printer Back in january. Persisted since 3.8.1 through to 3.9.1. rebuilt, retightened Y Axis many times. Belt rides straight. Bed moves smoothly. Tried changing ubolts to printed bearing mounts. Still persisted. Crashes happen at about same heights only on Y Axis. For truly no reason.

I noticed in the code for tmc2130.cpp that there used to be a different specific value for stallguard threshold just for the y Axis. This was since commented And the value set for all Axis. Maybe this threshold needs adjusting again.

clajarac commented 3 years ago

I have the same problem for a week, the printer stops at a similar height, check the bed, belts and also lubricate the bearings, for the moment deactivate the crush detection, does anyone have any solution?

morphias2004 commented 3 years ago

Exact same issue since updating firmware to v3.9.1

Disabling crash detection is the only workaround to stop all the scarring in the print.

Here is a video of it - happens at 0:29

https://youtu.be/IBGHnGUcXes

I read on the Prusa forum that host stepper drivers can cause this.

We were having consistent 35C+ days where I live, so this is a possible cause.

I am in the process of moving the printer into an enclosure with the power supply and EINSY board relocated to separate electronics enclosure that will have fan forced cooling. I'll also add some heatsinks to the drivers on the EINSY to further assist with heat dissipation.

I'll report back with my results.

strohbinsky commented 3 years ago

Hey there, Any news on these crash detection issues? I'm having the same issues you described with my Y Axis. Spurious crashes. Thanks, Sebastian

wavexx commented 3 years ago

Just out of curiosity, how do you know it's an Y axis crash? I guess you're reading the serial log (octoprint?).

Cooling the Einsy is good, however IC overtemp will not cause missed crash detection by itself (or at least, I don't think this should happen). What's more likely is that the motor itself gets hotter, increasing coil resistance and thus might require some threshold adjustment.

Can you do this test: start a print and after the first layer, and touch (or measure! if you can) the X/Y motors to see if they run roughly at the same temperature. When you get the Y crash, do the same again. Does the Y motor run a lot hotter at that time?

strohbinsky commented 3 years ago

When the next crash happens I will let you know. Thank you!

VVlasy commented 3 years ago

What I had to do to fix it, since prusa support failed to help me after swapping X and Y motors. Was to buy a new Y motor myself, which ended being the true fix for my Y axis crashes.

catid commented 3 years ago

In my case, using the Prusa V2 enclosure to print ASA, I was seeing frequent Y crashes above a certain Z level. The issue was caused by having the printer too far back into the enclosure, so that the cable bundle coming off the bed was hitting the back window. This also caused the belt self test results to report unexpectedly high values (270) or failing. By moving the printer forward to avoid this, the belt self test result is now reading 240. Furthermore this fixed the issue where crash detection was triggering during printing.

strohbinsky commented 3 years ago

In my case I do have a custom case made out of multiplex. It's big enough that the cable bugles cannot touch the walls. I know these are crashes on the Y Axis because the Fail Status page displays it. In my case the Y Crashes reduced the print quality by a lot. The printer was zeroing itself, in many cases it left ablob or a not connected line to the object and when it continued the nozzle crashed into that line in the next layer and everything got worse. I texted with Prusa Chat and they recommended to flip the printed object by 90° and try again. It became interesting after I sent them a video on a different topic (rambling noises even so I lubed all the bearings before assembly). I am often creating smaller objects and when the printer adds layers of infill the nozzle and the bed have to move very quickly. Since that was my first and only printer I had no reference to other printers and was not aware that these might induce vibrations. The friendly Prusa employee encouraged me to adjust speeds in the Slicer to achieve a smooth print. I did so and that improved my printing quality enormously. I had no more crashes. After all that I wonder why I didn't make up that idea by myself, I totally trusted in the Prusa Stock settings- and those are not made to match every print. Now printer happy, Strohbinsky happy.

wavexx commented 3 years ago

Just in case somebody is still running into this, I've created a PR #3089 that correctly shows which axis is affected during a crash, especially for repeated crashes that cause a "Resume?" prompt it would include all the axes affected.

jeremy-wendelken commented 3 years ago

Exact same issue since updating firmware to v3.9.1

Disabling crash detection is the only workaround to stop all the scarring in the print.

Here is a video of it - happens at 0:29

https://youtu.be/IBGHnGUcXes

I read on the Prusa forum that host stepper drivers can cause this.

We were having consistent 35C+ days where I live, so this is a possible cause.

I am in the process of moving the printer into an enclosure with the power supply and EINSY board relocated to separate electronics enclosure that will have fan forced cooling. I'll also add some heatsinks to the drivers on the EINSY to further assist with heat dissipation.

I'll report back with my results.

Did better cooling on the RAMBO board resolve the ghost Y axis crashes?

catid commented 3 years ago

Better cooling also helped me a lot, though didn't completely resolve it. I added a fan to the enclosure.

My final solution will be to just not print high temperatures with the Prusa and instead use my Voron printer.

jeremy-wendelken commented 3 years ago

Thanks Chris,

I've been printing in stealth mode for a while with no issue - so maybe that is a good solution for the ghost crashes as well!

Jeremy

On Sun, 18 Apr 2021 at 08:01, Chris Taylor @.***> wrote:

Better cooling also helped me a lot, though didn't completely resolve it. I added a fan to the enclosure.

My final solution will be to just not print high temperatures with the Prusa and instead use my Voron printer.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/prusa3d/Prusa-Firmware/issues/2653#issuecomment-821879391, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHNTXB7MEHA6GUNCN7JHKYDTJHSJNANCNFSM4MYHMP7A .

wavexx commented 3 years ago

@morphias2004 looking at the video it seems it always gets choppy when the extruder goes to the back-right corner, I guess it's the video only? I don't see anything in the print that would cause any non-smooth movement except the wipe at the seam, which is on the exact opposite corner anyway.

Crashing in relation with print height seems mostly because there's more weight on the plate here, although the position doesn't make sense to me.

One important thing could also be to ensure that the bed moves smoothy when it's hot. When printing enclosed with the bed at 100C and more, the bed flexes along with the spider underneath (and a bit everything around the printer). This can push the rods and the bearings can start to jam near the front or back depending on alignment. I never had this since I also print hot+enclosed frequently, but it wouldn't hurt to check:

preheat enclosure and printer for ASA, so that room temp is around 45C and bed 100C
let it sit for 15+ minutes minimum, so that the whole frame heats up

with steppers disabled, move the bed and check for resistance. Simply realigning the rods on the frame while the printer is hot could help.

@catid do you notice a relation with the object that you're printing, or does it simply fail seemingly randomly? How often does it happen?

wavexx commented 3 years ago

Who would be willing to test a branch that changes the crash threshold logic and report back on false or missed crashes?

https://github.com/wavexx/Prusa-Firmware/tree/sg_logic_tweaks

I can provide some prebuilt FW versions if someone is interested in testing. I also did get some false crash detections, but they're really rare for me and thus hard to test for.

ricardogsm01 commented 3 years ago

In my case I think it is related to the bed temperature expanding the bearings and tightening the U bolts... I tried lowering the bed temperature of an ABS piece from 100 to 85 and the failures disappeared, then with the printer off turned the Y axis bars by hand (place them in a position they were they had not yet been touched by the bearings) and in the next ABS piece the failures became much worse... I had to lower the temperature to 50 C to make the failures go away. Print quality improved a lot..I will loosen the U bolts a bit....hopefully I am on the right track

catid commented 3 years ago

I was printing the Voron parts out of ASA. On some of the parts that were simpler (boxy) it didn't have trouble. On other models it seems to happen consistently after about an hour of two of printing, when it gets to multiple island features that are smaller so the x/y axes needs to rapidly change direction more, z axis going up and down etc. Motors were getting too hot to touch!

dismantl commented 3 years ago

I am also having this issue and the problem goes away completely when printing outside of my enclosure due to the temperature difference. Is there a way to adjust the crash detection threshold in the firmware?

I am considering trying to swap out the Y axis motor to see if that makes a difference, since I'm only experiencing ghost crashes in the Y axis. @VVlasy what Y motor did you end up buying that fixed the issue for you?

VVlasy commented 3 years ago

Original prusa Y Axis motor for MK3S

parhelia commented 3 years ago

Experiencing phantom crashes on the Y-axis as well. Printer shows X and Y-axes belts' tension to be within the acceptable range (around 280). Rods lubricated many times with Superlube 21030. Bearings slide very well with just finger pressure without motor friction. Only thing I notice is that for some reason the belt on the Y-axis pulley is always right up against the flange closer to the motor, and no amount of tweaking can get it to be centered on the pulley. I'm not sure if this makes a difference because the prints complete fine without any artifacts if I turn off crash detection. Just want to get this feature working.

parhelia commented 3 years ago

I am also having this issue and the problem goes away completely when printing outside of my enclosure due to the temperature difference. Is there a way to adjust the crash detection threshold in the firmware?

I am considering trying to swap out the Y axis motor to see if that makes a difference, since I'm only experiencing ghost crashes in the Y axis. @VVlasy what Y motor did you end up buying that fixed the issue for you?

OK it is definitely heat related. The trigger point seems to be ambient temperatures of around 40˚C or higher. If I leave the enclosure door open and when printing PLA, there are no crashes. If I print the exact same model with PETG and the door closed, it will report the crash.

parhelia commented 3 years ago

Who would be willing to test a branch that changes the crash threshold logic and report back on false or missed crashes?

https://github.com/wavexx/Prusa-Firmware/tree/sg_logic_tweaks

I can provide some prebuilt FW versions if someone is interested in testing. I also did get some false crash detections, but they're really rare for me and thus hard to test for.

Willing to test this. Please prepare firmware for Mk3S+. Thank you.

leptun commented 3 years ago

BOARD_EINSY_1_0a.zip @parhelia Here you go. Builds from the sg_logic_tweaks branch. These are multi-lang builds, so use Prusa-Slicer firmware updater.

ricardogsm01 commented 3 years ago

Temperature is one cause as it expands the bearings and these pull on the u bolts. I got rid of the metallic bearings and changed them for plastic bushings while also loosening the u bolts. I also found that over time the bearings erode the rods. The nozzle dips into the print in the areas where the x axis rods are most eroded. There was a significant noise reduction, print quality improved and the Y crashes disappeared.

On Thu 10 Jun 2021 at 8:10 Alex Voinea @.***> wrote:

BOARD_EINSY_1_0a.zip https://github.com/prusa3d/Prusa-Firmware/files/6631624/BOARD_EINSY_1_0a.zip @parhelia https://github.com/parhelia Here you go. Builds from the sg_logic_tweaks branch. These are multi-lang builds, so use Prusa-Slicer firmware updater.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/prusa3d/Prusa-Firmware/issues/2653#issuecomment-858608435, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIJ74LCD7S5XIJWLDJJS353TSC2UDANCNFSM4MYHMP7A .

Hauzman commented 3 years ago

Hello I Have the same issue on my mk3+ having y crash. I also checked all but from time to time it give crash randomly. Yesterday i have printed something around 3h and sam thing around 5 to 10 mm height it started crashing in 15 min had 27 finished the print with detention off Restart the print and started at the first layer and at the brim to an the right side around the middle to the end and from fest side and i have noticed the rods ware quite hot I have let the printer to coll down to room temperature and restart the print and did not have any crash I was printing now Petg

parhelia commented 3 years ago

BOARD_EINSY_1_0a.zip @parhelia Here you go. Builds from the sg_logic_tweaks branch. These are multi-lang builds, so use Prusa-Slicer firmware updater.

Whatever you changed, it is very promising! I just finished a short test print (about 70 minutes) that would normally yield at least 7 Y-axis crashes, and it completed without incident. I performed a belt status check immediately upon the completion of the print, when the enclosure temperature was 49˚C, and the belt tension on both axes increased. Will continue to monitor. Thanks once again.

wavexx commented 3 years ago

@parhelia please report back. It would also be useful if you could run a small test print within the hot enclosure and test if crash detection is still working as intended and reacts quickly (just push the extruder away while printing after the first layer is completed).

parhelia commented 3 years ago

@parhelia please report back. It would also be useful if you could run a small test print within the hot enclosure and test if crash detection is still working as intended and reacts quickly (just push the extruder away while printing after the first layer is completed).

The problem is back, ambient is now at 39˚C, enclosure is about 50ºC. Again, no issues on X-axis, but Y-axis crashes at 3 spots while laying down the skirt. If I stop the print, clean the build surface, and restart it, it crashes at the same 3 spots each time. Again, printing with crash detection off completes successfully.

wavexx commented 3 years ago

On Thu, Jun 10 2021, parhelia wrote:

The problem is back, ambient is now at 39˚C, enclosure is about 50ºC.

I know you mentioned earlier on you tested that the axes move smoothly (without the belt). Did you also test this with the warmed-up printer?

Fastest method to check is to let the printer in the enclosure exactly as it is now, and just unscrew the Y idler.

I'd like to know if you can feel (even just minor) increases in friction in these spots, or if the carriage is fluid throughout even when hot.

Another test I asked was to measure the temperature of the Y motor. Doesn't need to be precise, but if you heave a thermocouple it would be nice. Ideally I'd like to know if it feels hotter than the X motor.

If I stop the print, clean the build surface, and restart it, it crashes at the same 3 spots each time.

If the carriage is absolutely smooth at that spot, then it's probably due to the rotor alignment to one phase. Another way to test this is by removing the idler, rotate the shaft of the Y motor 1/2 turn, then put back the idler without moving the carriage.

But since you're running the modified FW I'd like to know if the change did actually reduce the effect or not as it is first, since I hope to have improved a bit the handling of small mechanical friction increases which should be tolerated by the motor.

We definitely need SG temperature compensation for this. The motor is probably within the edge of the current thresholds, but as temperature increases it goes off.

Do you have the einsy inside the enclosure?

parhelia commented 3 years ago

I know you mentioned earlier on you tested that the axes move smoothly (without the belt). Did you also test this with the warmed-up printer? Fastest method to check is to let the printer in the enclosure exactly as it is now, and just unscrew the Y idler. I'd like to know if you can feel (even just minor) increases in friction in these spots, or if the carriage is fluid throughout even when hot.

As instructed, removed the idler and the carriage is still smooth even when the enclosure is hot. No binding whatsoever.

Another test I asked was to measure the temperature of the Y motor. Doesn't need to be precise, but if you heave a thermocouple it would be nice. Ideally I'd like to know if it feels hotter than the X motor.

Unfortunately I do not have a thermocouple. The Y motor doesn't feel any hotter or cooler than any of the other motors. The Z motors are always warm even when the entire enclosure and printer has cooled down, and "disable steppers" from the LCD panel has been selected.

If the carriage is absolutely smooth at that spot, then it's probably due to the rotor alignment to one phase. Another way to test this is by removing the idler, rotate the shaft of the Y motor 1/2 turn, then put back the idler without moving the carriage. But since you're running the modified FW I'd like to know if the change did actually reduce the effect or not as it is first, since I hope to have improved a bit the handling of small mechanical friction increases which should be tolerated by the motor. We definitely need SG temperature compensation for this. The motor is probably within the edge of the current thresholds, but as temperature increases it goes off. Do you have the einsy inside the enclosure?

So far it does seem correlated with heat. Haven't had enough printing time to tell if the number of crashes has reduced. Yes, the Einsy control board is within the enclosure as well.

wavexx commented 3 years ago

Ok.. I was just upgrading an MK3 from a ~1yr old FW to the current version for testing, and I started to get this as well, always on Y, despite using the same currents and everything else. This printer is in a temp-controlled enclosure and it just finished a print before starting a new one with the new FW failing already at the first layer.

There's definitely something odd here. Maybe I have a stab at finding the issue now.

adamdport commented 3 years ago

I'm starting to see this now too. I've got a lot of miles on my Y bearings but they feel fine with no binding. I've ordered replacements but it sounds like that hasn't helped people? If it's a temperature thing have people had success printing things on the back of the plate so that the bed never covers the motor?

wavexx commented 3 years ago

@adamdport did you notice when this started happening?

adamdport commented 3 years ago

maybe a few weeks ago? My latest crash, unlike OP, was on the very first layer while drawing the skirt. It was immediately following another PETG print though so the temperature theory is plausible. I have a handheld infrared thermometer and will try to record all my motor temperatures next time I see a crash.

wavexx commented 3 years ago

On Tue, Jun 29 2021, adamdport wrote:

maybe a few weeks ago? My latest crash, unlike OP, was on the very first layer while drawing the skirt.

Same for me, starts straight at the first layer. But in your case it didn't start suddently after a FW upgrade?

adamdport commented 3 years ago

Not that I noticed, no. Haven't upgraded in months. I'm on 3.9.3-3556

servant74 commented 3 years ago

Off topic: Pointer to directions on how to set up to compile Prusa firmware? Thanks.... direct message is ok to jack at coats dot org

On Tue, Jun 29, 2021 at 8:25 AM adamdport @.***> wrote:

Not that I noticed, no. Haven't upgraded in months. I'm on 3.9.3-3556

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/prusa3d/Prusa-Firmware/issues/2653#issuecomment-870598105, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAEUXEULLZBW67JY5AHBYDDTVHCVVANCNFSM4MYHMP7A .

-- ><> ... Jack

If you are not paying for something, you are not a consumer, you are the product. - Chamath Palihapitiya "Tell me and I forget. Teach me and I remember. Involve me and I learn." - Ben Franklin Win more eBay bids .. "http://www.ezsniper.com/refimg.php3?user=servant74"

prusa3d / Prusa-Firmware

[BUG] Ghost Y-Axis Crashes #2653