xblax / flashforge_ad5m_klipper_mod

Unoffical mod for Flashforge Adventurer 5M (Pro) 3D printers to run Moonraker, custom Klipper, Mainsail & Fluidd
GNU General Public License v3.0
190 stars 16 forks source link

MCU 'mcu' shutdown: Timer too close Adventurer 5M/5M pro #203

Open myxxx323 opened 6 months ago

myxxx323 commented 6 months ago

I started catching freezes on the Adventurer 5M pro and on the Adventurer 5M. Adventurer 5M Klipper Mod v00.05-beta installed. Hangs in different places, different parts. Slicers PrusaSlicer 2.7.4, OrcaSlicer 2.1.0 and Orca-Flashforge 1.0.2. ABS printing, temperatures: table 110, nozzle 270. I rebooted to stock and launched the same file, it froze with error E0011.

Начал ловить зависания на Adventurer 5M pro и на Adventurer 5M. Установлен Adventurer 5M Klipper Mod v00.05-beta. Зависает в разном месте, разные детали. Слайсеры PrusaSlicer 2.7.4, OrcaSlicer 2.1.0 и Orca-Flashforge 1.0.2 . Печать ABS, температуры : стол 110, сопло 270. Перезагрузился на сток и запустил тот же файл, зависло с ошибкой E0011.

2024-06-05-15-48-53-292 2024-06-03-10-06-40-009

klipper.zip

cmenard commented 6 months ago

Not sure if it's related, but I had similar issues with longer prints. Found that adding G92 E0 to the before layer change gcode has solved it. The profiles in ORCA were missing that. After adding it I haven't had any more errors or hangs while printing.

consp commented 5 months ago

The timeout issues are due to memory exhaustion, my guess would be that klipper runs out of memory due to something specific in the gcode file since it's both on original and modded versions.

What's the size of the gcode file?

myxxx323 commented 5 months ago

The timeout issues are due to memory exhaustion, my guess would be that klipper runs out of memory due to something specific in the gcode file since it's both on original and modded versions.

What's the size of the gcode file?

Various models, sizes from 10M to 50M

Разные модели, размер от 10М до 50М

thhdragon commented 5 months ago

make sure you have arcs disabled in orca

myxxx323 commented 5 months ago

make sure you have arcs disabled in orca

arcs disabled

jtenniswood commented 5 months ago

I had issues after adding the webcam, I turned down the frame rate and it's been solid ever since.

Probably not related but might help someone out.

consp commented 5 months ago

I turned down the frame rate and it's been solid ever since.

Since it looks like a memory issue I'm not surprized. Not running a gui will help as well.

More swap should sound like it might help but due to how the swap works it might actually not matter if you would increase the swap file size as it cannot keep up swapping out memory with respect to the requested amount.

Not really have a good solution here as to try to save as much as possible. Since it happens on the stock firmware as well it might be worth investigating if it's a klipper issue or something else but I've not seen it happen yet.

cyberpods commented 5 months ago

I found using Guppyscreen instead of Kilpperscreen helped my situation with this error, but it still happens once and a while.

consp commented 5 months ago

I found using Guppyscreen instead of Kilpperscreen helped my situation with this error, but it still happens once and a while.

Guppy uses about 24mb less memory than klipperscreen which causes this error to show up later. Using only the webinterfaces (no guid) will save you another 8-10mb (give or take a few mb's) which is a bit better than the flashforge OEM software.

Do you have a gcode file with which this happens (and are maybe willing to share), and do you know when it happens in the print (or before/after)?

If we can figure out where it starts hogging memory/cpu then we can figure out what the problem is and (maybe) fix it.

I usually print smaller parts and have not yet encoutered it other than during calibration which is a known issue.

harvinchan10 commented 5 months ago

Got this error as well.

I switched to no-GUI and so far so good. But haven’t really printed a big gcode yet since I switched.

not a solutiom but just wanna share my observations

the crash usually happens for me when I access Mainsail simultaneously on my computer + mobile phone. And it will surely crash if I open the webcam stream on another browser tab (accessing via http://192…:8080/?action=stream). Basically, if I stream the webcam on multiple devices simultaneously. it will most likely exhaust the RAM as mentioned here.

i noticed on the OEM firmware, they only allow one video stream. Say if I’m streaming the webcam on my computer. I wont be able to stream on another computer if I access the Local ip stream (http://192…:8080/?action=stream) I will just get a black screen or blank page if it’s already streaming elsewhere. It does work on the flashforge app though since it goes through their server maybe.

consp commented 5 months ago

i noticed on the OEM firmware, they only allow one video stream.

That might be a good idea to implement as well.

I'll have a look at monitoring the memory usage live during some prints.

Arnaud3767 commented 4 months ago

This has been happening to me since the orcaslicer 2.1 update too. New features may result in more gcode processing?

myxxx323 commented 4 months ago

I haven't used gyroid filling for a while now, and there are fewer errors.

Некоторое время не использую заполнение гироид, ошибок стало меньше.

jtenniswood commented 4 months ago

I switched to Guppyscreen and that fixed the memory issues. I wish they had spend a little more on the hardware (or that it was upgradable!)

myxxx323 commented 4 months ago

I switched to Guppyscreen and that fixed the memory issues. I wish they had spend a little more on the hardware (or that it was upgradable!)

There is also a problem on the drain. I monitored the memory on klipperscreen, at the time of the error it was not overloaded, it felt like the buffer was filling up.

Проблема есть и на стоке. Я наблюдал за памятью на klipperscreen, во время ошибки она не была перегружена, ощущение что заполняется буфер.

Arnaud3767 commented 4 months ago

Do you have to uninstall everything to install Guppyscreen?

jtenniswood commented 4 months ago

I just flashed it over the top, and worked perfectly out of the box. If you have made changes to the config files, you might want to grab a backup (as it will replace them). Otherwise just the same as.

consp commented 4 months ago

Do you have to uninstall everything to install Guppyscreen?

No, but do check if you made changes to the configs if they are still there (hint: always backup).

consp commented 4 months ago

it felt like the buffer was filling up.

What happens is the memory starts swapping out to flash to avoid memory saturation. The lilely problem (in my oppinion) is the flash cannot keep up and the kernel most likely will either hit an outstanding page limit or simply wait for pages to come available causing the memory access to have massive write delay penalties resulting in klipper timing out.

myxxx323 commented 4 months ago

Сегодня целый день печатаю, ни одного сбоя. Использую гироид, арки, все ок. Изменил в конфиге эти значения baud: 250000, было 230400 :

[mcu eboard] serial: /dev/ttyS1 baud: 250000 restart_method: command

[mcu] serial: /dev/ttyS5 baud: 250000 restart_method: command

Проверьте.

uart

consp commented 4 months ago

I don't understand Cyrillic so my guess is you think the baud rate mismatch is the problem. 8% is a bit much but not outside the realm of possibilities of most chips. On the other hand you cannot rule it out either way.

Is this the table for the NC clone chip or for the STM32?

myxxx323 commented 4 months ago

I don't understand Cyrillic

Воспользуйтесь переводчиком, весь мир так делает :)

KaruroChori commented 4 months ago

Воспользуйтесь переводчиком, весь мир так делает :)

Fair, but following this logic, most messages here would rapidly devolve in a mess of German, Italian, French, Mandarin, Russian and more.
You cannot expect each of us to need a multi-lingual translator just to understand what someone else wrote in this repo. It is perfectly fine if you want to attach the original message as you did at the beginning, so that there is a clear reference in case the machine translation is a bit off. But please, provide an English translation as well, or you are just making the experience for everyone else worse.

myxxx323 commented 4 months ago

Воспользуйтесь переводчиком, весь мир так делает :)

Fair, but following this logic, most messages here would rapidly devolve in a mess of German, Italian, French, Mandarin, Russian and more. You cannot expect each of us to need a multi-lingual translator just to understand what someone else wrote in this repo. It is perfectly fine if you want to attach the original message as you did at the beginning, so that there is a clear reference in case the machine translation is a bit off. But please, provide an English translation as well, or you are just making the experience for everyone else worse.

ok, I agree with you ок, согласен с Вами

consp commented 4 months ago

Languages aside:

Is this the table for the NC clone chip in the AD5M or for the original STM32?

The reason I asked is that sometimes the PLLs on clone chips are a bit different, as are the sampling characteristics of the peripherals. Might be worth a try though. I'll give it a try to see what happens.

IIRC the baud rate is set at the MCU level during compile time, so it's hardcoded in the firmware. Seems odd they chose 250k (the klipper default) for building the firmware and changed it to 230.4k in the configs.

AJolly commented 4 months ago

G92 E0 does not solve the issue for me (including on guppyscreen)
I do wish it was easier to just run klipper off an external box.

skylatian commented 4 months ago

bump, this also happens to me when hitting the "print again" button on klipperscreen or in the UI after a print finishes. not a huge issue overall, but definitely not ideal

consp commented 4 months ago

I do wish it was easier to just run klipper off an external box.

That might solve it yes.

bump, this also happens to me when hitting the "print again" button on klipperscreen or in the UI after a print finishes. not a huge issue overall, but definitely not ideal

Good to know, was this a relatively small print or a large one?

skylatian commented 4 months ago

it just happened again, both with small prints (one was 20mins, the other 5). This time, it happened when I went to print another file that was already uploaded to the printer. It seems to only happen when you try to start another print immediately after printing something else, I'm pretty sure that it hasn't happened when I go to change filaments or home the machine after a print, but I could be wrong. Happy to provide more extensive logs, but this is what shows in the console:

$ HYPERLAPSE ACTION=STOP
03:55:38 PM 
$ _SET_TIMELAPSE_SETUP ENABLE=True VERBOSE=False PARK_ENABLE=False PARK_POS=back_left CUSTOM_POS_X=10.0 CUSTOM_POS_Y=10.0 CUSTOM_POS_DZ=0.0 TRAVEL_SPEED=100 RETRACT_SPEED=15 EXTRUDE_SPEED=15 RETRACT_DISTANCE=1.0 EXTRUDE_DISTANCE=1.0 PARK_TIME=0.1 FW_RETRACT=False
03:55:27 PM 
// Klipper state: Disconnect
03:55:27 PM 
$ FIRMWARE_RESTART
03:54:46 PM 
!! MCU 'mcu' shutdown: Timer too close
03:54:46 PM 
// MCU 'mcu' shutdown: Timer too close
// This often indicates the host computer is overloaded. Check
// for other processes consuming excessive CPU time, high swap
// usage, disk errors, overheating, unstable voltage, or
// similar system problems on the host computer.
// Once the underlying issue is corrected, use the
// "FIRMWARE_RESTART" command to reset the firmware, reload the
// config, and restart the host software.
// Printer is shutdown
03:54:46 PM 
!! Homing failed due to printer shutdown
03:54:46 PM 
!! Homing failed due to printer shutdown
03:54:46 PM 
!! Homing failed due to printer shutdown
03:54:45 PM 
// Klipper state: Shutdown
draskostar commented 4 weeks ago

Is there anyone that can confirm this? I am on Guppy screen but Klipper screen is my favorite so I am thinking to give it a try.

https://www.reddit.com/r/FlashForge/comments/1cu1rnq/psa_disable_arc_fitting_in_orcaslicer_on_ad5m/

myxxx323 commented 4 weeks ago

Switched to stock firmware, removed the mod, after updating the firmware to 2.7.6 - not a single crash, printed about 5 kg of plastic. Printed for 20 minutes and for 6 hours.

Перешел на стоковую прошивку, удалил мод, после обновления прошивки на 2.7.6 - ни одного вылета, отпечатал порядка 5кг пластика. Печатал и на 20 минут и на 6 часов.

draskostar commented 3 weeks ago

Switched to stock firmware, removed the mod, after updating the firmware to 2.7.6 - not a single crash, printed about 5 kg of plastic. Printed for 20 minutes and for 6 hours.

Перешел на стоковую прошивку, удалил мод, после обновления прошивки на 2.7.6 - ни одного вылета, отпечатал порядка 5кг пластика. Печатал и на 20 минут и на 6 часов.

I can confirm that. You have all my respect. Thank you very much for sharing the solution.

Doker96 commented 1 week ago

I also encountered the same problem. It appears at any time during printing, at different stages. It doesn’t matter whether a camera that consumes enough RAM is connected or not. When switching to stock firmware 2.7.6, the printer prints without any problems.

Berkano24 commented 3 days ago

Same on my side.. it even appeared after printing has been finished.

xblax commented 3 days ago

If think the most important thing, in order even try get to the bottom of this issue, would be if someone has a G-code that does very reliably trigger the issue, without needing a long print.

My guess, why it happens with the mod but non on stock, is that it's somehow related to Moonraker putting additional load onto Klipper (Stock has no Moonraker) in combination with the shortage of memory.

If this issue was easily reproducible, one could at least try to understand the root cause and see how it can be mitigated.

Berkano24 commented 12 hours ago

I see it now on a nearly daily base, even sometimes after some short prints (don't believe it has something to do with the memory). myxxx323 already mentioned that it might have something to do with the baud rate. Did somone follow this approach in the meantime?