[Feature Request] Power Resume from SD Print

[houseofbugs]

As you all know the Creality CR-10S machines have implemented a power resume feature that is working for SD prints without any additional hardware. They are using a RAMPS type board that is custom for their printers but the pinouts are pretty much the same (no SD detect pin used though).

The goal here is to start a discussion as to how this can be implemented and achieved.

The ATMEL chips have a built in voltage meter of sorts. I suspect that the way this was achieved is by monitoring for a voltage drop going to the chip and then saving the current GCODE position somewhere (EEPROM or on the SD itself). On power up you will have a menu that you can resume which will read the stored value that will correspond with the GCODE file that was being printed and the position in that file.

Info about the Arduino meter: https://provideyourown.com/2012/secret-arduino-voltmeter-measure-battery-voltage/

Or maybe the simpler option is have the board continually write the progress location as it goes while it is printing? Thought this would create strain on the SD or EEPROM with all the writes/re-writes.

I could be off base here but I plan on messing with this in the IDE in the next few weeks. Curious to see what others have to say about this.

Also there is another machine that popped up that has power loss resume. They provided the source on the SD card. Someone sent it to me so maybe this would be of help as well. I am looking into this code to see if there is anything useful for the power resume. I am also working on getting the machine here so I can see if they have additional hardware in it.

Original ZIP from them: https://1drv.ms/u/s!Ap5qRe33fRYtjLs88Dx1M2Rsglp6ZQ Posted to my Git: https://github.com/houseofbugs/TH3D-Marlin/tree/alfawiseu10-powerresume

[mylife4aiurr]

Hey Tim. Would be great to have this feature built into Marlin. Prusa Mk3 has this feature as well. Aren't they open source, is it possible to try and get the resume function/procedure from them.

[anton-piliugin]

We already have an issue thread on this topic: https://github.com/MarlinFirmware/Marlin/issues/2085

[Roxy-3D]

The big problem is a large print can make millions of nozzle moves. But any EEPROM storage can only tolerate 100,000 writes. If we knew we had 1/2 second of power, we could tidy things up and save where we left off. If we had a way to detect power failure 1/2 second before things really failed... It would be straight forward to solve this in a clean way.

[marcio-ao]

@mylife4aiurr : The Prusa Mk3 has a custom PCB for monitoring the high-voltage side of the power supply. When a dip in the AC voltage is detected, it sends a signal to the board to begin the process of saving state. This consists of shutting down all heaters (to conserve power), writing the position to EEPROM and using whatever juice is left in the power supply caps to drive the head up 1mm or so away from the print.

There's a lot going on in their implementation and it cannot be replicated without the custom hardware.

There are other closed threads regarding this #9819, #8042, #2085, #6028 and even one I created myself and cannot now locate.

[marcio-ao]

@Roxy-3D: Saving state is not even the biggest problem. I had suggested a method whereby the printer would simply record to the EEPROM that a print was in progress. In the case of a power failure, the printer would measure the distance needed to home X, Y, Z and this would be a rough estimate of coordinate where the print was interrupted. It could then attempt to resume. But the big issue that people pointed out to me is that the nozzle would leave a big blob on the print and it is hard to recover from that. Prusa's aggressive power monitor allows them to raise the nozzle by a small distance and this is enough to make all the difference in the world.

So, I think the answer is that we could probably make a poor software only recover feature, but then it wouldn't work very well and people would complain :)

[Roxy-3D]

But the big issue that people pointed out to me is that the nozzle would leave a big blob on the print and it is hard to recover from that. Prusa's aggressive power monitor allows them to raise the nozzle by a small distance and this is enough to make all the difference in the world.

Actually... I don't think this one is that bad... If we have a 1/2 second of power, we can raise the nozzle away from the print. And for that matter... Retract the filament 1mm or 2mm. The problem I think that will be very hard to handle is re-homing the machine with a print in the middle of the bed.

[Grogyan]

I have been working on an implementation using the available functions for using an SD card. Progress is slow, as I am not anywhere near as. proficient at coding as I would like. cardreader.cpp has all the required functions ready to go, just requires a bit of coding knowledge.

[Grogyan]

Regardless of using the EEPROM or SD card, the main thing to realise is that the print must be paused, this allows you to use the getStatus() to get the last byte, getFilename() of the current print, get the current heatbed and extruder(s) temperature, and storing them in EEPROM or SD card. Restoring them all when resume() is called is the next challenge, which I have not even got to. Again, most of the functions are already available in cardreader.cpp. Ultralcd.cpp will need a resume print entry, but that is a lot easier than the rest.

I am using the M25 in Marlin_main.cpp for debugging, because, once it is nailed in, then it can be applied to the kill() and powerpanic() functions too.

[p3p]

Even if you implement a controlled emergency pause (a nice feature even without a power failure), which would have to discard as much of the planner buffer as possible while still decelerating within the hardware constraints? how do you know the last actually (partially) executed line of gcode? not the last parsed.

[Grogyan]

A partially executed line of gcode is the worst thing to figure out. And is way over my head. Until Creality publishes their source code, I will hope that that someone here can implement the feature.

Until then I will plod along, with this plan, even if it isn't that good of a plan.

[Roxy-3D]

how do you know the last actually (partially) executed line of gcode?

If I was writing this code... I think I would abort the currently executing line of GCode. And I would restart it later. If it was a print line where filament is being extruded, that isn't ideal, but probably that small amount of filament will just get squished into the print. If it is an outer perimeter, very possibly, the user can cut the extra off with a razor blade when the print is done.

[Grogyan]

2085 I see that Josef has released the firmware for the MK3

[Grogyan]

After reviewing the code Prusa supplied, it appears I was too naive that I could even code this. I will leave this for someone more capable at coding to have a crack at it.

[fiveangle]

[insert we have this thread every week meme here]

It's not helping to continue to conflate the MK3 custom-hardware implimentation with something that can generally support average marlin based printers. Having their code is of no use without the custom hardware contained within their power supply module.

Of all the very active discussions, issue #9185 linked to most of the other threads, but also died with the same reasonable conclusion: not possible w/o custom HW.

In https://github.com/MarlinFirmware/Marlin/issues/9185#issuecomment-358090878 I pointed out that power-out pause/resume support could be added relatively easily right now for anyone with an external UPS and USB monitoring host such Octoprint running on a RaspberyPI. If using Octoprint to manage printing (not from SD), Oprint would monitor the UPS daemon and when notified that shore power was out, use custom command feature (in the yaml config) of Oprint to trigger pause within Octoprint. By adding ATX power-control (or RPi relay control hat, both easily implimented) Oprint could then power-off printer and remain on running on UPS for a long time since RPi takes so little power. This automated solution is possible today to someone with the initiative to put together all the pieces.

My next suggestion for a more Marlin-centric solution was to augment Marlin M25 pause command to support saving/resuming M25 pause data to/from SD: https://github.com/MarlinFirmware/Marlin/issues/9185#issuecomment-358096170

So either user could notice power was out (printer still running on UPS) and manually initiate pause via LCD or terminal host via M25, saving pause location to SD card (likely with new paramter to M25 command). Then user could power printer off. Once shore power was back, user would power it all on, ensure conditions were okay to atempt resume, and issue the yet-to-be-implimented M25 resume-from-saved-SD-data command and printer would try to pick back up where it left off. But nobody took my suggestion to submit a request focused on just this missing M25 piece and I've not seen anyone discussing it since.

All the other suggestions across the many threads seem more just users interested in the feature but not really understanding what's required to implement it and what the limitations are (and continuing to conflate the auto-power-cut MK3 implimentation with something generically possible w/o custom HW).

I personally don't see any value in this for me, since power here in San Francisco has been solid since the rolling PG&E black outs of 10+ years ago ;) but if I had power going out on me all the time, it would not be difficult to craft the Octoprint solution immediately, and at least draft a well-thought-out FR for the pause/resume-from-SD feature I suggested. If clarified well in a FR someone with the interest and skills will likely eventually come along and shoudn't have too much difficulty adding the feature to to the M25 routine of marlin.

But doing this without any additional HW in the form of either custom monitoring HW or a full on UPS seems somewhat of a pipe dream as of technology available today :man_shrugging:

-=dave

[Grogyan]

The additional hardware is available, ai have already got a mains loss detector coming from China that is pretty cheap, if just a little concerning with a certain connector from [AliExpress] (https://m.aliexpress.com/item/32818398064.html?pid=808_0000_0109&spm=a2g0n.search-amp.list.32818398064) used. Which is just then tied to a interrupt pin. A couple of resistors via a resistor divider and a beefy capacitor for the Arduino power are just as simple. @fiveangle I can also suggest that the feature can also be very useful for long prints, that may not be printed in a single go.

[thinkyhead]

Duplicate of #2085

[thinkyhead]

we could probably make a poor software only recover feature

The power loss recovery feature on the Creality CR-10S uses no extra hardware, and it is reportedly a bit unreliable.

[mylife4aiurr]

Now that the Creality's implementation has been released (thanks Naomi Wu), im hoping marlin dev's can make it more reliable and add it to marlin officially.

[anton-piliugin]

Nothing special in the source code, they don't even measure the voltage. They just saving the current XYZ position, temperature, speed, etc. into a file on SD card CONSTANTLY! This can wear out the card very fast in my opinion. What do you guys think?

[thinkyhead]

@anton-piliugin — Yes, the contractor that Creality hired seems to have pulled a fast one. That is exactly how they implemented it, and it is an SD card killer. If you own a CR-10S and are using the firmware they supply with it, your SD card will very quickly die. (The card won't be written as often as the data is being changed, but it will be written as often as it synchronizes.)

[anton-piliugin]

I'm looking at a ZMPT101B AC voltage sensor module now, it needs 3 pins on board to connect it: VCC, GND, OUT(Analog signal). It may become a solution for power resume function

[thinkyhead]

I've been reading up on how microSD cards work and their expected lifespan. Assuming that an SD card is large, and assuming that the file is small, and assuming that the card smartly cycles the data around so it isn't writing to the same cels continually (which only some do automagically), the continuous-writing method might not kill an SD card until after many months of continuous printing. But… it's still not ideal.

I'm looking at a ZMPT101B AC voltage sensor module now

I'll also do some tests on the code samples I have which purportedly can detect power-loss on 2560 and some other MCUs with no added hardware to see how reliable they are.

[p3p]

and assuming that the card smartly cycles the data around so it isn't writing to the same cels continually (which they do automagically)

We can't guarantee that as far as I remember, wear levelling is not part of the spec and upto the manufacturer. This may have changed with more recent standards? Old data keeps turning up in the eeprom.dat files which I didn't think would happen on delete/recreate cycles.

[thinkyhead]

Ok! I've created a placeholder account for Creality3D and posted all the changes that Creality made for the CR-10S firmware with one commit per added feature: https://github.com/ShenzhenChuangxiang3D/Marlin/commits/CR-10S

[davidlevy247]

@thinkyhead Would you be willing to add some info/docs or comment section to your fork, I was looking to add the Naomi CR-10s power resume stock firmware link for others that wanted to check out the original firmware but there's no docs or anything going on for the fork page. Nice work btw! Thank you

[thinkyhead]

@LevyImage — Once 1.1.9 is released we'll finish up the documentation. And POWER_LOSS_RECOVERY still needs some work before it will be ready for prime-time.

[hussainsail2002]

Hi,

I have a few machines with me where I would like to implement this feature, is there any documentation available that would help me set this up. Any hardware that needs to added along with the ramps board ?. I am a mechanical Engineer and have limited exposure to the programming side. Any documentation would be helpful.

Thanks. Hussain

[thinkyhead]

@hussainsail2002

The POWER_LOSS_RECOVERY feature uses the SD card to save the machine state once per layer. No extra hardware is needed, but a detection circuit may be added to avoid writing to the SD card on every layer. It simply needs to be attached to a free digital pin, then set POWER_LOSS_PIN to that pin.

POWER_LOSS_RECOVERY only works when printing from the SD card and requires an LCD controller. Some hosts, notably OctoPrint, may have their own resume from power-loss feature that doesn't require POWER_LOSS_RECOVERY to be enabled in Marlin.

/**
 * Continue after Power-Loss (Creality3D)
 *
 * Store the current state to the SD Card at the start of each layer
 * during SD printing. If the recovery file is found at boot time, present
 * an option on the LCD screen to continue the print from the last-known
 * point in the file.
 */
//#define POWER_LOSS_RECOVERY
#if ENABLED(POWER_LOSS_RECOVERY)
  //#define POWER_LOSS_PIN   44     // Pin to detect power loss
  //#define POWER_LOSS_STATE HIGH   // State of pin indicating power loss
#endif

[hussainsail2002]

@thinkyhead Thank you !! Will try this out and let you know.

[hussainsail2002]

@thinkyhead I have been testing the power failure recovery for a while now.

The issue that I have been having is that the Z axis drops down by a few mm after the X and Y have been homed while resuming the print.

This seems to be happening alternately the first power loss resumes correctly, the second power loss the bed moves down and does not move back up again. I kill the print after the second power loss.

Is there some way to make sure that the z axis just maintains its position after the power is turned back on ?

Thanks, Hussain

[eried]

Hey @hussainsail2002 did you wired anything extra? I want to enable power loss recovery too but writing to the sd card after each layer seems excessive

[thinkyhead]

@hussainsail2002 — You are seeing the results of bugs in the implementation. There's no way around the problems you are seeing until the feature is fixed. It's on the TODO list.

[hussainsail2002]

@eried I have not wired anything yet, will do once am able to test it successfully.

@thinkyhead Do you think I can add a z axis max optical limit switch that will be used only for power recovery. I know a few programmers who help me out. Could you point me in the direction where I can modify the code and test it ?

Thanks ! Hussain.

[thinkyhead]

Do you think I can add a z axis max optical limit switch that will be used only for power recovery.

Anything is possible with open source! But we don't support anything quite like that at this time.

[thinkyhead]

Could you point me in the direction where I can modify the code and test it?

It's not a trivial task. To point out all the changes you'd need to make would be taxing.

[hussainsail2002]

@thinkyhead

I totally understand. Is there any way I can contribute to this feature by testing it out ?. There are few more bugs that I have come across while testing this feature.

[thinkyhead]

Since this feature will be getting a minor overhaul, we'll be fixing the existing bugs and creating some all-new ones.

[github-actions[bot]]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.