[Feature Request] Macros stored as files

[fugalster]

Due to Marlin's lack of M98 M99 support (subprogram start/end), I'm trying to do a workaround using macros. Your plugin is almost exactly what I need, except the length of my macro can't be input through the UI, and manually stuffing it in the config.yaml file seems... abusive.

Could there be a way to define macros as the content of another file? Or better still, can the macro just be a call to a separate gcode file (as uploaded through the octoprint sidebar)?

[cp2004]

How large is the macro that you are trying to create? When I created the plugin, I was intending it to be small gcode snippets - I never tested a maximum limit, so I'm not sure what that would be. Can you explain a bit more about your use case? I like how simple the code is at the moment, but I am open to extending the plugin if it is not too complex 🙂

[fugalster]

I print a lot of a particular part, and it's small so I can fit many on my print bed. But I don't want to print all at once (a.k.a. normally) because if there's a problem in the job I loose an entire bed worth of parts. If I print one at a time then I only ever loose a single part. This also gives me flexibility of canceling the print part way through and still having good parts.

My current work flow is to slice the part in many locations, then manually splice the gcode together including some manual moves in-between parts. This works, but is hard to maintain and prevents me from making optimizations to the part and printer settings because the time investment in splicing code is high.

If I could slice the part once and store it as a separate macro file, then have the "main" file with the header and footer gcode along with movements to the different parts of the bed, setting machine offsets, and calling the macro file, the workflow would be dramatically improved. I could set up the main file once, then change out the macro file as needed.

My specific "macro" code is on the order of 100k lines long. Perhaps at this point "macro" is the wrong word to describe it. It's more like a subroutine (which the aforementioned-but-absent-from-Marlin M98 and M99 codes are supposed to do). But conceptually it's the same thing.

[cp2004]

That's an interesting use case there for it.

I do have some idea, since these macros support Jinja2 template rendering (for quite basic variables/logic), whether you would be able to 'upload' (or somehow transfer) a template, which you could include in a macro as {% include 'mytemplate' %}. In theory this would not take that long to create, rather than having a whole file handling system let Jinja do it for me. I just need to configure the template loader to tell it where to look on the Pi, that you could add files to.

I will keep it in mind, but I have a huge amount to do at the moment so there won't be any progress for some time.

[CmdrCody51]

Just here to +1 this request. Marlin 'used' to have macros but if you called the SD card macro - OctoPrint ignored the 'SD not printing' event and happily just sat there. This would resolve this problem by feeding the gcode from OctoPrint. My +1 request is to facilitate tool change wipe towers. It would be easy using G60/G61 slots and file-stored macros.

[napter]

I was thinking about this for a different use case: I would like to use my Macros across many printers - so if they were stored in some central repository that would make them easier to maintain.

[cp2004]

In 1.2.0 (just released) there is the option to include external files in macros using the Jinja2 syntax {% include "other_file.gcode" %}.

This allows you to store macros as files, although you will have to manually manage this for now. It looks for files in the plugin's data folder, so ~/.octoprint/data/gcode_macro/ on a Linux/OctoPi type install. I created a quick documentation page, linked from the plugin to suggest it for longer macros. That should avoid the issues with macros being hard to manage in config.yaml or not loading on the settings API.

I am actively looking into a solution for possibly storing the macros themselves as files, but this does make the code much more complicated than is currently (I like the simplicity in a way). I don't like storing large text/gcode scripts in the config.yaml file, but it makes life much much easier. Dealing with the command, description and content is harder in a file so a hybrid system may be needed for the extra metadata.

Please test 1.2.0 and if you have any questions or problems please ask, I will answer!

[CmdrCody51]

Notes on first pass\: 1) Both "documentation links" in the editor takes the browser to the page rather than opening a new tab. 2) Need 'examples' of Jinga2 stuff. I figure it's the same as the OctoPrint GCode Scripts but then the only 'variables' I can find is 'pause_position' from the OcroPrint docs.

[cp2004]

I will fix the documentation links now, and then whenever I need to create a new release that change will come out then.

With the Jinja2 stuff - I have some basic examples on the README. There is not an awful lot you can do with them at the moment, since there are no extra variables injected. I looked into the feasibility of this (it was requested in #7) but decided I couldn't replicate the context in the same way.

[fugalster]

Thanks for adding the feature request!

There does seem to be a bug though. Previously I had a macro for running my bed leveling routine. This macro included a call to @BEDLEVELVISUALIZER for the Bed Visualizer plugin and it worked fine. But with this new version if I call @BEDLEVELVISUALIZER from within the macro and run the macro (e.g. from the terminal window), it pauses the terminal updates for a moment and then resumes. No commands can be seen to be sent though the terminal window like when any other macro is called.

Here's the macro in question, and to reiterate @BEDLEVELVISUALIZER is not a macro I've written, it's a call to the Bed Visualizer plugin.

M104 S175 ; Wait for nozzle temp
G1 E-2 F150 ; Retract some filament to prevent oozing
M190 S60 ; Wait for bed temp to rise somewhat

; Bed Leveling (delete section if not wanted)
M155 S30 ; reduce temperature polling
@BEDLEVELVISUALIZER ; Tell octoprint to watch for incoming mesh
G29 T ; Auto Level Bed
M155 S3 ; Reset temperature polling
G0 E0 F5000 ; Return filament to nominal position

[CmdrCody51]

Do you mean none of the gcode shows up in the terminal window? Did you check octoprint.log for errors? It took me awhile but I even got macros running with 'variables' (but not parameters) I do get a few "OctoPrint refusing to send blank lines." errors. But I could see it refusing to parse what looks like a macro that it can't find. I do chain a couple tho.... but I use the 'include' or 'import'.

[fugalster]

Q: Do you mean none of the gcode shows up in the terminal window? A: Correct, none of the gcode shows up, not even the "non-offending" code. Q: Did you check octoprint.log for errors? A: octoprint.log doesn't have any output that might relate to this issue.

I don't run any variables or parameters, so nothing there. I did some checks just now to make sure blank lines weren't causing any problems. Other macros with blank lines are running fine. I have been using and continue to use nested macros. That functionality still works.

In the macro I posted above, if I comment out the line @BEDLEVELVISUALIZER then the macro works (though I don't get the feedback into the other plugin). This used to work with GCODE Macros, but doesn't any more. If I manually send all the lines above (including @BEDLEVELVISUALIZER) everything works as expected.

[CmdrCody51]

What I'm thinking is that the macro errors out in the parsing stage. Therefore no code is output at all. In fact I just tried: @AtCall set to: G28 @BEDLEVELVISUALIZER G28

Received No Output octoprint.log has: 2022-08-08 00:43:43,229 - octoprint.util.comm - ERROR - Error while processing hook gcode_macro for phase queuing: Traceback (most recent call last): File "/home/pi/OctoPrint/venv/lib/python3.9/site-packages/octoprint/util/comm.py", line 4663, in _process_command_phase hook_results = hook( File "/home/pi/OctoPrint/venv/lib/python3.9/site-packages/octoprint/util/init.py", line 1688, in wrapper return f(*args, **kwargs) File "/home/pi/OctoPrint/venv/lib/python3.9/site-packages/octoprint_gcode_macro/init.py", line 79, in gcode_queueing return self.render_macro(command) File "/home/pi/OctoPrint/venv/lib/python3.9/site-packages/octoprint_gcode_macro/init.py", line 112, in render_macro result += self.render_macro(cmd, level=level + 1) TypeError: 'NoneType' object is not iterable

So there is an error logged. And It looks like jneiliii put in patch 28 days ago that may or may not have been included.... Hey Charley

[cp2004]

🐛

[cp2004]

Should be fixed in 1.2.1

[CmdrCody51]

Didn't blow up. I use MBL so I wasn't going to thru all that. But I got my two G28'S.

[fugalster]

Should be fixed in 1.2.1

Confirmed that the issue is resolved. Thanks!

[cp2004]

I'll try not to break things again 🙂

[cp2004]

With version 1.3.0, macros are fully stored as files, with just the command name & description stored in config.yaml.

The Jinja2 sub-template support is of course still there, as you might still experience performance issues trying to load thousands of lines into the UI.