Custom GCode commands

[Protoneer]

Please describe the feature you would like implemented I am trying to setup a pick and place machine. But it requires special machine specific gCode commands. Example : M115

This is used by openPNP to detect the machine type and configure the app.

Why do you think this would improve Grbl_ESP32? Grbl_ESP32 is already very configurable with regards to pin assignment. Being able to extend vanilla GRBL with custom command would make it super extendable. This could also be used in place of macro's, right?

Will this feature appear to a lot of users? This is definitely aimed at custom machines

[bdring]

We use gcodes as defined by LinuxCNC wherever we can. This is based off the NIST standard and is well thought out with defined modal groups.

Marlin's list is a hot mess and appears that anyone can add a command without following any standard. I don't want to add commands just because they are important to one person (not talking about you here).

It is possible we could create an easy way for people to add G or M codes for their own use. Grbl_ESP32 would parse the gcode and see a command it does not know. It would then pass that to to a custom file to possibly process it.

[Protoneer]

We use gcodes as defined by LinuxCNC wherever we can. This is based off the NIST standard and is well thought out with defined modal groups.

Marlin's list is a hot mess and appears that anyone can add a command without following any standard. I don't want to add commands just because they are important to one person (not talking about you here).

I totally agree with this. This is actually why people prefer GRBL over other implementations.

I also don't want that to change.

[Protoneer]

It is possible we could create an easy way for people to add G or M codes for their own use. Grbl_ESP32 would parse the gcode and see a command it does not know. It would then pass that to to a custom file to possibly process it.

That is exactly what I would like to do , like the custom_code_template.cpp it would hook onto the end of the gcode processor.

In my case I would like to control between 5-100 solenoids / actuators . It would be the same as the M62 command but with way more IO.

[joedirium]

I could also like to use the chance to add custom G-Code, as I would like to add codes to initiate http requests. Having a structured way to add such with extra files without modifying the original code would be great. Having WiFi in ESP32 is then another a huge advantage over Arduino.

[Protoneer]

I could also like to use the chance to add custom G-Code, as I would like to add codes to initiate http requests. Having a structured way to add such with extra files without modifying the original code would be great. Having WiFi in ESP32 is then another a huge advantage over Arduino.

Interesting.... What kind of http requests would that be?

Totally agree that this needs to be on a separate layer keeping the the original code in place.

[joedirium]

Well, I would like to add Tasmota based equipment (community firmware for ESP8266 based hardware) to be controlled by grbl. Simple example: turn on/off the spindle using a cheap Tasmota power plug. That would keep high voltages completely away from my setup. I would prefer this over the use of relays. https://github.com/arendst/Tasmota A "job finished" notification on my mobile would also be nice. But there is certainly more great potential for the use of http calls from G-Code.

[MitchBradley]

One of-used approach to extensions that do not fit well within an established language framework is "smart comments", i.e. wrapping the new syntax inside comments. In the GRBL world, though, that suffers from the fact that some senders rip out comments and do not send them to GRBL.

Another solution that probably works better is to invent an M code. There is much GCode precedent for machine/controller specific M codes; indeed, one could argue that they were intended for that purpose.

But there are still some problems:

1. Adding M codes is what Marlin does, and there is great resistance in the GRBL community against doing anything that reeks of Marlin
2. Whatever one does has to fit within GRBL line length limitations - so URLs for http: requests would need to be short-ish

[bdring]

@MitchBradley is right. Justifying a change "because Marlin does it" does not help in the Grbl community.

That means a generic method to add commands is a better approach.

I have some basic code working.

How should Grbl_ESP32 respond to M115?

Should the EXTENDED_CAPABILITIES_REPORT be supported and how should that look?

Cap:EEPROM:1 Cap:AUTOREPORT_TEMP:1 Cap:PROGRESS:0 Cap:AUTOLEVEL:1 Cap:Z_PROBE:1 Cap:SOFTWARE_POWER:0 Cap:TOGGLE_LIGHTS:0 Cap:EMERGENCY_PARSER:1

[Protoneer]

@bdring , I have a quote from the openPNP forum below with more background on it...

At the minimum M115 should rerurn FIRMWARE_NAME: XXXX and FIRMWARE_VERSION: X.XXX

FIRMWARE_NAME:GRBL_ESP32 FIRMWARE_VERSION:1.3 [20210101]

https://groups.google.com/g/openpnp/c/VzSW0rScx2c/m/T5fshkY-BQAJ?pli=1

There is no real standard (as far as I know), just conventions. There is this https://reprap.org/wiki/M115_Keywords

but it seems nobody really adheres to it or maintains it.

These are known responses: FIRMWARE_NAME:Smoothieware, FIRMWARE_URL:http%3A//smoothieware.org, X-SOURCE_CODE_URL:https%3A//github.com/markmaker/Smoothieware/tree/feature/best-for-pnp, FIRMWARE_VERSION:feature/best-for-pnp-c82d0b45, X-FIRMWARE_BUILD_DATE:Oct 22 2020 21:21:58, X-SYSTEM_CLOCK:120MHz, X-AXES:5, X-PAXES:5, X-GRBL_MODE:0, X-CNC:0, X-MSD:1

FIRMWARE_NAME: RepRapFirmware for Duet 3 MB6HC FIRMWARE_VERSION: 3.2-RC1+1 ELECTRONICS: Duet 3 MB6HC v1.01 or later FIRMWARE_DATE: 2020-12-23

FIRMWARE_NAME:TinyG, FIRMWARE_URL:https%3A//github.com/synthetos/TinyG, FIRMWARE_VERSION:0.97, FIRMWARE_BUILD:440.21, HARDWARE_PLATFORM:1.00, HARDWARE_VERSION:8.00

Note: as the ":" is used as a delimiter, you can't use it inside values. Therefore encode those as "%3A".

I guess you get the idea. There are essential properties such as FIRMWARE_NAME and FIRMWARE_VERSION that Issues & Solutions uses. Other than that, you're pretty free. If there are important compile time defines or configuration options, it would be cool to communicate these through custom properties (tell me how to interpret). As an example Issues & Solutions interprets the X-AXES and X-PAXES properties on Smoothie as explained here, plus X-GRBL_MODE, because it affects the homing command.

Don't forget to implement M114 too ;-)

Can you tell us more about your Grbl port? _Mark

[Protoneer]

@MitchBradley is right. Justifying a change "because Marlin does it" does not help in the Grbl community.

That means a generic method to add commands is a better approach.

Definitely agree on a generic method. My thoughts would be that it would get defined/linked or implemented in the machine cpp.

That way each machine can add extra bits to it like the SMOOTHIE example above.

[Protoneer]

Should the EXTENDED_CAPABILITIES_REPORT be supported and how should that look?

Cap:EEPROM:1 Cap:AUTOREPORT_TEMP:1 Cap:PROGRESS:0 Cap:AUTOLEVEL:1 Cap:Z_PROBE:1 Cap:SOFTWARE_POWER:0 Cap:TOGGLE_LIGHTS:0 Cap:EMERGENCY_PARSER:1

For openPNP I don't think its needed for a basic setup

[bdring]

It seems to work ok. M115 is a pretty simple command. You would have to know Grbl very well to add complicated ones. There are probably limitations too. User beware.

Grbl 1.3a ['$' for help]
m115
FIRMWARE_NAME:Grbl_ESP32, FIRMWARE_VERSION:1.3a, FIRMWARE_BUILD:20210308
ok

I have it sending the response to all clients. This is similar to the probe command. There are not too many gcodes that send a response other than "OK"

It is on a branch from devt called UserDefinedGcode

[Protoneer]

Amazing stuff!!! Thanks for that!!!

[joedirium]

Suggestion: a framework to add G-Codes should not only allow adding some functionality, but should also allow putting grbl into certain states, as for example to put the machine into alarm mode. In my example: http request to switch a power plug running the spindle "on" should result in alarm mode if this http request failed.

[Protoneer]

@bdring ...

I am trying to compile the test branch(UserDefineGcode) but it says it can't find the file.

Alternatives for Custom/user_gcode_example.cpp: []C:\Users\IT\AppData\Local\Temp\arduino_build_368994\sketch\src\CustomCode.cpp:7:34: fatal error: Custom/user_gcode_example.cpp: No such file or directory

ResolveLibrary(Custom/user_gcode_example.cpp)

When I comment out...

define CUSTOM_CODE_FILENAME "Custom/user_gcode_example.cpp"

in the test_drive.h it compiles again.

[Protoneer]

Tried to compile the Simple_OLED branch with the same results but just for the oled_basic.cpp file.

Could this be a Windows path issue?

[bdring]

At this time you must use PlatformIO to compile when the custom folder is used. The custom folder is not under the src folder to prevent all files in the folder from automatically being compile. That could cause problems, especially if some of the files contain the same funnction names, such as machine_init().

[bdring]

@joedirium There are better ways of handling alternative I/O. Similar to the I2S outputs, you could create a new class of I/O and then any feature could use it without changing existing code. It would add another output option like these.

#define SPINDLE_OUTPUT_PIN   GPIO_13 
// or
#define SPINDLE_OUTPUT_PIN   I2SO(12)
// or
#define SPINDLE_OUTPUT_PIN   Tasmota(3)

With that said, I don't know anything about those devices, I don't have any. It is not something I am interested in working on myself.

[joedirium]

@bdring Thank you for your suggestion! Sounds very promising to be able to use existing G-Code. So I tried to see if I can do this, but obviously my C++ skills are by far not enough. Coming from the world of Java I expected to find an interface or abstract class for I/O to implement, but couldn't find such. I am getting lost in all these #define and #ifdef For I2SO I expected to find a class, but actually found this

define I2SO(n) (I2S_OUT_PIN_BASE + n)

which looks like a constant number with offset and not like an object to me. C++ is really confusing me. My respect to all who can handle this in complex projects like this one!

[MitchBradley]

GRBL was originally written in plain C to run on a small AVR processor with limited memory, with #defines all over the place to save memory. Grbl_Esp32 was then converted to C++ so it could be compiled by the Arduino IDE, but the conversion was mostly "C dressed up as C++". We are in the process of changing parts of it into proper C++ to use abstract classes, but it takes time. The settings framework, spindle support, and many aspects of motors now use classes, but lots of the core is still in (dressed up) plain C. There is branch NewPinClass that brings IO pins into the C++ class fold, but it has not yet been deployed. It is very high on our list of development priorities. But frankly, "small" feature requests like this one keep distracting us...

[MitchBradley]

Further thoughts about the Tasmota situation:

1. Whatever we come up with, the name should not be "Tasmota". The name should reflect the network protocol, not the particular firmware that implements it. Tasmota implements multiple communications protocols, including MQTT, HTTP, and serial. There are other firmware solutions besides Tasmota that implement these protocols. So calling the interface "Tasmota" is simultaneously too specific and not specific enough.
2. The protocol most often used with Tasmota is MQTT. So if we assume that MQTT is the transport for spindle pin changes, then the name would be some variant of "MQTT". But there is a lot of configuration involved, including such issues as how to specify the MQTT broker address, what are the names of the MQTT subscribe variables, and what is the class of service.
3. Alternatively, one could use Tasmota's HTTP (REST-style) interface - but that too has similar configuration requirements.
4. The abstract interface to the new Pin Class framework will be capable of expressing a simple pin change via such an interface, but the performance could be problematic, because what appears to be a simple "pin change" actually involves a deep dive into the network stack. Add in the fact that pin changes - including those that control spindles - often need to be done from interrupt service routines - where you cannot call into the network stack - and you end up with a can of worms of the first order. Conclusion: This is WAY more involved than it would seem to be on the surface.

[Protoneer]

At this time you must use PlatformIO to compile when the custom folder is used. The custom folder is not under the src folder to prevent all files in the folder from automatically being compile. That could cause problems, especially if some of the files contain the same funnction names, such as machine_init().

@bdring .... haha... I was looking at the code in VC and compiling it in Arduino.

I have switched to PlatformIO and its much easier to work with.

I have tested with openPNP and it is accepting the the M115 results!!!! 🥇

[Protoneer]

Conclusion: This is WAY more involved than it would seem to be on the surface.

@MitchBradley you are describing the universe... :wink:

[bdring]

@Protoneer

You also have to "clean" before each compile. The automatic clean does handle the custom folder.

[joedirium]

@MitchBradley Your are right. Please see Tasmota only as an example. It has a huge community, but some generic way to do HTTP requests would cover this anyway. I assume that a HTTP request library is already linked. Not sure if the parser can read an URL as a parameter of a G-Code. So a custom G-Code with URL support is maybe the better way. (example: Q99 http://test/power=on pauseonerror=1). Configuration would then not be required. I know it is always a time consuming task to manage configurations, build the UI, make values persistent, validate... Doing replacements in existing G-Code to make them compatible to proprietary commands is nasty, but simple and can be automated.

MQTT could make sense as well, but I would not try to find a solution which covers both in one. If HTTP is solved, it should be easy to add MQTT in a similar way adding a MQTT library and do some changes. MQTT should be a bit more light wight, but is more specific on the other side.

My naive thinking was that spindle control would not be done while any motor is running and so concluded that CPU consumption or any other delay should not be an issue at this moment. But this was just a black box assumption.

[Protoneer]

@bdring

You also have to "clean" before each compile. The automatic clean does handle the custom folder.

Thanks!!!

I am looking at M114 next. That is reporting available axes and position data....

I have the following and that seems to work okay.

grbl_sendf(CLIENT_SERIAL, "X:%f Y:%f\r\n", gc_state.position[0], gc_state.position[1]);

Results:

Grbl 1.3a ['$' for help] g0x100 ok ok m114 X:100.000000 Y:0.000000 ok

[bdring]

It is probably better to discuss this type of thing on Discord.

[whoim2]

@bdring

You also have to "clean" before each compile. The automatic clean does handle the custom folder.

Thanks!!!

I am looking at M114 next. That is reporting available axes and position data....

I have the following and that seems to work okay.

grbl_sendf(CLIENT_SERIAL, "X:%f Y:%f\r\n", gc_state.position[0], gc_state.position[1]);

Results:

Grbl 1.3a ['$' for help] g0x100 ok ok m114 X:100.000000 Y:0.000000 ok

Hello, how is the process going between esp32grbl and openpnp? I have not yet fully assembled my machine, but am looking for any information on this. If you could tell me briefly about future problems, it would help me a lot.

For example, during the initial setup, I had to remove the M400 command from openpnp in order for the machine to start moving. Will this affect the actual performance of the machine? Will there be a problem that teams will be ahead of real movements? Thank you!

[MitchBradley]

M114 is redundant with the existing grbl command '?'. ? is much better than M114 because it is handled immediately as an individual character without polluting the serial input buffer. M114 is almost impossible to handle correctly in a sender. cncjs tried many different techniques to get timely position reports from Marlin, but never found a solution that yielded timely reports without causing interruptions in GCode data streaming. Marlin systems often (usually) have attached LCD displays that are updated internally by Marlin. Furthermore, Marlin is most often used for 3D printers with very long job times, where fine-grained position reports are rarely needed.

The Grbl_ESP32 developers are unlikely to accept patches that try to turn Grbl into Marlin. Such suggestions have come up many times, and have been consistently rejected. In some cases, useful features that Marlin might have could be accepted, but the syntax would generally need to be similar to NIST RS274 / LinuxCNC rather than the Marlin dialect.

[whoim2]

openpnp allows you to fine-tune both command sending and reading / processing using regex. Most likely, many (if not all) tasks can be solved by correctly configuring openpnp for the capabilities and responses of a specific firmware (esp32 grbl). The openpnp developers are interested in esp32grbl support and are willing to make changes if needed. And this is correct, but only so far the issues can be resolved by openpnp.