Pi4/Mac UGS "Grbl has not finished booting" Error

[wbatie]

I am trying to use Universal G-code Sender with grblHAL on a MAC and Pi4. I get a connection, but when every I run a command I get the error "Grbl has not finished booting. I am using version "grblHAL-3XU-compat.hex" on a Teensy 4.1-GrblHAL board. bCNC works fine on the same setup.ie Any suggestions?

Thank you, Bill Bag

[terjeio]

do you have a proposal for the specification of 0x87 and $I+?

0x87 just returns a complete real-time report including any on-change only elements and extensions. IMO this is needed regardless as it is not desirable to soft-reset the controller on connect.

$I+ returns additional information in two categories - one aimed at senders for configuration the other aimed at support. An example from the controller I have running now:

[VER:1.1f.20201201:]
[OPT:VNMSL,35,1024,3,0]
[NEWOPT:ENUMS,*EEPROM,ES,TC,SS,PID]
[FIRMWARE:grblHAL]
[DRIVER:MSP432]
[DRIVER VERSION:201125]
[BOARD:CNC BoosterPack (Trinamic)]
[PLUGIN:TMC2130 v0.01]
[PLUGIN:MODBUS v0.02]
[PLUGIN:HUANYANG VFD v0.02]
[PLUGIN:ODOMETERS v0.02]
ok

NEWOPT and possibly FIRMWARE is for sender configuration, the rest after that is for support(?). NEWOPT options should be from a standard list? Having a support category means that anyting needed for that can be added without breaking the extended protocol. Note that I do not like the idea to use [MSG:...] as part of this response.

About the welcome message, could it be solved by adding a real time command for requesting a new one?

I do not think this is needed, and would require another message to indicate a reset? It would break backwards compatibility too? A separate channel for the repsonse is not neccesarily available when UART communication is used, for some of my drivers only the Rx pin of the secondary UART is brougth out, Tx from the primary is shared.

0x87 paired with $I+ and $+ (for requesting all settings) is IMO a clean way to bring the protocol forward and still maintain backwards compatibility. The exception will be those senders that relies upon getting the welcome message on connect, this will no longer be universally true and is not always possible to fix on the controller side due to hardware or firmware limitations.

A 0x87 request could be used by the controller as well, if received the controller then knows that the sender is capable of handling protocol extensions?

[MitchBradley]

I propose that we discuss one subtopic at at time. In the standards committees that I have been on, the only way we could make progress was to start with a draft, then break it down into small subtopics and discuss each one individually. I think the draft sections provide a reasonable list of subtopics, namely

• Welcome message format
• Triggering welcome messages
• Identification message format
• List of standard ID messages
• Triggering ID messages
• Guidelines for sender implementers

I suggest that we start with "Identification message format".

[MitchBradley]

On the topic of "identification message format":

I proposed the [MSG:tag: value] encapsulation because [MSG: ...] is already part of the legacy protocol, so it is likely to work with a lot of existing senders. Old senders will not understand the information contained in the tagged messages, but they will probably not break at the protocol parsing level. The likely result is that the sender will just display the message somewhere that the user can see. Even if the sender cannot directly use the information, the user can see it and report it to support people.

An additional advantage of the [MSG: encapsulation is that they are asynchronous, so senders are supposed to handle them at any time. That further increases the chances that they will not break existing parsers.

One alternative is to dispense with the [MSG: prefix and just use [tag: value] directly. That is more likely to confuse legacy senders that may not be prepared for arbitrary messages.

Discussion?

[breiler]

Upsides by using the [MSG:]-encapsulation:

1. easily extendable
2. should not effect current sender implementations

Downsides by using the [MSG:]-encapsulation:

1. we are building on the legacy protocol which could be confusing over time
2. we would be using it in an unintentional way given the original protocol
3. need a secondary identification system for controllers that doesn't yet use the message tag, i.e welcome message

Upsides by creating an extended/new command (i.e. $I+)

1. easily extendable with a new protocol

Downsides:

1. More steps for protocol negotiation ([welcome message + 0x87] + $I+)
2. Need a secondary identification system for controllers that doesn't yet use it, i.e welcome message

[MitchBradley]

we are building on the legacy protocol which could be confusing over time

I'm not sure I understand this point. I don't see how the addition of tag structure to MSG: is any more confusing over time than the addition of new messages that are not prefixed with MSG:. It seems the same to me with respect to future evolution.

we would be using it in an unintentional way given the original protocol

It boils down to whether or not MSG: values that are not in the set that is listed on the Grbl wiki confuse senders. There is evidence that it does not, namely the fact that Grbl_Esp32 uses MSG: for a lot of new messages and is known to work with a lot of senders.

need a secondary identification system for controllers that doesn't yet use the message tag, i.e welcome message

That is true regardless of whether new information is encapsulated in MSG: or not. Old controllers do not use either format. Senders that wish to handle old controllers have to adapt using whatever information is already available.

Upsides by creating an extended/new command (i.e. $I+)

I think that is more properly discussed under the "triggering ID messages" subtopic. I think that a new trigger like $I+ is a good idea. Perhaps you are suggesting that new messages can only appear in response to the new trigger, so we don't need to shoehorn into an existing protocol loophole. That is plausible, but less valuable, I think, than allowing messages to appear at other times, guarded by an already-existing safe encapsulation.

[terjeio]

I suggest that we start with "Identification message format".

Ok, IMO that is what I did with my previous message.

Old senders will not understand the information contained in the tagged messages, but they will probably not break at the protocol parsing level

Old senders should never see the new format - if not updated they are stuck in the past. What is needed as step one is for senders that are actively developed to be able to tell what they are talking to so they can make use of new features. One way to do that is to add a request character. Since I had to add one, 0x87, for my sender to get a complete status report from grblHAL this could be used.

An additional advantage of the [MSG: encapsulation is that they are asynchronous, so senders are supposed to handle them at any time. That further increases the chances that they will not break existing parsers.

Parsers needs to be updated anyway to handle extensions to messages. Guidelines for how to write parsers "passively" should be published - if followed protocol extensions should not cause them to fail.

One alternative is to dispense with the [MSG: prefix and just use [tag: value] directly. That is more likely to confuse legacy senders that may not be prepared for arbitrary messages.

[MSG... is a push message and, as I wrote above, I do noot like the idea of including them in response messages. By adding $I+ (and $+ for settings as I understand Grbl_Esp32 already has) legacy senders does not see any protocol extensions at all. The challenge then boils down to how a sender can get to know what kind of controller it is talking to. Requesting $I+ and wait a bit for a response could be one solution, another is the full status report request and there may be others.

More steps for protocol negotiation ([welcome message + 0x87] + $I+)

IMO forget about the welcome message, it cannot be used without causing trouble. For some of the processors/cards that grblHAL supports it is not possible nor desirable to do a hard or soft reset on connecting. And I do not want to exclude them from the fold. If legacy sender does not handle that then the user has to switch to a compatible sender or a different processor/board.

Need a secondary identification system for controllers that doesn't yet use it, i.e welcome message

I do not understand this statement. If a controller does not use the new protocol then it should work as before - at least for processors/cards that is hard reset on connect? BTW I have made it configurable/selectable in my sender how a serial connection is initiated: toggle DTR, toggle DTR or do nothing. This means the user can decide whether a hard reset is performed on connect for cards that support that.

[terjeio]

@MitchBradley Do you understand the passage These messages are "pushed" from Grbl and may appear at anytime. in the interface specification to mean that these are even allowed in request responses? And handled correctly by all senders? IMO then the welcome message, a full realtime report and an alarm message (if an alarm is active) can be added to a $I request without breaking the protocol...

[MitchBradley]

"All senders" is an impossibly high threshold because there are so many of them, some of which are very badly written and break even with some classic GRBL implementations. I do know that Grbl_Esp32 uses [MSG: ...] extensively to report lots of configuration and debugging information beyond the gnea list of [MSG: variants, and that does not seem to cause sender problems. (The senders that have problems with Grbl_Esp32 tend to be ones that expect very specific welcome messages, and ones that expect an automatic welcome message; those are problems to address separately).

I would not advocate adding welcome, realtime status, and alarm to $I, since none of those are included in the "may appear at any time" section. Especially not welcome, because it indicates that GRBL is in a "reinitialized state".

If the concern is that including [MSG: between $I and ok might be dangerous, that could be solved by issuing the [$MSG reports after the ok.

[MitchBradley]

The Grbl 1.1 message format is explicitly designed to be context-free - c.f. Message Summary

In v1.1, Grbl's interface protocol has been tweaked in the attempt to make GUI development cleaner, clearer, and hopefully easier. All messages are designed to be deterministic without needing to know the context of the message.

I think that feature is very important, since it greatly simplifies the response parsing task, decoupling it from the sending task in event-driven senders like CNCjs. Having new messages that can only be understood in the context bracketed by $I+ .. ok violates that core philosophy.

I do not have a problem with the addition of a new command like $I+ for the purpose of triggering some messages, but I don't think that it should create a parsing context for understanding the messages. Context-dependent parsing can cause all sorts of timing/sequencing dependencies, especially when you consider the possibility of multiple senders listening on different interfaces.

[breiler]

Both are making strong cases and we might not be able to agree on anything. The thing is that people from grblHAL reached out to me because we should implement support for it in UGS whereas Grbl_Esp32 just worked seemingly complying with the legacy protocol extending it carefully.

GrblHAL has in my eyes already broken stuff, for instance:

• $5 which now is a bitmask instead of a boolean (in compability mode 0)
• Not able to always send the welcome message upon startup (which by the way was the recommendation by Sonny)
• When using a MPG it's assumed the sender are asking for a status reports to figure out if the sender is allowed to send messages (if I understood its usage correctly from the sparse documentation)

This is all good stuff with good intentions, but if it can't be used by other senders you have a lot of work in front of you implementing both the firmware and sender.

Even if it is maybe semantically not correct to use the MSG-tag for sending version data it looks like it's already working in the Grbl_Esp32 implementation. Which is why I'm leaning towards that solution because I believe that, in the end this will attract a larger user group - the users don't care about protocol stuff they just want their machines to work.

The MSG-solution is well written up and is something that I can work with. The grblHAL-solution requires a bit more work from our side but can probably eventually be done. I would appreciate if you (@terjeio) could write a guide for the protocol negotiating with GrblHAL.

Thank you both for your time trying to solve this!

[MitchBradley]

I like the idea of one or more new inquiry commands, and I agree that welcome messages have many problems. So shall we discuss the best way to extend the inquiry?

Here are my current thoughts, based on what we have talked about so far. Many of these are restatements of what others have suggested or noted.

• The welcome format is so problematic that it might be best, going forward, for controllers to revert to a very simple format like

  Grbl 1.1d ['$' for help]

  That would cause the fewest compatibility problems with random old senders. Well-maintained senders would still want to accept a wider range of welcome strings to accommodate random controllers.

• Senders cannot depend on automatic welcome message issue, therefore it is prudent to have an alternative way to identify that Grbl is present.
• Let's first consider "probe for Grbl" possibilities that work with existing controllers: It is tempting to send a ? and look for a <..> status report response, but that fails in the circumstances when the ? response is absent or delayed. $I is also tempting - but that has the problem of working only in idle and alarm states. On the other hand, the resulting error message is also indicative of grbl presence, and the fact that it is busy or alarming is useful knowledge. Finally, there is $G. $G works in any state, generating a one-line response with a predictable format. It has no side effects, so it should be safe to issue when operations are already in progress.
• So I think that $G is a workable way for a sender to detect that it is talking to some form of grbl. It doesn't give you version information, but it is a good way to both get the "high order bit - this is some kind of grbl", and also to synchronize with the protocol in the face of possible garbage characters. For ultimate robustness, the sender should probably issue newline before $G , and be prepared to discard characters before the opening [ of the [GC: state message.
• Having detected Grbl and synchronized via $G, we need a way to get extended version information. I like the $I+ idea, with the following stipulations:
  • $I+ can be issued in any state - which implies, as in implementation detail, that it should get its information from memory instead of having to go out to EEPROM like $I does in some versions.
  • $I+ responds with a variable-length list of [MSG:tag: value] tagged messages. Such messages could appear at other times too, so the sender must not assume that they are specific to the $I+ command. $I+ is just a guaranteed way to make them appear. If $I+ results in an error, the controller code is old and does not support the new extended ID scheme, so the sender must fall back to whatever ad hoc identification procedures it currently uses.
  • We need to pin down the specific list of tags and their meanings
• Is a new realtime command necessary? Does the $G then $I+ scheme solve the problems?

[terjeio]

@MitchBradley I'll start from the bottom.

Is a new realtime command necessary?

I want it for my sender to configure itself on connect. I have added several new elements to the real-time report, some of which are reported only on changes - I use the new one to be able to get all.

There is a "bug" in legacy grbl, and perhaps in most ports too, that need to be fixed though, also for the standard real-time report, see this comment.

We need to pin down the specific list of tags and their meanings

IMO we only need to pin down the ones needed for the sender. Other tags may appear to aid support - the sender should make them available somewhere. From my sender:

Note that I have a copy to clipboard button here, this outputs the $I/$I+ response as comments and all setting values. This makes it easy for a user to collect data needed for support.

Tags appearing in the scrollable list are for support purposes and should not be parsed. I have added one tag I need for my sender that contains a comma separated list of options in mnemonic form, an example: '[NEWOPT:ENUMS,*FRAM,ES,SD,ETH,TC]' This, or a similar tag, should be part of the protocol.

$I+ responds with a variable-length list of [MSG:tag: value] tagged messages. Such messages could appear at other times too, so the sender must not assume that they are specific to the $I+ command. $I+ is just a guaranteed way to make them appear. If $I+ results in an error, the controller code is old and does not support the new extended ID scheme, so the sender must fall back to whatever ad hoc identification procedures it currently uses.

Agree, but I really want the additional real-time report request character - this as it solves several problems at once.

$I+ can be issued in any state - which implies, as in implementation detail, that it should get its information from memory instead of having to go out to EEPROM like $I does in some versions.

Ideally it should be made available when in locked mode as well - see the comment link above.

Finally, there is $G. $G works in any state, generating a one-line response with a predictable format. It has no side effects, so it should be safe to issue when operations are already in progress.

A new real-time request character plus $I+ means that this IMO is not needed. And it does not work in locked mode...

The welcome format is so problematic that it might be best, going forward, for controllers to revert to a very simple format

Agree, it cannot be relied upon going forward.

@breiler

GrblHAL has in my eyes already broken stuff

It is impossible not to break the original protocol when moving forward. My intention is to break it in a way that follows the message formats/structures in the original specification where possible.

I would appreciate if you (@terjeio) could write a guide for the protocol negotiating with GrblHAL.

I have started that already, I just finished some new stuff that may be of benefit to users and sender developers alike (but surely represents a challenge for developers). I'll come back to documentation once this is commited to github.

Some links to the Wiki, some information is outdated - will be revised soon:

Changes from grbl 1.1 Report extensions

[MitchBradley]

Is a new realtime command necessary?

I want it for my sender to configure itself on connect. I have added several new elements to the real-time report, some of which are reported only on changes - I use the new one to be able to get all.

Does that have to be a single-character command? Would it work to have $I+ trigger a full report? Assuming that $I+ works in locked state.

There is a "bug" in legacy grbl, and perhaps in most ports too, that need to be fixed though, also for the standard real-time report

Good point. I had forgotten about that, since it is fixed in Grbl_Esp32, the code base that I spend the most time with.

You are right that $G fails in locked mode, so it is useless for probing. So we can drop the $G idea entirely.

We cannot assume that controllers in the field will ever be fixed WRT ?, thus robust senders must assume that ? might not do anything. But that does not imply that a new single-character report request is necessary. A new single-character report might (or might not) be a good idea, but it will not help for existing controllers that do not implement it. Which leaves us back at the place where there is no obvious way for a sender to reliably detect a GRBL . But how about this:

Try $I+ first. If that doesn't respond, try ?, and if it does not respond, try $G, and if that does not respond, try ^x . The ^x has the side effect of resetting the system, but if the system is locked, that is the best you can do.

Ideally $I+ should be made available when in locked mode as well

Agree. That is absolutely necessary.

IMO we only need to pin down the ones needed for the sender.

Yes, that is what I meant. The draft distinguishes between "standard" tags - prefixed with _ - and others. Only the standard ones need agreement.

[terjeio]

Is a new realtime command necessary?

I want it for my sender to configure itself on connect. I have added several new elements to the real-time report, some of which are reported only on changes - I use the new one to be able to get all.

Does that have to be a single-character command?

Yes.

Would it work to have $I+ trigger a full report?

Only single character real-time commands can be easily added/processed when in locked state without a rewrite of input processing. Anyway, if triggered it should be sent after the ok delimiter.

Assuming that $I+ works in locked state.

Good question, it does not. $I+ should get its own single top-bit set character variant as a workaround?

Try $I+ first. If that doesn't respond, try ?, and if it does not respond, try $G, and if that does not respond, try ^x . The ^x has the side effect of resetting the system, but if the system is locked, that is the best you can do.

I do not agree. I send a 0x87 first for a complete real-time report and wait for a short time for a reply. If I get it then I am pretty sure I am talking to grblHAL. If so I also want to know if the controller is in locked/alarm state, currently I do that by adding the alarm code as a substate to the state text: <Alarm:14:... This is added only if configured in the controller - perhaps 0x87 should always add the alarm code?. Or should there be some other method for getting it? Depending on the alarm code the sender can decide whether to send a ^x or not.

0x87 can be used for detecting a controller with an expanded protocol, the controller state (sys.state), if the controller in locked mode or not and getting all other real-time report elements in one go.

[MitchBradley]

0x87 works only for new controllers that support it. The $I+, then ?, then $G, then ^x procedure was intended to be a way for senders to handle old controllers.

[terjeio]

The $I+, then ?, then $G, then ^x procedure was intended to be a way for senders to handle old controllers.

Ok. Old controllers responds with error:3 on a $I+ so I assume you mean $I. Most old controllers, at least 8-bit variants, respond with the welcome message on connect since they are hard reset by toggling RTS or DTR - this is common practice? If they are not hard reset they will not respond at all if in locked mode.

What do you think about adding the alarm code as a substate to Alarm when a real-time report is requested via 0x87? Senders needs to be updated to use 0x87 anyway, and IMO they should be able to handle substates for all states by ignoring them as a minimum requirement. An example: I have added an optional substate to the Run state that can be used for primitive probe protection, and one when a feed hold is pending (pending since it was requested during a spindle synced motion - if allowed on a lathe then stopping motion is not neccesarily good for the cutter). Both of these can safely be ignored by the sender if the developer does not want to add functionality for them.

[HuubBuis]

I have been watching this issue for some time now.

As a grbl developer I want that many people use the software. Most of the grbl users probably prefer the sender they are used to and don't want to change that sender. Keeping grblHAL compatible to current grbl (senders) is required to be attractive for most people. IMO this is the most important requirement.

As a Sender developer I need to identify the controller type so I can adapt to the controllers capability's. Currently I sent a soft reset followed by a $X after the connection is made. This brings the controller in a "know state" and it allows me to read the build information. I use the build information to Identify the controller type. Until now this works fine on all supported (grbl, grblMega, grbHAL, grbl_ESP32) controllers.

I use this routing to identify the controller type

        public bool SetGrblControllerType()
        {
            // arduino uno: [VER:1.1g.20200519:]<cr>[OPT:VX1HS,13,128]<cr>ok<cr>

            GrblControllerType = GrblControllerTypes.UnKnown;               //default setting is unknown
            if (GrblBuildInformation.Contains("Using machine:ESP32"))
            {
                GrblControllerType = GrblControllerTypes.grblESP32;
                return true;
            }
            if (GrblBuildInformation.Contains("DRIVER VERSION:"))
            {
                GrblControllerType = GrblControllerTypes.grblHAL;
                return true;
            }
            if ((GrblBuildInformation.Contains("[VER:1.1f")) | GrblBuildInformation.Contains("[VER:1.1g") | GrblBuildInformation.Contains(".Mega"))
            {
                GrblControllerType = GrblControllerTypes.grbl;
                 return true;
            }
            return false;     // return the result
        }

[terjeio]

@HuubBuis In order to unlock new features the current protocol has to be extended.

Keeping grblHAL compatible to current grbl (senders) is required to be attractive for most people. IMO this is the most important requirement.

And it is the least attractive for me - there are a number of 32-bit ports that stops at implementing about the same functionality as the 8-bit version. I did not want to just add another one to that list. Breaking changes had to be added to the protocol for new features- and made it neccesary to add the compatibility compile time setting for those wanting to use older senders.

Currently I sent a soft reset followed by a $X after the connection is made. This brings the controller in a "know state" and it allows me to read the build information.

Soft resetting the controller will in many cases be ok to do, but not always. For example for my lathe I have the MPG that I can use for manual control without the sender. And there could be a simple UI attached where stand-alone streaming from local storage (SD card) could be running. In such scenarioes resetting the controller on a sender connect is not always desirable as it will interrupt everything running, lose the homed status and potentially other stuff like current tool or G92 offsets. So having a way for a sender to query a running controller, get status from and it use this to connect gracefully will be a good thing. For grblHAL it could even be used to get rid of the compatibility options since the query could be used to set compatibilty mode at run-time.

It will be a good thing if we can agree on a way to move forward that makes it easier for sender developers to keep their programs usable, hopefully without the need to update too often - just to match controller development.

An example could be settings handling, grblHAL is getting new plugins that needs settings and the sender needs to handle this. New alarm codes are sometimes needed too. Is it possible to avoid sender updates for most of these? I think so, and I am about to finalize the first version of enumerations that can be fetched from the controller: alarm codes, error codes, settings and setting groups. They are dynamic too, add a plugin with settings and they will show up - even for user supplied plugins. Settings are enumerated with a name, a short description, group binding, datatype, format and min/max value allowed, this so that a nice UI can be made in the sender. I do not know if this is going to be adopted going forward - I did it for making sender development/maintenance easier for myself.

[HuubBuis]

@HuubBuis In order to unlock new features the current protocol has to be extended.

Keeping grblHAL compatible to current grbl (senders) is required to be attractive for most people. IMO this is the most important requirement.

And it is the least attractive for me - there are a number of 32-bit ports that stops at implementing about the same functionality as the 8-bit version. I did not want to just add another one to that list. Breaking changes had to be added to the protocol for new features- and made it neccesary to add the compatibility compile time setting for those wanting to use older senders.

I also want to that grblHAL will be extended but in such a way that old senders are still functional (if possible). In time, users will change the sender if they can't use the new features.

Soft resetting the controller will in many cases be ok to do, but not always.

It is just the way I handle it currently. If we can work out a uniform way to Identify the type of controller, it can be added to the Wiki to make it easier for sender developers to adjust their code.

It will be a good thing if we can agree on a way to move forward that makes it easier for sender developers to keep their programs usable, hopefully without the need to update too often - just to match controller development.

I totally agree

I am about to finalize the first version of enumerations that can be fetched from the controller: alarm codes, error codes, settings and setting groups.

I like this

I suggested in a post (I can't find it any more) to use a 32 bit Hex to identify the controller (16 bit controller ID like the Vendor ID of a USB device) and it's capabilities (16 bit). This could be extended by a 16 bit status value. The controller ID could be distributed by a Git repository on request. We could define part (8 bit) of the capabilities and status value as a kind of standard leaving 8 bits for controller specific needs.

when a real-time report is requested via 0x87?

This could be the place to handle this as all other options won't work when the controller is locked!

[MitchBradley]

Identifying controllers with magic numbers causes a lot of trouble going forward, because it is really difficult to maintain a "registration authority" mapping number to controller. Furthermore, the sender faces a never-ending problem of maintaining a mapping from controller to semantics.

A much better approach would be to define an inquiry mechanism for asking fine-grained questions about features.

[MitchBradley]

@HuubBuis FYI, Grbl_Esp32 currently supports the following inquiry functions about alarms, errors, and settings:

$A or $Alarms/List lists the alarm codes and their descriptions, for example

$Alarms/List
0: None
1: Hard Limit
etc

Similarly $E or $Errors/List lists errors:

$Errors/List
0: No error
1: Expected GCode command letter
etc.

You can ask about a specific alarm or error number by adding an argument, as with:

$E=8
8: Command requires idle state

You can get a list of all non-setting commands with $Commands/List or $CMD:

$CMD
$Home or $H
$System/Sleep or $SLP
$Commands/List or $CMD
etc

Settings can be listed with $$ for Grbl-compatible numbered settings, with output the same as for classic Grbl. For extended settings, use $Settings/List or $S:

$Settings/List
$AP/SSID=GRBL_ESP
$Report/Inches=Off
$Firmware/Build=
$Limits/Soft=Off
$Limits/Hard=Off
$Homing/Enable=Off
$Homing/DirInvert=XY
$Homing/Squared=
$Homing/Feed=200.000
$Homing/Seek=2000.000
$Homing/Debounce=250.000
$Homing/Pulloff=1.000
$GCode/MaxS=1000.000
$GCode/MinS=0.000
$Laser/FullPower=1000
$GCode/LaserMode=Off
$GCode/Line1=
$GCode/Line0=
$Spindle/Enable/Invert=Off
$Spindle/Enable/OffWithSpeed=Off
$Spindle/Delay/SpinDown=0.000
$Spindle/Delay/SpinUp=0.000
$Spindle/PWM/Invert=Off
$Spindle/PWM/Frequency=5000.000
$Spindle/PWM/Off=0.000
$Spindle/PWM/Min=0.000
$Spindle/PWM/Max=100.000
$Spindle/Type=NONE
$X/StepsPerMm=100.000
$Y/StepsPerMm=100.000
$Z/StepsPerMm=100.000
$A/StepsPerMm=100.000
$B/StepsPerMm=100.000
$C/StepsPerMm=100.000
$X/MaxRate=1000.000
$Y/MaxRate=1000.000
$Z/MaxRate=1000.000
$A/MaxRate=1000.000
$B/MaxRate=1000.000
$C/MaxRate=1000.000
$X/Acceleration=200.000
$Y/Acceleration=200.000
$Z/Acceleration=200.000
$A/Acceleration=200.000
$B/Acceleration=200.000
$C/Acceleration=200.000
$X/Home/Mpos=0.000
etc.

As usual, any setting can be changed by supplying "=value", using either the classic numeric setting name or the text name.

Axis masks can be specified either numerically or by listing the axes:

$Homing/DirInvert=7
$Homing/DirInvert=XYA

The value of an individual setting can be shown by saying its name with no =, for example

$Homing/DirInvert
$Homing/DirInvert=XYA

Setting names are hierarchical, for example axis/subsystem/parameter . The hierarchy can extend to more levels as needed.

You can ask to see related groups of settings by using a substring of the full name, as in:

$spin
$Spindle/Enable/Invert=Off
$Spindle/Enable/OffWithSpeed=Off
$Spindle/Delay/SpinDown=0.000
$Spindle/Delay/SpinUp=0.000
$Spindle/PWM/Invert=Off
$Spindle/PWM/Frequency=5000.000
$Spindle/PWM/Off=0.000
$Spindle/PWM/Min=0.000
$Spindle/PWM/Max=100.000
$Spindle/Type=NONE

You can get a JSON-encoded listing of WebUI-related settings with

$WebUI/List
{"EEPROM":[
    {"F":"network",
      "P":"Sta/SSID",
      "H":"Station SSID",
      "T":"S",
      "V":"OpenWrt-Bradley",
      "S":"32",
      "M":"1"
    },
    {"F":"network",
      "P":"Sta/Password",
      "H":"Station Password",
      "T":"S",
      "V":"******",
      "S":"64",
      "M":"8"
    },
    {"F":"network",
      "P":"Sta/IPMode",
      "H":"Station IP Mode",
      "T":"B",
      "V":"1",
      "O":[
        {"DHCP":"0"},
        {"Static":"1"}
      ]
    },
etc

Complete documentation can be found at https://github.com/bdring/Grbl_Esp32/wiki/Settings . The language was carefully chosen so that it does not conflict with existing sender usage. All new capabilities in the setting language were invalid in classic Grbl, and old statements are implemented with their previous syntax, value representations, and meanings.

[terjeio]

Can we agree on the following?

A new realtime request character, 0x87, can be used to get a complete real time report inluding "on change only" elements. If in alarm state the alarm code is added as a substate in the same way as done for Hold or Door. A controller may or may not add the alarm code and the sender shall be able handle this.

A new $-command $I+ can be used to get extended information about the controller. This information can also be retrieved by a new realtime request character, 0x??, in order to make it available when in a locked state (do we need this?).

---

I would like to see the NEWOPT tag adapted as a way to convey information about controller capabilities. The following options (in a comma separated list) are currently defined in grblHAL:

SD - SD Card streaming available. BT - Bluetooth streaming available. WIFI - Wi-Fi streaming available. ETH - Ethernet streaming available. TC - Manual tool change (M6) available. ATC - Automatic tool change (M6) available. LATHE - Lathe mode configured. SS - Spindle sync available. PID - PID log data available. OS - Optional stop signal/handling available. BD - Block delete signal/handling available. ES - E-stop signal available. PC - Probe connected signal/handling available. MPG - MPG mode available (via secondary UART input). ENUMS - code enumerations are available.

[HuubBuis]

A new $-command $I+ can be used to get extended information about the controller. This information can also be retrieved by a new realtime request character, 0x??, in order to make it available when in a locked state (do we need this?).

If you need information about the controller and don't want to reset the controller, we need this.

I would like to see the NEWOPT tag adapted as a way to convey information about controller capabilities. The following options (in a comma separated list) are currently defined in grblHAL:

I need single axis homing on a lathe (can't home Z-axis when there is no chuck for turning between centers). Currently I check the settings and options. A NEWOPT (HS or HSX or HSXZ) would be handy.

Can we agree on the following?

I can

[terjeio]

I need single axis homing on a lathe (can't home Z-axis when there is no chuck for turning between centers). Currently I check the settings and options. A NEWOPT (HS or HSX or HSXZ) would be handy.

FYI, grblHAL reports homed status in the extended real-time report. Both if completed as configured and the corresponding axismask if single axis homing is enabled. So IMO no need to add this to the NEWOPT report if we can agree to add this to the real-time report instead.

if(sys.report.homed && (sys.homing.mask || settings.homing.flags.single_axis_commands || settings.homing.flags.manual)) {
    axes_signals_t homing = {sys.homing.mask ? sys.homing.mask : AXES_BITMASK};
    hal.stream.write_all(appendbuf(2, "|H:", (homing.mask & sys.homed.mask) == homing.mask ? "1" : "0"));
    if(settings.homing.flags.single_axis_commands)
    hal.stream.write_all(appendbuf(2, ",", uitoa(sys.homed.mask)));
}

[HuubBuis]

FYI, grblHAL reports homed status in the extended real-time report. Both if completed as configured and the corresponding axismask if single axis homing is enabled. So IMO no need to add this to the NEWOPT report if we can agree to add this to the real-time report instead.

I am interested in the capability and will need it only once at startup. Adding it to the extended real-time report will certainly do for me.

[terjeio]

A draft "For sender developers" has now been added to the grblHAL wiki. The report extensions wiki page has been updated to better match the grblHAL version found in the test branch.

[breiler]

I've skimmed through the docs, nice work.

I think the detection process especially with the MPG are way too specific and we will probably not focus on implementing support for it right now. If it were possible to detect the firmware version first then load the specific dialect implementation that would have been ok. If these parts of the protocol gains popularity then we'll have a look at it again.

The 0x87 and $I+ are great - but only if they are supported by other firmwares as well, otherwise it the same problem as with the MPG support. If we need to implement the specific detection process for every GRBL fork it wouldn't scale very well...

I don't know what you meant by Do not hardcode a settings user interface but it sounds like it would limit us from working on the user experience, ie. for when configuring the hardware. Is this because you want the freedom to change the settings contract?

[terjeio]

I think the detection process especially with the MPG are way too specific and we will probably not focus on implementing support for it right now.

That is perfectly ok for now, IMO it will be some time before this option is picked up by user - if ever.

If we need to implement the specific detection process for every GRBL fork it wouldn't scale very well...

That is why it would be great to get consensus among ports on how to expand the protocol.

I don't know what you meant by Do not hardcode a settings user interface but it sounds like it would limit us from working on the user experience, ie. for when configuring the hardware. Is this because you want the freedom to change the settings contract?

Yes, I (and I guess other port maintainers) would like to have the freedom to add settings. But not by limiting a rich settings UI in the sender. The latest changes I have made to grblHAL means that I can now provide a rich UI in my sender without the problem of how to keep the two in sync.

IMO senders supporting the grbl 1.1 format specs should as a minimum have support for using the provided csv files for mapping codes for alarms, errors and settings to textual representations. Or at least not fall over if a new setting code is encountered...

I will provide updated versions of csv files for grblHAL going forward if it is interest for that, however for all parties involved it will be nice if, in the future, this just works "out-of-the" box. Fetching enumerations with enough details from the sender solves that.

Here are examples of the new settings UI in the next version of ioSender:

Adding settings, e.g by activating a plugin with settings, means that these will now automagically appear as the data needed for creating the UI elements dynamically is provided by the settings enumeration.

Note that the detailed description is still fetched from a file, due to both limited flash in some supported controllers and the quality of the text. Fetching translations from a file could still be an option, if a translation is not available for a code then there is always the option of using english from the enumeration data.

The sender is still working as before if the enumeration data is not available if anybody is concerned about that.

[HuubBuis]

I will provide updated versions of csv files for grblHAL going forward if it is interest for that, however for all parties involved it will be nice if, in the future, this just works "out-of-the" box. Fetching enumerations with enough details from the sender solves that.

I currently use the csv files for settings, alarms and error messages. I "just" need to update these CVS files but doing so limits the backward and upward compatibility a bit. Your solution is better and easier to maintain. To make use of the these new features I am going to change the code (make a class for every controller type) what will make it easier and safer to implement specific controller features. This is scheduled for 03-2021.

[breiler]

I wasn't talking about adding settings, I was talking about changing them. But that is fine we will just disable the features that isn't compatible. But in the way you reason we will probably never be able to add them again, and settings such as the $50-55 would not be reliable to use.

We are already adding the CSV-file for GRBL and provide a dynamic settings dialog so I have no problem adding them for GrblHAL as well. But this also only works if you don't change your contract, i.e change the units for a setting from "mm" to "inch". If we download and bundle the csv:s one day and you change the units for a setting the next this will not be reliable.

[terjeio]

@breiler Ok, I get it now I think. The settings contract itself has to be worked out, and if possible should be flexible as keeping settings numbers/ids in sync across ports is likely to be problematic. A standardised way to convey the meaning of a setting could be agreed to - at least for settings a sender would like to get at the value for. Perhaps tagging them the way new settings in Grbl_Esp32 are added is a way forward? Anyway a settings number should IMO be the core id in any case, this to facilitate easy parsing and lookup of potential translations (in a file).

For grblHAL once a setting number is defined its meaning is set in stone, except for bitfields or radiobuttons where additional bits or values may be defined. Bitfield and radiobutton labels are listed in the settings enumeration in order to making dynamic representation possible. Some setting options should even be made available in the $I+ NEWOPT response, lathe mode configured in grblHAL is an example of that.

If we download and bundle the csv:s one day and you change the units for a setting the next this will not be reliable.

That is why I have added the enumerations. Files could still be used but then only for translations of translatable elements. If additional elements are required for creating the UI then these should be added to the enumerations.

[risototh]

Hi, I've a similar issue, and I tried it on Mac and on RPi with Raspian + UGS. I have grblHAL on Teensy 4.1, and short description is, that I think that the problem is, that the grblHAL sends the "welcome string" only once after connection is made via the USB. When I disconnect the UGS (or any serial communicator), and then reconnect, the grblHAL sends nothing, so the sender "cannot see it". I tried the USB_SERIAL_CDC 1 or 2, USB_SERIAL_WAIT enabled or disabled, COMPATIBILITY_LEVEL 0 or 1, and this problem is persistent.

$I outputs [VER:1.1f(IMXRT1062).20201103:] [OPT:VNMSL,35,1024,4]

You can find the video from UGS here https://www.youtube.com/watch?v=DlLO5A7ADAs

Any advice guys?

[terjeio]

Not easy to fix, perhaps for e few controllers but not generally as discussed above and here.

I do not know of any upcall (event) when a USB serial connection is made that can be used to emit the welcome message for drivers that uses Arduino libraries. Some, such as the Teensy, waits for the first connection before finalizing the startup. Some supports a hard reset via the DTR or RTS signals on connect, some not. So a sender wanting to connect cannot rely upon the welcome message, it should query the controller for a message that is guaranteed to be delivered, such as the real-time status.

The only form for connection where a connect can reliably detected is via Telnet or WebSocket, for these grblHAL do output the welcome message on connect. This was added a while back.

[risototh]

OK, got it. I will try bCNC for now, and maybe I will implement the Telnet into the UGS. Maybe, as Java is not my "home language" :) There is already some support for the Websockets, but this does not work for me, and I cannot figure out why. Not sure, if there is some issue on the grblHAL side, or on the UGS side... Will have to look at it, but I need to install some Java dev-env.

BTW, is there anybody, that tried to run the ioSender on Linux? :D

[phil-barrett]

There is at least one person running ioSender in a windows vm on linux - https://www.grbl.org/romanian-router. Sorry I don't have more info for you.

[terjeio]

Not sure, if there is some issue on the grblHAL side, or on the UGS side...

I would like to believe that grblHAL is ok as it works with ioSender (which uses a third party Websocket lib) and ESP32 WebUI. WebUI uses the native browser API? There is a simple test program that runs in a browser window that works as it should too (it is gone from the repo, I'll readd it).

BTW, is there anybody, that tried to run the ioSender on Linux? :D

It uses WPF libs which is not supported natively by Linux.

There is at least one person running ioSender in a windows vm on linux - https://www.grbl.org/romanian-router. Sorry I don't have more info for you.

A bit more here.