luc-github / ESP3D-WEBUI

A Web UI for ESP8266 or ESP32 based boards connected to 3D printers / CNC
GNU General Public License v3.0
762 stars 305 forks source link

[FEATURE REQUEST]Add grblHAL support : https://github.com/grblHAL #252

Closed luc-github closed 2 years ago

luc-github commented 2 years ago

As started to discuss here https://github.com/grblHAL/ESP32/issues/39

terjeio commented 2 years ago

I see that the ESP800 request does not add json=yes

You may use an old version of webUI if json=yes is no used

The issue is not with the WebUI but with the embedded code which I assume is to be returned to the client if the WebUI code is not found. It was last updated 2 years ago, perhaps that is the reason?

Yes it is possible, I do not encourage this because of flash wearing and small size of flash, I can add it if you really want but I am confused I though you hide the flash partition

I no longer do, but I may doing something wrong since it shows up on client startup:

image

GrblHal can have custom Flash it is possible, currently all targets use same but I can add custom one for GRBHal like for SD

Thanks - I remember now, will look at this again. I believe I managed to hide Flash and make SD the default a while back.

The webUI is flexible ^_^

Yes I know - but it takes time to understand your code, IMO it is a large and complex project.

... and cannot compile :

I have broken AP mode, sorry about that. A quick fix is to replace file with response in line 424 and 489.

luc-github commented 2 years ago

It was last updated 2 years ago, perhaps that is the reason?

ho yes the embedded page code follow the webUI one -so if it is 2 years old it is definitly outdated - Ho boy I just realized what you did you took the one in server/public which is a test server for upload not the embedded page itself I have just removed from ignore the embedded dist directory - but it should be modified for your FW because the links go to esp3d repository and webui repository - I may suggest to do a copy of the embedded and change the menu.js as well as the favicon and build scripts as you do not need to convert binary to c files https://github.com/luc-github/ESP3D/tree/3.0/embedded/dist this is for 3.0 for 2.1 compatibility it is https://github.com/luc-github/ESP3D/tree/2.1/embedded/

the embedded page is actually a sub project used by esp3d and esp3dlib I should make it as project to allow multiple targets but until now was only 2 so I did not saw the need ... I guess I could make a repo for it and it could become a submodule for each project

Yes I know - but it takes time to understand your code, IMO it is a large and complex project.

Ho yes that is why no one to handle it ^_^ but yours is not bad neither supporting many platforms is also a challenge

I have broken AP mode, sorry about that. A quick fix is to replace file with response in line 424 and 489

Ok will check that thank you - no worry

terjeio commented 2 years ago

I have just removed from ignore the embedded dist directory

Thanks, now it works! For testing you can replace main/index.html.gz with the one from dist and recompile/reflash.

... as you do not need to convert binary to c files

I have to for platforms other than ESP32, but I'll do it myself as I use a different format.

I should make it as project to allow multiple targets but until now was only 2 so I did not saw the need ... I guess I could make a repo for it and it could become a submodule for each project

Perhaps so, I would guess this code is fairly static so not a high priority? And for the time beeing I am not going to support OTA firmware upgrades.

... supporting many platforms is also a challenge

Yes, abstraction and architecture are the magic words. What I need is a web frontend for building binaries for the various platforms/configurations supported. Maybe I can get some ideas from your .js code?

A minor issue, my code reports an error on the /files request:

image

It looks like filename used for filtering when action is list, aside from all may I ask what kind of filtering can be specified and what is the default?

luc-github commented 2 years ago

I have to for platforms other than ESP32, but I'll do it myself as I use a different format.

The script to convert binary to c is a js so it is easy to modify - I wanted to avoid external application to be platform independant

Perhaps so, I would guess this code is fairly static so not a high priority? And for the time beeing I am not going to support OTA firmware upgrades.

Actually it display according esp800 - so if not OTA then ota tab won't show, same for filesystem, so only menu is actually static

Yes, abstraction and architecture are the magic words. What I need is a web frontend for building binaries for the various platforms/configurations supported. Maybe I can get some ideas from your .js code?

You may want to have a look to https://github.com/luc-github/ESP3D-Configurator it is a simple front end using json file for configuration - it work online and offline, I can imagine you can use the github-action to generate assets / binaries based on the front end configuration

A minor issue, my code reports an error on the /files request:

To be honest it is a fake url : it is same as /files?path=..., it is not even mentionned in https://github.com/luc-github/ESP3D-WEBUI/blob/3.0/Memo/Handlers.md#files , I just did it to have consistent url when I read the devt panel, but actually it is not used - you can safely ignore filename when action is list

terjeio commented 2 years ago

The script to convert binary to c is a js so it is easy to modify

Found it now - yes, easy to modify for grblHAL. But why is the generated file called tools.html.gz and not index.html.gz? Something to do with the webserver? For grblHAL the latter name is easiest to handle.

you can safely ignore filename when action is list

I have just commited a fix for this and the bugs you reported. I have also updated the embedded file to point to the grblHAL repo.

Thanks for the tip about the configurator, will take a look later.

luc-github commented 2 years ago

tool.html.gz is the esp3d v2.1 version, in v3 i use same api as webpack which is index.html

the name has no meaning for the embedded part because is not a file but http answer stored on flash

dresco commented 2 years ago

@dresco you may consider adding this to the STM32H7xx driver - there is ample flash available for that(?) and only a smallish low-level driver is needed

Cool thanks, I'll take a look when I have time. The H743 'only' has 2MB flash, but can use QSPI flash chips (and the WeAct board already has two on it - so may be good place to start testing)..

luc-github commented 2 years ago

@terjeio I have updated sources with latest one and so far so good compilation is fine

About ESP400 output:

Here a screen cast
grblhal

You may want to re-arrange some sections now the rendering is visible

Let me know what you think and feedback/request any change you feel necessary ^_^

Note: the job monitoring support from ? report is not yet implemented - it will be done soon

terjeio commented 2 years ago

I have renabled the feature tab and use a logo for grblHal I did myself in svg to replace the esp3d one, feel free to give me another one if it does not fit your graphical specs

Great - I have to decide if this is the one to use. Here are some other proposals. What do you think?

I have added a detection for the SSID button in features tab to launch the wifi scan but it is seems not working/supported.

Bad code, will fixed in the next commit.

Do you plan to keep the emulation of bitmask to keep 2.1 compatibility support ? Just to know if it worst to add bitmask native support in V3 if you keep emulation for 2.1

v2 and v3 is now handled in separate files so can differ. I can keep emulation for v3 if you think the rendering looks ok. What is missing is the exclusive bitmap where bit 0 controls enable of the rest, could this be handled via emulation and a new flag to control the behavoiur?

I have also noticed you use different method than me for the single page support, your implementation block any new connection, when mine allow any new connection but close old one

This is something I will look into, easier now that the websocket subprotocol is in place. I want the messages to be transparent to the core - to keep normal streaming available. I have tried to implement the PING response but I cannot get it work, it would be helpful if the bracketed elements was explained - not all are obvious. E.g. what is the datatype and expected value ranges for the <time left> and <time out> elements? To be compliant with websocket terminology perhaps the PING response should be PONG...

You may want to re-arrange some sections now the rendering is visible

It is on my todo list to add sorting, perhaps to match ioSender.

Let me know what you think and feedback/request any change you feel necessary ^_^

I believe we are getting there now, basic functionality seems robust to me - not too many rough edges left.

luc-github commented 2 years ago

Great - I have to decide if this is the one to use. https://github.com/terjeio/grblHAL/discussions/268. What do you think?

Thay are all nice - you may also consider the logo should be readable at lower resolution like in screenshoot, and if not, you may consider a variant for the favicon that is consistent with logo IMHO, but I am not good in design and logo is something you must feel confortable with. image

Bad code, will fixed in the next commit.

Ok so I do not change anything

v2 and v3 is now handled in separate files so can differ. Ok so I can work on implement native version then - could you generate an [ESP400] with native settings for bitmask and exclusive bitmask for me to work on final implementation ? as described in https://github.com/luc-github/ESP3D-WEBUI/blob/3.0/Memo/ESP3D%20%5BESP400%5D%20format.md the types are :

  • M: for bits mask (only grblHal)
  • X: for exclusive bitsfield (only grblHal) the descrptions should be defined using - O: is an array of {label:value} used for possible values in selection or bits labels list

it would be helpful if the bracketed elements was explained - not all are obvious

Ok sure I will complete this part and also error messages code and messages

About Ping / Pong I guess the webUI should send the ping and FW answer pong but currently if no authentication currently ESP3D does not answer, webUI ping is to be sure connection is still alive, no need answer, it just try it can send ping, when ping from FW is just to send session information to inform user 30s before end the session will be expired , I used Ping from web UI because was convenient but usage are not really linked - may be some clean up is necessary - but as it was working until now it was not in my top priority list 😅

I believe we are getting there now, basic functionality seems robust to me - not too many rough edges left.

Cool ^_^ I will continue to improve as described above and also add support of the SD stream from ? report, it was not done before as I did not saw it in https://github.com/grblHAL/core/wiki/For-sender-developers 😉

terjeio commented 2 years ago

you may consider a variant for the favicon that is consistent with logo

IMO the small logo for the tab should be yours, but you may consider replacing ESP3D with something processor neutral representing the WebUI? BTW the correct way to write grblHAL is "grblHAL" not "GrblHal" or other variants, same goes for WebUI, it should always be WebUI?

if no authentication currently ESP3D does not answer

I used Ping from web UI because was convenient but usage are not really linked - may be some clean up is necessary

Ok, I think I understand now... The websocket protocol has native ping/pong messages so I guess I was confused by that. FYI currently grblHAL sends a websocket ping message every 3 seconds, if no pong reply is received in 9 seconds then the client is disconnected. The pong reply is handled automatically by the browser so you do not see it.

I see binary frames are reserved for file transfer, have you considered using the ymodem protocol? 128 or 1024 bytes payload size, with CRC and ACK/NAK handling (client resend on NAK). I have implemented that in the sdcard plugin - with a small change that allows the client to initate a transfer. When not in file transfer mode grblHAL will buffer characters for normal processing, however only real time commands are accepted when streaming from sdcard. ymodem transfers should work out of the box with WebUI provided that is has been enabled, I can enable it by default if grblHAL is compiled with the WebUI backend if you are going to support it.

luc-github commented 2 years ago

the esp3D logo is already on interface (webui setting), I though I have already corrected grblHAL typo everywhere (I just noticed the title of this thread and now is corrected) - are there other locations I missed ?

I used websocket binary for printer stream because when I started the arduino websocket library cannot handle text without carriage return which was not suitable as was a stream which I never can control what will come and what size will be delivered And as initially there was no need for ESP3D to do file transfert but on flash for little file I did not think how to use the websocket for file transfert, and as I need a channel for webUI to communicate with board quickly I decided to use the text part of websocket - not really thinked situation - just based on oportunties. Using websocket differently is something I had thinked about but more for the 4.0 as currently is not broken - and I prefere to fix what is not broken when all others things are finished Using webSocket will allow faster file transfert then esp3d webserver - not sure about http idf web server file transfer performances

WebUI does not allow multiple commands at once - when transfering file no commands are sent but websocket ping, this to control the esp3d websocket limitation that only a connection at once especially on esp8266, 5 on esp32, so I limit to 1 for safety

terjeio commented 2 years ago

are there other locations I missed ?

I keep my eyes open, some loactions are more important. And I know the controller startup message is not correct... How to write webui? Is it don't care?

websocket library cannot handle text without carriage return

I'll check if that is a part of the websocket spec. If so a bit strange?

WebUI does not allow multiple commands at once - when transfering file no commands are sent but websocket ping, this to control the esp3d websocket limitation that only a connection at once especially on esp8266, 5 on esp32, so I limit to 1 for safety

Yes - this is important for safety, currently the grblHAL websocket code will deny new connections if a client is connected. I'll try to implement your switchover protocol, the latest changes I made to the websocket daemon should allow that.

FYI I have just commited an update with fixes/changes for ESP400, ESP410 and ESP420.

luc-github commented 2 years ago

I keep my eyes open, some loactions are more important.

I have fixed docs with proper typo - scripts use all lower cases for all names so I did not changed - I need to check the test server as I cannot rename the test directory under windows - it does not see the case modifications for files - I will check / fix on linux later

I'll check if that is a part of the websocket spec. If so a bit strange?

May be was lib issue at this time (was 6 years ago ) or me that mess up, I just kept things as it was ok

FYI I have just commited an update with fixes/changes for ESP400, ESP410 and ESP420.

Cool I am on esp32 idf / lvgl self training right now but will check later

terjeio commented 2 years ago
  • X: for exclusive bitsfield (only grblHal)

This does not render as expected, ESP400 response for $22:

image

Renders as a drop down where one value can be selected:

image

ioSender rendering, bit 0 (Enable) not set:

image

Bit 0 set allows the rest of the bits to be set:

image

Will it be possible to achieve something similar? Or should I emulate this in the same way as I now do for bit and axismasks and ignore bit 1+ if bit 0 is not set when saving?

I have not yet checked how you render bitmasks (M).

luc-github commented 2 years ago

No no, I did not implemented bitmasks neither exclusive bitsfield rendering yet - code for control is ready but not implemented in rendering - I wanted you give me the ESP400 output to be sure the format was ok for you to fill in first - so I can I work on it directly to avoid back and forth

Yes that was the rendering I had in mind - if current output is fine - no need any emulation, it should be ok

About webui typo - there is none really defined actually, I never mentioned it other than ESP3D-WEBUI or ESP3D-WebUI, or WebUI when already in ESP3D context as it is a part of ESP3D project

terjeio commented 2 years ago

websocket library cannot handle text without carriage return

It is not a requirement in the Websocket RFC that text frames has to be terminated with a control character. The arduino subprotocol may add this, but if the Arduino websocket implementation allows no subprotocol to be requested (it should?) then you may avoid it?

When I add currentID: and activeID: messages I get a Websocket connection error message repeatedly. Could this be due to you requiring a control character to terminate the message? And is there any limit to number of digits in the connection id? E.g. can I use the 32 bit address of the internal websocket handle?

No no, I did not implemented bitmasks neither exclusive bitsfield rendering yet

Ok, my fault then - I had the impression you already had done so.

You can enable M-style bitmask formatting by commenting out these two lines in the source.

You cannot do that for the axismask type, perhaps emulation is ok for that? If not you have to get number of axes configured in the controller from somewhere.

luc-github commented 2 years ago

When I add currentID: and activeID: messages I get a Websocket connection error message repeatedly. Could this be due to you requiring a control character to terminate the message? And is there any limit to number of digits in the connection id? E.g. can I use the 32 bit address of the internal websocket handle?

No control character from me, they are just 2 text messages one only for the new connected (current id) and the other is a broadcast (active id) for all websocket clients : https://github.com/luc-github/ESP3D/blob/3.0/esp3d/src/modules/websocket/websocket_server.cpp#L111-L116

You can enable M-style bitmask formatting by commenting out these two lines in the source.

Ok thank you

You cannot do that for the axismask type, perhaps emulation is ok for that? If not you have to get number of axes configured in the controller from somewhere.

It is up to you, yes I get the number of axis from the ? report

terjeio commented 2 years ago

Found the websocket error bug - my fault by not setting the "fin" flag in the message header...

I have managed to upgrade the websocked daemon to allow multiple connections and thus the client switchover and timeout functionality. Settings has been added for session timeout and real time report interval.

This means that grblHAL support for WebUI v3 is now pretty complete?

luc-github commented 2 years ago

I plan to implement the SD streaming support from ? report and implement the float and exclusive bitmask rendering this week end So it should be ok I guess if no issue / missing support is found

luc-github commented 2 years ago

Float support, BitMask and Exclusive Bit mask support is now implemented V 3.0.0.a21

masks

I have also fixed some small issues not grblHAL related, I still need to implement the SD stream report from ? it took me more time than expected for the mask support, but sd stream should be ok Monday

I have noticed that several settings do not have all constraint set : e.g: SSID can be empty, some delay or feedrate can be negative, same for acceleration or resolution, etc... I guess it will be fine tuned later

Getting close to the end 😸

terjeio commented 2 years ago

Great 😊

I have noticed that several settings do not have all constraint set : e.g: SSID can be empty, some delay or feedrate can be negative, same for acceleration or resolution, etc... I guess it will be fine tuned later

I have an input mask available for numeric settings, I can use this to set the lower/upper limit if not defined in the settings specification. Or even better if you could implement input masks at some point? The framework you use support input masks? Currently there is no way to tell if a setting can be blank or have a minimum number of characters - I have to come up with a solution for this that does not break ioSender...

Some suggestions for improvements - for v4?:

Add number of axes to ESP800 (only if > 3?), then it would be easy to implement the axismask in a similar fashion as bitmask (it looks good!)?

grblHAL has a short description available for settings, eg:

$sed=14
ok
[SETTINGDESCR:14|Inverts the control signals (active low).\nNOTE: Block delete, Optional stop, EStop and Probe connected are optional signals, availability is driver dependent.]
ok

To aid the user add a button behind the setting values that opens the description in a modal window? For this add an option to ESP800 to flag availability and a new ESP command to fetch the description for a particular setting?

luc-github commented 2 years ago

The framework you use support input masks?

I do not understand how is this rendering works sorry - I have checked your IOsender App and cannnot see the setting you linked - do you have a sample of expected rendering ?

Currently there is no way to tell if a setting can be blank or have a minimum number of characters - I have to come up with a solution for this that does not break ioSender

You may just add a hash table for each setting and check for each settings have constraint : currently [ESP400] support:

-   S: is max size if type is string, and max possible value if value is number (byte, integer)
-   M: is min size if type is string, and min possible value if value is number (byte, integer)
-   MS: is additionnal min size if type is string (e.g for password can be 0 or 8, so need additional min size), M should be the more minimal value
    so MS value must be between M and S
-   R: need restart to be applied 

Talking about ESP400 I have defined according your wiki some extra settings in https://github.com/luc-github/ESP3D-WEBUI/blob/3.0/Memo/ESP3D%20%5BESP400%5D%20format.md:

    -   E: is integer for exactess / precision of float/double value (only grblHAL)
    -   U: is text unit of value (only grblHAL)

But I do not see them in latest build so currently they are not handled - should I remove them from specs ? or do you plan to add them later ?

Add number of axes to ESP800 (only if > 3?) add any extra in [ESP800] is not an issue - but this should handle laser NB_AXIS=2 ? so better to put for all level (even 1) currently this number is automaticaly detected by parsing the ? report so even not send it will be updated at first ? report I originaly though if settings is mask (M) and option is empty then automaticaly use axis list for it - but you did emulation so currently it is not implemented

grblHAL has a short description available for settings

What about a generic link to your wiki and use setting id as anchor ? this would be super generic and do not increase FW size, if anchor does not exists it will still open wiki page so it is fine IMHO

For this add an option to ESP800 to flag availability and a new ESP command to fetch the description for a particular setting Do you mean a flag in ESP400 report ? and only target specific setting ? well I suggest more my solution above, it avoid any extra work and no need to update FW when wiki is ready and it allow you to have always up to date wiki

Note: These changes do not need to wait V4 as they are not general breaking feature because they as additionnal flag - if not present they are ignored

Additionnal questions:

terjeio commented 2 years ago

The framework you use support input masks?

I do not understand how is this rendering works sorry - I have checked your IOsender App and cannnot see the setting you linked - do you have a sample of expected rendering ?

An input mask filters illegal input, try Stepper > 0 Step pulse time in ioSender - you are not allowed to enter anything that is not specified in the mask - "#0.0". Max 2 digits before the decimal point, only one after and no sign. It can be hard to implement but nice to have when working.

Talking about ESP400 I have defined according your wiki some extra settings

I have missed those, I'll add them.

Settings descriptions:

What about a generic link to your wiki and use setting id as anchor ? this would be super generic and do not increase FW size, if anchor does not exists it will still open wiki page so it is fine IMHO

grblHAL allows third party drivers/plugins to define settings, so hard/impossible to keep a file or web page in sync. To solve this I added the option of having descriptions in the controller itself. So better to fetch it from the controller than link to a web page. BTW I build the ESP400 output by looping through the setting descriptors so any third party settings will show up automatically in the WebUI.

Anyway, displaying setting descriptions would just be a "nice to have" feature so not important.

Additional ESP800 flags/elements:

Note: These changes do not need to wait V4 as they are not general breaking feature because they as additionnal flag - if not present they are ignored

Ok, IMO then for number of axes should be added? Or better a string with the axis letters, "XYZ" for a 3 axis machine, "XYZA" or "XYZU" for four etc. String length = number of axes.

file download from FS and SD give 404 - do I need to disable feature for grblHAL or it will be supported later ?

I'll have to check this again - it should work so do not disable.

do you use mDNS ? I see some mDNS code in your FW, but I cannot see the device in mDNS browser

Not yet - should be easy to add as lwIP has code for it.

I have started to see what is the file stream rendering you mentioned above but cannot see it :

Odd, this is an example of what I get:

<Run|MPos:4.304,5.920,0.000|Bf:0,1023|FS:31,500|Pn:P|SD:0.6,Landscape.nc>

image

I have no good explanation for why you are not seeing it.

[ESP701] seems not giving status neither - do I miss something ? the file is running but no stream report anywhere.

Not even in the networking log? This is what I get:

image

And it shows up correctly.

Have you enabled automatic status reports and disabled polling from the WebUI?

I need to confirm rendering is : SD:0.1,pcb_zebra.gcode is present when processing and not present when processing is finished or no processing

It is (or should be), when processing is finished the standard Grbl message "Pgm End" is sent, if the job is aborted an error message.

what commands now should control the SD stream (pause/resume/stop)? [ESP701] like you did before and currently implemented in WebUI ? or use GRBL commands Hold (!)/ Cycle start (~) / Reset (ctrl-x) ?

Both methods are supported, the Grbl commands can also be sent via a websocket binary frame.


I believe you could easily add support for the grblHAL tool change protocol, when you see the 'Tool' real time status just send the real-time command character 0xA3. Thats it - the SD card streaming code will handle the rest. I should be able to modify your source and verify this.

terjeio commented 2 years ago

Forget about adding the axismask type - I can of course map these to the mask type on my end...

image

luc-github commented 2 years ago

input mask filters illegal input

I did not know input mask - but seems need to add extra library (https://www.npmjs.com/package/inputmask) - why not directly use a regex ? also I do not see reading doc how you can do for AP password that need : no char or at least 8 -64 char when in regex is easy I did not used regex myself because ESP400 is old format I used before understand the regex possibilities - in V4 the ESP400 will definitly go to regex - I am too lazy to spend time on each setting currently to establish the proper regex patern as I am not yet aware of all capabilities / features of regex 😊

I have missed those, I'll add them.

Ok I will implemeted them when ready - no problem

Anyway, displaying setting descriptions would just be a "nice to have" feature so not important.

Ok so if help is a coming from controler no issue if command does not have help, then it can display generic message saying doc is not available yet - so it is fine

Ok, IMO then for number of axes should be added? Or better a string with the axis letters, "XYZ" for a 3 axis machine, "XYZA" or "XYZU" for four etc. String length = number of axes.

Currently webui only support XYZABC https://github.com/luc-github/ESP3D-WEBUI/blob/3.0/src/targets/CNC/grblHAL/TargetContext.js#L121-L133, and in this order - if user actually do not want to use one axis that is present in ? report then it is possible to hide it and all related controls if ? show only 4 positions then webui consider it is an XYZA , if only 2 then onlyX and Y axis for jogging will be displayed, etc... I did not know XYZU notation

So in short term only number is enough I think

Have you enabled automatic status reports and disabled polling from the WebUI?

Ho automatic report and polling have different format ? I have enabled auto report (I think) disabled polling command image

but nothing show up - I may miss one setting image

luc-github commented 2 years ago

additionnaly here the terminal and input commands

>[ESP701] json
{
 "cmd": "701",
 "status": "ok",
 "data": {
  "status": "no stream"
 }
}
>$F=/grbltest.nc
ok
[MSG:Pgm End]
>$F=/grbltest.nc
>[ESP701] json
{
 "cmd": "701",
 "status": "ok",
 "data": {
  "status": "no stream",
  "code": "9"
 }
}
>?
ok
<Run|MPos:13.620,10.000,0.000|Bf:23,1023|FS:100,0|Pn:PXYZS|WCO:0.000,0.000,0.000>
>?
ok
<Run|MPos:14.236,10.000,0.000|Bf:31,1023|FS:100,0|Pn:PXYZS>

It looks like there is an error but it process file without reporting it and status is run

I provide my test file for ref grbltest.txt

terjeio commented 2 years ago

I did not know input mask - but seems need to add extra library (https://www.npmjs.com/package/inputmask) - why not directly use a regex ?

Perhaps that would work in C# too - as input masking was not trivial. However, I am using the same mask for input as I use to format the output, having two is an invitation for bugs to creep in? If regex could be used then the current mask can be converted programmatically?

Anyway, displaying setting descriptions would just be a "nice to have" feature so not important.

Ok so if help is a coming from controler no issue if command does not have help, then it can display generic message saying doc is not available yet - so it is fine

If none available the controller returns "N/A", this can be changed to something else when generating json output for the WebUI.

if ? show only 4 positions then webui consider it is an XYZA , if only 2 then onlyX and Y axis for jogging will be displayed, etc... I did not know XYZU notation

There has been requests to remap ABC axes to UVW, e.g foam cutters use U instead of A (and some V as well?). The next build will allow remapping, so it is coming. Also, the grblHAL core supports up to 8 axes, XYZABCUV. The next build will include one driver/board map that can be extended to all those. It is not something I expect to become a common configuration, and definitely not for WebUI users? Anyway I will add Axisletters to the ESP800 response as this is future proof. You can derive both number of axes and axis labeling from that.

Ho automatic report and polling have different format ?

No, at least it should not. Technically automatic polling queues a "?" request in the same way as from the input stream. I'll recheck.

but nothing show up - I may miss one setting

You forgot to reenable polling in preferences.json? I have missed that a few times when testing.

terjeio commented 2 years ago

It looks like there is an error but it process file without reporting it and status is run

This might be a minor bug in the SD card streaming code, since the program is short and not terminated with M30 it will report PgmEnd when all the commands has been buffered. Add M30 as the last command and it will work as expected.

luc-github commented 2 years ago

Perhaps that would work in C# too -

yes C# support regex for what I can see https://learn.microsoft.com/en-us/dotnet/api/system.text.regularexpressions.regex?view=net-6.0 but it may depend of your version of .Net

If regex could be used then the current mask can be converted programmatically?

I do not know it is possible easily

There has been requests to remap ABC axes to UVW, e.g foam cutters use U instead of A (and some V as well?). The next build will allow remapping, so it is coming. Also, the grblHAL core supports up to 8 axes, XYZABCUV.

if this is coming it will be integrated in webUI for sure - in that case yes better to provide the letters and I will adjust to use these letters instead of XYZABC

Also, the grblHAL core supports up to 8 axes, XYZABCUV.

Ok so I will extend to 8 in it is not an issue

You forgot to reenable polling in preferences.json? I have missed that a few times when testing.

Sorry I do not understand does enabling auto report from grblHAL need more than put WebUI Auto report Interval to 3 => 3sec ?

what preferences.json are you refering to ?

M30

Sorry I did not know need M30 at the end of file - I am still newbie on cnc/grbl, I will test and report

terjeio commented 2 years ago

You forgot to reenable polling in preferences.json? I have missed that a few times when testing.

Sorry I do not understand does enabling auto report from grblHAL need more than put WebUI Auto report Interval to 3 => 3sec ?

If auto reporting is enabled in grblHAL it should be disabled in the WebUI preferences.json and vice versa:

  "enablepolling": true,
  "pollingcmds": [
   {
    "id": "samplepolling",
    "name": "3s",
    "cmds": "?;[ESP701]json",
    "refreshtime": "3000"
   }

Either enablepolling should be changed accordingly or samplepolling should if there are other polling commands present? This has to be done manually now, ideally this should be done automatically based on a flag in the ESP800 report?

In a similar vein, perhaps "Show machine settings tab" in the Interface settings should be removed for grblHAL?

Sorry I did not know need M30 at the end of file - I am still newbie on cnc/grbl, I will test and report

It is bad practice not to terminate a gcode program as this might leave the spindle and coolant on, and the controller in an unknown state. M30 (and M2) stops the spindle, switches off coolant and resets the controller to a known state.

The SD card streaming code will report 99.9 percent finished for a file when processing queued commands (after finishing streaming and closing the file), this until it receives a program end event. If percent complete is to reflect what has actually been completed line numbers has to be added to files and reporting changed to use that. Since line numbers are seldom used these days I have not bothered to add that option.

Here is a larger test program I use all the time:

Landscape.zip

luc-github commented 2 years ago

If auto reporting is enabled in grblHAL it should be disabled in the WebUI preferences.json and vice versa:

yes it is disabled in webUI and enabled in feature settings and board even been restarted - but I got no ? report

In a similar vein, perhaps "Show machine settings tab" in the Interface settings should be removed for grblHAL

This is a settings - it is not frozen as I wanted to see the weight of adding grblHAL custom settings on global [ESP400] parser, currently it add 700 Bytes to global package of Marlin for example - if we reach 1KB I will have to change the architecture to move Generic ESP400 parser / controls to each Firmware target - I am still thinking to a solution for this

Either enablepolling should be changed accordingly or samplepolling should if there are other polling commands present? This has to be done manually now, ideally this should be done automatically based on a flag in the ESP800 report?

You do not have to edit preferences.json all is done in WebUI interface, also you can put any other commands not present in auto reporting per need - indeed ? command could be disabled if auto reporting is enabled, that is possible

I think I found the problems:

I am testing now the job processing in ? report and it is fine but pause / stop commands seems causing trouble or being ignored: With my file + M30 at the end nothing stop

<Idle|MPos:0.000,0.000,0.000|Bf:35,1023|FS:0,0|Pn:PXYZS>
$F=/grbltest.nc
ok
<Run|MPos:1.388,2.776,0.000|Bf:17,1023|FS:100,0|Pn:PXYZS|SD:99.7,grbltest.nc>
<Run|MPos:3.624,7.248,0.000|Bf:17,1023|FS:100,0|Pn:PXYZS|WCO:0.000,0.000,0.000|SD:99.7,grbltest.nc>
<Run|MPos:5.860,11.720,0.000|Bf:17,1023|FS:100,0|Pn:PXYZS|Ov:100,100,100|SD:99.7,grbltest.nc>
<Run|MPos:8.096,16.192,0.000|Bf:17,1023|FS:100,0|Pn:PXYZS|SD:99.7,grbltest.nc>
#FEEDHOLD#
<Hold:0|MPos:10.500,20.000,0.000|Bf:18,1023|FS:78,0|Pn:PXYZS|SD:99.7,grbltest.nc>
[MSG:]
<Run|MPos:15.276,20.000,0.000|Bf:18,1023|FS:100,0|Pn:PXYZS|SD:99.7,grbltest.nc>
<Run|MPos:20.000,19.840,0.000|Bf:19,1023|FS:100,0|Pn:PXYZS|SD:99.7,grbltest.nc>
<Run|MPos:20.000,14.840,0.000|Bf:19,1023|FS:100,0|Pn:PXYZS|SD:99.7,grbltest.nc>
<Run|MPos:19.948,10.000,0.000|Bf:20,1023|FS:100,0|Pn:PXYZS|SD:99.7,grbltest.nc>
[ESP701]action=PAUSE
<Run|MPos:14.960,10.000,0.000|Bf:20,1023|FS:100,0|Pn:PXYZS|SD:99.7,grbltest.nc>
<Run|MPos:10.020,10.000,0.000|Bf:21,1023|FS:60,0|Pn:PXYZS|SD:99.7,grbltest.nc>
<Run|MPos:14.760,10.000,0.000|Bf:21,1023|FS:100,0|Pn:PXYZS|SD:99.7,grbltest.nc>
<Run|MPos:19.760,10.000,0.000|Bf:21,1023|FS:81,0|Pn:PXYZS|SD:99.7,grbltest.nc>
<Run|MPos:15.520,10.000,0.000|Bf:22,1023|FS:100,0|Pn:PXYZS|SD:99.7,grbltest.nc>

With your file :

<Idle|MPos:0.000,0.000,0.000|Bf:35,1023|FS:0,0|Pn:PXYZS|Ov:100,100,100>
<Idle|MPos:0.000,0.000,0.000|Bf:35,1023|FS:0,0|Pn:PXYZS>
$F=/Landscape.nc
ok
<Run|MPos:2.832,2.536,0.000|Bf:0,1023|FS:190,500|Pn:PXYZS|A:S|SD:0.4,Landscape.nc>
<Run|MPos:3.300,4.888,0.000|Bf:0,1023|FS:132,500|Pn:PXYZS|SD:0.5,Landscape.nc>
[ESP701]action=PAUSE
<Run|MPos:5.804,6.576,0.000|Bf:0,1023|FS:90,500|Pn:PXYZS|SD:0.8,Landscape.nc>
<Run|MPos:6.452,5.180,0.000|Bf:0,1023|FS:125,500|Pn:PXYZS|SD:1.0,Landscape.nc>
<Run|MPos:8.136,5.132,0.000|Bf:0,1023|FS:190,500|Pn:PXYZS|SD:1.4,Landscape.nc>
#FEEDHOLD#
<Hold:0|MPos:8.580,6.936,0.000|Bf:1,1023|FS:96,500|Pn:PXYZS|A:S|SD:1.5,Landscape.nc>
<Hold:0|MPos:8.580,6.936,0.000|Bf:1,1023|FS:96,500|Pn:PXYZS|A:S|SD:1.5,Landscape.nc>
[ESP701]json

also in HOLD mode it seems websocket is disconnected - I know ? / command is not possible but even not streaming, the HOLD commands seems catched ,because I see Hold mode for a while, then seems back to idle

<Idle|MPos:0.000,0.000,0.000|Bf:35,1023|FS:0,0|Pn:PXYZS>
<Idle|MPos:0.000,0.000,0.000|Bf:35,1023|FS:0,0|Pn:PXYZS>
#FEEDHOLD#
[MSG:]
<Idle|MPos:0.000,0.000,0.000|Bf:35,1023|FS:0,0|Pn:PXYZS>
<Idle|MPos:0.000,0.000,0.000|Bf:35,1023|FS:0,0|Pn:PXYZS>
#FEEDHOLD#
<Hold:0|MPos:0.000,0.000,0.000|Bf:35,1023|FS:0,0|Pn:PXYZS>
[MSG:]
<Idle|MPos:0.000,0.000,0.000|Bf:35,1023|FS:0,0|Pn:PXYZS>
<Idle|MPos:0.000,0.000,0.000|Bf:35,1023|FS:0,0|Pn:PXYZS>
<Idle|MPos:0.000,0.000,0.000|Bf:35,1023|FS:0,0|Pn:PXYZS>
#FEEDHOLD#
[MSG:]
<Idle|MPos:0.000,0.000,0.000|Bf:35,1023|FS:0,0|Pn:PXYZS>
<Idle|MPos:0.000,0.000,0.000|Bf:35,1023|FS:0,0|Pn:PXYZS|WCO:0.000,0.000,0.000>
<Idle|MPos:0.000,0.000,0.000|Bf:35,1023|FS:0,0|Pn:PXYZS|Ov:100,100,100>
terjeio commented 2 years ago

yes it is disabled in webUI and enabled in feature settings and board even been restarted - but I got no ? report

Do you see the request in the network traffic log?

BTW when try to feedhold I get an empty command request in the network log (no content length), this was working before for me - a regression?

image

<Run|MPos:3.300,4.888,0.000|Bf:0,1023|FS:132,500|Pn:PXYZS|SD:0.5,Landscape.nc>
[ESP701]action=PAUSE
<Run|MPos:5.804,6.576,0.000|Bf:0,1023|FS:90,500|Pn:PXYZS|SD:0.8,Landscape.nc>

The S (cycle Start) input is asserted (from Pn:PXYZS above), is this input left floating? Does it help to ground or invert it it?

it seems the [ESP444]RESTART do actually a reset and not board restart so settings was not take in account - could it be the issue ?

The HAL reboot entry point is not set for ESP32, I just checked and it no longer crashes the controller permanently so can be reenabled. Will do so in the next build - you can uncomment for now.

luc-github commented 2 years ago

If not run nothing happen as in idle mode - but I see command image

if run and streaming image

but on serial I have:

<Idle|MPos:0.000,0.000,0.000|Bf:35,1023|FS:0,0|Pn:PXYZS>
<Run|MPos:1.732,1.552,0.000|Bf:0,1023|FS:391,500|Pn:PXYZS|A:S|SD:0.4,Landscape.nc>
<Run|MPos:3.300,4.904,0.000|Bf:0,1023|FS:0,500|Pn:PXYZS|SD:0.5,Landscape.nc>
<Run|MPos:5.684,6.972,0.000|Bf:0,1023|FS:126,500|Pn:PXYZS|SD:0.8,Landscape.nc>
<Hold:0|MPos:4.896,7.812,0.000|Bf:1,1023|FS:94,500|Pn:PXYZS|A:S|SD:0.9,Landscape.nc>

***ERROR*** A stack overflow in task Grbl has been detected.

Backtrace:0x400817fe:0x3ffb48d0 0x4008acb9:0x3ffb48f0 0x4008ddba:0x3ffb4910 0x4008c624:0x3ffb4990 0x4008adb0:0x3ffb49c0 0x4008ad62:0x00000000 |<-CORRUPTED

ELF file SHA256: 7269fae0e79488e7

CPU halted.

so it seems there is a crash

luc-github commented 2 years ago

Here latest version of webUI : index.html.gz

luc-github commented 2 years ago

the stack decoding does not help a lot... image

terjeio commented 2 years ago

Ok, I'll test with the latest. What is strange is that the STM32F756 I use for development/debugging works - did/do you send the feed hold command via the websocket and an empty http request as well?

terjeio commented 2 years ago

I see now that RT commands come in via the websocket (unlike before?). When I ensure both the feed hold and cycle start is not asserted (H and S does not show up in the |Pn element) it works with the ESP32. This is a bit odd as the ESP32 board I test with has optocoupled inputs and should be relatively noise proof. The driver should only inform the core when there is a state change for these inputs so something is amiss...

terjeio commented 2 years ago

Yep - something is triggering the control input interrupts (the radio?), wonder if this is one of the silicon bugs that affects the MCU. I'l have to check this again - IIRC the something was an AD converter? Or I need stronger pull ups when the optocouplers is not pulling the associated inputs low? Anyway, grounding the feed hold and cycle start pins should help - start with that.

luc-github commented 2 years ago

No no the command is only send by http - I just echo the command in terminal for user readibility

terjeio commented 2 years ago

No no the command is only send by http - I just echo the command in terminal for user readibility

Oops - I put the debug message in the wrong place. But how come some http cmd's have no content length?

Anyway, I have just commited an update with the latest changes to ESP400 and 800 and a workaround for the silicon bug generating spurious pin 36/39 interrupts.

luc-github commented 2 years ago

Anyway, grounding the feed hold and cycle start pins should help - start with that.

yes it is already as you told me to do before rn_image_picker_lib_temp_553747f3-9409-4b03-ad70-d0e44d322957

luc-github commented 2 years ago

Anyway, I have just commited an update with the latest changes to ESP400 and 800 and a workaround for the silicon bug generating spurious pin 36/39 interrupts.

Ok will check tomorrow - bed time for me now 😴

luc-github commented 2 years ago

E and U flags are now supported : image

I have added a tooltip on unit area to let user to see what is expected precision

To clarify : the choice to not put specific error message for each possible input error is just due to fact that the index.html must be as small as possible and putting all possible cause of error text will make the final package big for minimal benefit - it also make every translation pack bigger

So I choose a generic error message incorrect value to keep final footprint as lowest as possible

luc-github commented 2 years ago

The hold issue seems solved with latest git : using control commands is working now as expected but [ESP701]action=PAUSE and [ESP701]action=ABORT commands seems not working - only [ESP700]json status is working Should I remove the corresponding buttons ?

also I got [MSG:] when sending Cycle Start command is that normal ?

<Idle|MPos:0.000,0.000,0.000|Bf:35,1023|FS:0,0|Pn:PXYZS|WCO:0.000,0.000,0.000>
<Idle|MPos:0.000,0.000,0.000|Bf:35,1023|FS:0,0|Pn:PXYZS|Ov:100,100,100>
<Idle|MPos:0.000,0.000,0.000|Bf:35,1023|FS:0,0|Pn:PXYZS>
<Idle|MPos:0.000,0.000,0.000|Bf:35,1023|FS:0,0|Pn:PXYZS>
$F=/Landscape.nc
ok
<Run|MPos:0.300,0.272,0.000|Bf:0,1023|FS:265,500|Pn:PXYZS|A:S|SD:0.4,Landscape.nc>
<Run|MPos:3.300,3.908,0.000|Bf:0,1023|FS:226,500|Pn:PXYZS|SD:0.5,Landscape.nc>
<Run|MPos:4.848,7.140,0.000|Bf:0,1023|FS:141,500|Pn:PXYZS|SD:0.7,Landscape.nc>
[ESP701]action=PAUSE
<Run|MPos:6.500,6.300,0.000|Bf:0,1023|FS:73,500|Pn:PXYZS|SD:0.9,Landscape.nc>
<Run|MPos:9.152,5.160,0.000|Bf:0,1023|FS:58,500|Pn:PXYZS|SD:1.3,Landscape.nc>
<Run|MPos:8.544,6.236,0.000|Bf:0,1023|FS:155,500|Pn:PXYZS|SD:1.5,Landscape.nc>
<Run|MPos:5.328,9.768,0.000|Bf:0,1023|FS:191,500|Pn:PXYZS|WCO:0.000,0.000,0.000|SD:1.8,Landscape.nc>
<Run|MPos:10.396,9.836,0.000|Bf:0,1023|FS:105,500|Pn:PXYZS|Ov:100,100,100|A:S|SD:2.1,Landscape.nc>
<Run|MPos:12.752,6.336,0.000|Bf:0,1023|FS:166,500|Pn:PXYZS|SD:2.5,Landscape.nc>
<Run|MPos:12.404,8.672,0.000|Bf:0,1023|FS:173,500|Pn:PXYZS|SD:2.7,Landscape.nc>
<Run|MPos:11.112,7.292,0.000|Bf:0,1023|FS:203,500|Pn:PXYZS|SD:3.0,Landscape.nc>
<Run|MPos:15.180,6.368,0.000|Bf:0,1023|FS:230,500|Pn:PXYZS|SD:3.3,Landscape.nc>
[ESP701]json
{
 "cmd": "701",
 "status": "ok",
 "data": {
  "status": "processing",
  "total": "188318",
  "processed": "6403",
  "type": "SD",
  "name": "Landscape.nc"
 }
}
<Run|MPos:17.288,5.372,0.000|Bf:0,1023|FS:170,500|Pn:PXYZS|SD:3.6,Landscape.nc>
<Run|MPos:19.712,4.816,0.000|Bf:0,1023|FS:380,500|Pn:PXYZS|SD:3.8,Landscape.nc>
<Run|MPos:15.016,9.384,0.000|Bf:0,1023|FS:122,500|Pn:PXYZS|SD:3.9,Landscape.nc>
<Run|MPos:18.208,4.788,0.000|Bf:0,1023|FS:153,500|Pn:PXYZS|SD:4.3,Landscape.nc>
#FEEDHOLD#
<Hold:0|MPos:23.336,4.824,0.000|Bf:0,1023|FS:80,500|Pn:PXYZS|SD:4.6,Landscape.nc>
<Hold:0|MPos:23.336,4.824,0.000|Bf:0,1023|FS:80,500|Pn:PXYZS|SD:4.6,Landscape.nc>
<Hold:0|MPos:23.336,4.824,0.000|Bf:0,1023|FS:80,500|Pn:PXYZS|SD:4.6,Landscape.nc>
<Hold:0|MPos:23.336,4.824,0.000|Bf:0,1023|FS:80,500|Pn:PXYZS|SD:4.6,Landscape.nc>
<Hold:0|MPos:23.336,4.824,0.000|Bf:0,1023|FS:80,500|Pn:PXYZS|SD:4.6,Landscape.nc>
<Hold:0|MPos:23.336,4.824,0.000|Bf:0,1023|FS:80,500|Pn:PXYZS|SD:4.6,Landscape.nc>
<Hold:0|MPos:23.336,4.824,0.000|Bf:0,1023|FS:80,500|Pn:PXYZS|SD:4.6,Landscape.nc>
<Hold:0|MPos:23.336,4.824,0.000|Bf:0,1023|FS:80,500|Pn:PXYZS|SD:4.6,Landscape.nc>
#CYCLESTART#
[MSG:]
<Run|MPos:30.672,6.656,0.000|Bf:0,1023|FS:146,500|Pn:PXYZS|A:S|SD:4.6,Landscape.nc>
<Run|MPos:29.740,7.808,0.000|Bf:0,1023|FS:124,500|Pn:PXYZS|SD:4.9,Landscape.nc>
<Run|MPos:36.104,6.156,0.000|Bf:0,1023|FS:76,500|Pn:PXYZS|SD:5.1,Landscape.nc>
...
<Run|MPos:25.756,25.784,0.000|Bf:0,1023|FS:56,500|Pn:PXYZS|WCO:0.000,0.000,0.000|SD:20.8,Landscape.nc>
<Run|MPos:27.872,24.940,0.000|Bf:0,1023|FS:304,500|Pn:PXYZS|Ov:100,100,100|A:S|SD:21.0,Landscape.nc>
[ESP701]action=ABORT
<Run|MPos:37.044,23.836,0.000|Bf:0,1023|FS:77,500|Pn:PXYZS|SD:21.7,Landscape.nc>
<Run|MPos:29.136,24.736,0.000|Bf:0,1023|FS:47,500|Pn:PXYZS|SD:22.3,Landscape.nc>
<Run|MPos:35.032,25.988,0.000|Bf:0,1023|FS:263,500|Pn:PXYZS|SD:22.9,Landscape.nc>
<Run|MPos:28.884,26.488,0.000|Bf:0,1023|FS:234,500|Pn:PXYZS|SD:23.8,Landscape.nc>
<Run|MPos:26.720,26.384,0.000|Bf:0,1023|FS:180,500|Pn:PXYZS|SD:24.1,Landscape.nc>
#SOFTRESET#
[MSG:Reset during streaming of SD file at line: 2258]
ok
ALARM:3
GrblHAL 1.1f ['$' or '$HELP' for help]
[MSG:'$H'|'$X' to unlock]
<Alarm|MPos:28.652,22.272,0.000|Bf:35,1023|FS:0,0|Pn:PXYZS|Ov:100,100,100>
<Alarm|MPos:28.652,22.272,0.000|Bf:35,1023|FS:0,0|Pn:PXYZS>
<Alarm|MPos:28.652,22.272,0.000|Bf:35,1023|FS:0,0|Pn:PXYZS>
<Alarm|MPos:28.652,22.272,0.000|Bf:35,1023|FS:0,0|Pn:PXYZS>
<Alarm|MPos:28.652,22.272,0.000|Bf:35,1023|FS:0,0|Pn:PXYZS>
<Alarm|MPos:28.652,22.272,0.000|Bf:35,1023|FS:0,0|Pn:PXYZS>
$X
[MSG:Caution: Unlocked]
ok
<Idle|MPos:28.652,22.272,0.000|Bf:35,1023|FS:0,0|Pn:PXYZS>
...

The latest build is here : https://github.com/luc-github/ESP3D-WEBUI/blob/3.0/dist/CNC/GRBLHal/index.html.gz

terjeio commented 2 years ago

Settings rendering is looking really good now! However, there is a browser hang (Firefox) when trying to change Network Services.

image

but [ESP701]action=PAUSE and [ESP701]action=ABORT commands seems not working - only [ESP700]json status is working Should I remove the corresponding buttons ?

My bad, I test for lowercase version of the actions strings. Are they defined as case insensitive so I should uppercase before testing?

also I got [MSG:] when sending Cycle Start command is that normal ?

Yes, I added that as I had problems with removing status line messages in ioSender at the correct point in time, IIRC tool change related messages. Sending a blank message was an easy way to fix it and should not break any senders.

luc-github commented 2 years ago

However, there is a browser hang (Firefox) when trying to change Network Services.

Do you mean these settings ? I cannot duplicate on Firefox / windows 11 / i7 neither firefox/linuxMint/i5

image

Do you have more details ?

terjeio commented 2 years ago

I have WebDAV enabled, e.g. when I change the Telnet setting WebDAV changes too and I get the hang. Note that the settings flags for masks might not be consecutive, if the corresponding functionality is not enabled flags will be left out.

image

luc-github commented 2 years ago

I think it is duplicate ID problem I will recompile grblHAL with same settings as you to verify

luc-github commented 2 years ago

Hmm I do not know if it is an issue but I see a problem: Where are the bits 4 -5 -6 ? I have a definition issue here

Currently I check Option size = size of the mask here 5

if you use index out of the array I may be in trouble, can you clarify the behavior ? should I look for max index to know the size of the integer array instead of the options array size itself?

{
   "F": "Networking/Networking",
   "P": "70",
   "T": "M",
   "V": "7",
   "H": "Network Services",
   "R": "1",
   "O": [
    {
     "Telnet": "0"
    },
    {
     "Websocket": "1"
    },
    {
     "HTTP": "2"
    },
    {
     "FTP": "3"
    },
    {
     "WebDAV": "7"
    }
   ]
  },

I reproduce the hang only if I change webdav actually