search

luc-github / ESP3D-WEBUI

A Web UI for ESP8266 or ESP32 based boards connected to 3D printers / CNC
GNU General Public License v3.0
766 stars 305 forks source link

[REFACTORING]3.0 Preact refactoring->3.0.a #157

Closed luc-github closed 2 years ago

luc-github commented 3 years ago

ESP3D-WBUI 3.0 is using Preact and bootstrap 4.0 Current preact implementation is to be honnest,not really well done - that is my first Preact project and I have a lack of Preact knowledge - so I should consider it as a draft even it is currently workable.

So a rewrite is definitly necessary: @alxblog did a great start on POC It is based on spectre which seems lighter than bootstrap but meet all needs of ESP3D-WEBUI and even more.

The POC approach was to start a full rewrite to not be limited, but that is a huge task and unfortunatly @alxblog seems busy with others things - when on another side my preact knowledge is still pretty low 😊 But several approaches are really the way the webui need to gp:

luc-github commented 3 years ago

OK I have started several online trainings for web development - so this month I will focus on them

luc-github commented 3 years ago

quick update as ESP3D activity is slowed down recently

I am still following webDev training - I have ordered 30 days for all trainings, and actually it is like pulling infos and realizing at each new training, I need more training orz...^_^

I have already followed CSS/HTML (110 lessons) then Javascript (standard and advanced, 198 lessons) and definitly I was outdated 😓

I am now on NodeJS training part (standard/advanced, 224 lessons), then I will follow react training (standard/advanced, 195) and may be will also follow the typescript training (58 lessons) TBC

To pratice I am rewriting the embedded page which is simple but use several bases of code for webUI I am trying to fit spectre POC UI for consistency : webpack I will include the esp3d emulator and offline test currently I am around 2KB by disabling the transpiling as the target will be modern browser and I do not use any new javascript feature

I do not think I can finish all training within the registered month (now 10 days left and still a lot to learn) so I will register for additional month.

I hope results will be visible soon ^_^ , so thank you for your patience

luc-github commented 3 years ago

Update - busy but still ongoing on rewrite here current embedded vs new embedded webUI : webui version Final size should be almost same as current but build process using webpack and E06 allows test version, I am trying to simplify and keep a consistent UI with coming WebUI 3.0

luc-github commented 3 years ago

The new embedded page is now part of ESP3D, the build process use webpack and so it will be easier to keep build libraries up to date unlike today, the UI is inpired from spectre css and I have tried to simplified a much as I could.

It has same features a latest embdedded page but refactored code.

The offline test is now available with simple ESP3D emulator, not yet the one done by @alxblog because it is still under devt the gain in size is 10% 6,55 KB => 5,94 KB The webpack build process cover :

So in embedded directory:

Note: the tool bin2c is still necessary to convert the binary file to a text file to include in sources

Feel free to test and comment, there is always area of improvement for sure 😸

Now back to react / preact training

luc-github commented 3 years ago

Some update: The rewrite using proper Preact coding method is ongoing , most of work is based on @alxblog POC that I am studying to apply properly - thank to his patience for all my questions and for completing my training on PreactJs 😅 The UI is based on PreactJs / Spectre.css - we keep general flow of (3.0) but simplify several parts which make it lighter

I am still on WebUi API itself implementation (settings management / websocket support / error rendering / modal dialogs implementation / code splitting, etc...)- not yet on real pages rendering

Here some ongoing test with menu implemented and http call test for the about page FYR:

screen2

luc-github commented 3 years ago

Now connection / disconnection pages will look like this

empty

luc-github commented 3 years ago

Update : image

Current size of 3.0 rewrite is 30.6KB at same state 3.0 was 35.7KB,

UI update : 3.0.0.a4 : 30.6KB | 0.0.0.6 : 35.7KB

but rewrite has already authentication support as well as all translations, connection status, mobile view, so benefit of rewrite is already noticeable even some refactoring could still be done but I am limited with with my current Preact knowledge This looks definitely very promising 😺

ETE-Design commented 3 years ago

@luc-github This is so great, and seems like you have also added Update FW Support... Is this only working for ESP3D or also 3D Printer Mainboard?

luc-github commented 3 years ago

Update ESP3D FW update is supported in all webUI version about printer FW update support: only ESP32 based boards are supported (GRBL_ESP32, Marlin with esp3dLib) Anyway, on recent 32bits boards, usually it just need to be present on SD and restart printer board, so this should be covered when https://github.com/luc-github/ESP3D/issues/575 will be implemented, but it will also need printer FW code modification, so not for short term I think.

ETE-Design commented 3 years ago

Update ESP3D FW update is supported in all webUI version about printer FW update support: only ESP32 based boards are supported (GRBL_ESP32, Marlin with esp3dLib) Anyway, on recent 32bits boards, usually it just need to be present on SD and restart printer board, so this should be covered when luc-github/ESP3D#575 will be implemented, but it will also need printer FW code modification, so not for short term I think.

Okey, have an MKS Robin Lite v1.1... So Wifi is possible with a addon module... So if you need help testing the FW Uodate for Mainboard I can help on that part... Also know Marlin, so what need yo be added for it to work?

luc-github commented 3 years ago

on MKS when theoricaly it is possible - in reality : 1 - MKS marlin coderename the uploaded file to dos filenname with .GCO extension 2 - MKS bootloader use Robin_nano35.bin only for FW update

So need fix in bootloader to support dos name as Marlin only support Dos name to write on SD, and prevent Marlin code to rename file extension if .bin

Both are out of ESP3D Scope sorry

luc-github commented 3 years ago

I was busy with some IT issues ( my Nas died, Travis become slow, etc..) and some personnal things kept me busy Here the first part of Features settings UI implementation

features-1

Still need to implement the save / validation / import and export / restart and wifi AP scan dialog, but rendering is now more consistent . Some features will also need so code update in [ESP400] output format, but currently I have only modified the test server part

I hope this part to be done next week

luc-github commented 3 years ago

the features tab is done - it has some new features that need an update of ESP3D FW it allow to do all changes then save once, FW also now send information about necessary restart or not features

Next step is settings interface tab - which will follow features tab style

ETE-Design commented 3 years ago

@luc-github Seem's really nice, is it also possible to add SSID / Security Options when network is hidden?

luc-github commented 3 years ago

what do you mean ? when Network is hidden you must type SSID and password - what is different ?

luc-github commented 3 years ago

Update, the web UI settings page is not almost done, it still miss: 1 - proper language selection, based on what language pack is available 2 - proper theme selection, based on selected pack 3 - Macro editor 4 - extra panel / page editor

I will add features in above order

luc-github commented 3 years ago

The theme and language support by external pack is now implemented, for the language packs, UI still need to be finished to validate the list of string to translate - so current language packs are just for POC but API is ready For theme support - it is currently limited to css, a POC of Dark mode is done as well as another purple theme just as example, I suggest to wait the UI is finalized before doing a final theme but feel free to play

theme

luc-github commented 3 years ago

Some update : I got busy and some family things got my attention, which also result of a big lack of motivation for a couple of month. So the development was really slow down, I also found a nasty bug that fried my brain but it is now fixed, so next week I should be back on track and focus on webUI - the macro / extra panels configuration code should be implemented soon, code preparation and base is ready - now need to do the implementation. :crossed_fingers:

luc-github commented 3 years ago

Macros and Panels configuration are now implemented in settings, I have removed the color button customization because it is actually useless for panel / page, and it simplify the customization UI and the preferences.json file, also it may complexify the code at it would need also a push state color and hover state color that should be calculated, using space that could be used for better usage, and finally it may be weird with theme management as not take in account at all.

image image

The error conditions in extrapanels / macros settings are not yet implemented because not yet well defined, TBD...

Next steps are more functionnals because they will use settings: 1- implement polling commands (:+1: done) 2 - implement the extra pages display 3 - implement terminal panel in dashboard page 4 - implement jog panel in dashboard page

luc-github commented 3 years ago

Update of UI with some conditions added settings

DeeEmm commented 3 years ago

Luc, do you have any view on a size comparison between V3 and 2.1?

How does the size of deployed code for the configuration interface compare to 2.1?

luc-github commented 3 years ago

No I did not do size tracker when doing 2..x , but 3.x refactoring work in more generic way than 2.x and bring more features

Refering final package size: 2.1 english only is 106kb, 3.0 english only is 95.6kb 3.0 refactored should be 10kb smaller Ithan initial 3.0 I hope, when 3.0 has more features than 2.1 and refactored version has also few more features than 3.0

in all case the target is to have smaller size between 2.0 vs 3.0

DeeEmm commented 3 years ago

Thanks Luc.

I recall some discussions about the 3.0 interface being modular, does this mean that for a CNC application we will not need to include the 3D printing components like temperature graphing and extruder control? (and vice versa) which saves space for other CNC specific features, or is there just a single deployment for all machines?

Just trying to get my head around the structure of the new code and potential available space for writing new features.

luc-github commented 3 years ago

yes cnc and 3Dprinting are now splited avoiding to increase size for not used features and also allowing different UI between cnc and 3Dprinting, that said cnc ui is not yet done in 3.0

luc-github commented 3 years ago

2 - implement the extra pages display (finally done :+1:) got some nasty bugs, which drive me crazy, but finally I understood my mistakes and fixed them :sweat: Next: 3 - implement terminal panel in dashboard page, and panel menu as sugested by @alxblog to have more generic UI Clicking on menu item will make appear the panel if it was closed, and also put in top position just under menu, this will avoid any scrolling when want quick access to panel like in current 3.0 - the change is the drop down menu vs a button just for terminal and Files list

4 - Implement FS file list panel 5 - implement jog panel in dashboard page

ETE-Design commented 3 years ago

@luc-github Is ESP32-Cam now implemented, with features you think is needed for it...

luc-github commented 3 years ago

what you mean by all features ?

ETE-Design commented 3 years ago

@luc-github Like send files direct to it, and when done parse it over to the sd-card in the printer... This should speed up file transfer right? Also Timelapse Screenshot mabye?

luc-github commented 3 years ago

what is done in webui rewrite of 3.0 is what it is described in this thread , so no as you see terminal is not yet implemented, other webui features will follow as described, if not described yet, it means not done yet in rewrite

about snapshot it can be added but I do not see how to manage timelapse for the moment, streaming is managed by webui not esp3d. being gcode host is not done yet in ESP3D and not related to web UI

luc-github commented 3 years ago

I have just fixed several glitches when doing user case tests and also implemented the hold /stream button as well as the snapshot button image Back to dev now

luc-github commented 3 years ago

I have almost done the information flow to filter / render data flow and dispatch accordingly The Terminal is not yet finished but send command and receive answer is working well, the extension support is also working well so new feature added 😸 I did some test on OSX/Safari and when I have no issue on firefox / chrome /opera - wel....l safari need some tuning, several css need to be added for what I can see - I have issues with SVG crashing the dropdown menu and generating errors when displayed....and there are a lot of svg in Ui...

I did test on my 2011 / Sierra mac air, so let say: he is slow like hell and I cannot debug in such conditions. I will try to setup a virtual Machine because better to clean up the rendering problem before going further and redo lot ot things.

So originally I expected to finish the terminal this week but as in Safari I cannot even drop down the panel menu..., I think It will be postponed to next week. in 2.1 / 3.0 I did some ugly hack for safari that I would like to avoid from now.

Just hope can do a proper virtual machine ASAP as I cannot afford to buy a new Mac 😢 Let see what can be done - and finger crossed no hard issues will come 🤞

luc-github commented 3 years ago

Ok finally setup the OSX virtual machine took longer than fix the css issues, I did not do anything about SVG as the error actually are not seen but just catched by console so... the terminal panel is now done, I still have some work on data processing and filtering but it is working, Dashboard

The extensions feature is also implemented (https://github.com/luc-github/ESP3D-WEBUI/issues/185 )- it should allow to add additionnal features easily :extension

DeeEmm commented 3 years ago

Great work Luc,

I'm guessing that at this stage there's no documentation on how to create an extension, but is there an existing extension that would serve as a good example to take a look at?

/DM

On 22 Aug 2021, at 6:01 pm, Luc @.***> wrote:

Ok finally setup the OSX virtual machine took longer than fix the css issues, I did not do anything about SVG as the error actually are not seen but just catched by console so... the terminal panel is now done, I still have some work on data processing and filtering but it is working, https://user-images.githubusercontent.com/8822552/130347782-0e2647ca-e0f1-4eb1-bd07-aad162ba2281.gif The extensions feature is also implemented (#185 https://github.com/luc-github/ESP3D-WEBUI/issues/185 )- it should allow to add additionnal features easily : https://user-images.githubusercontent.com/8822552/130347887-36908557-6f4c-45f0-b6af-5c0211a931db.gif — You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/luc-github/ESP3D-WEBUI/issues/157#issuecomment-903233432, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAXF35VKP7B2BV6NSTS75PTT6CYWBANCNFSM4WT57DNQ. Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email.

luc-github commented 3 years ago

Check from here https://github.com/luc-github/ESP3D-WEBUI/issues/185#issuecomment-899082944 sample code is also in server directory

DeeEmm commented 3 years ago

Looks good Luc,

I'll take a more in depth look this week.

Thanks for all your hard work :)

/DM

On 22 Aug 2021, at 6:14 pm, Luc @.***> wrote:

Check from here #185 (comment) https://github.com/luc-github/ESP3D-WEBUI/issues/185#issuecomment-899082944 sample code is also in server directory

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/luc-github/ESP3D-WEBUI/issues/157#issuecomment-903234866, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAXF35SUTZAS4DYLTIOIRS3T6C2GFANCNFSM4WT57DNQ. Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email.

luc-github commented 3 years ago

Extra panels are now part of dashboard image I got some random panel corruption on extension panel and kiri:moto panel when closing then opening panels, but it was solved by leaving / going back to dashboard - not sure what the issue is, because it does not happen on test server - so I am guessing some race issue...TBC

Next steps are the macro panel then the Filesystem panel

luc-github commented 3 years ago

Macro panel is now implemented image

the URI macro has now a new parameter to not show the window but act as an ajax command, just add [SILENT] in front of URL image

Current size is now 50.4KB when with same features 3.0 was 53.9KB.

Not only smaller, the rewrite has more features than the 3.0:

So rewrite benefit is definitly visible 😸 - it worse the work and time spend on it, 😅

I still have couple of issues these I need to handle due to my current limited knowledge of Preact, (getting better but still on learning curve...) :

So lot of things to do...

luc-github commented 3 years ago

Quick update,

Conclusion it is so far so good,😸

filesPanel

Next chalenge is to catch and display the files list from serial / websocket (e.g. M20) in a smooth way - it should be done for this week end 🤞

luc-github commented 3 years ago

Stream API catching is now implemented for Serial SD and should work with any others sources, TFT SD / TFT USB, etc.. Adding any additional FS : Shared SD, TFT SD / TFT USB (BTT and MKS) just need one definition file per new FS, but as all my devices are now packed for my moving and in luggages - I will add them next month when in new location.

streamcatching

After some code refactoring the size is now 53KB in rewrite for a equivalent of 57.1KB in 3.0 but with more feature in rewrite and better browser rendering - so still on good path.

Current code is now 99% Preact oriented and I did not saw any issue, all previous issues are now solved 🎉 One random behaviour I recently noticed is: Modal randomly take time to show the first time when select a large size file, I am not sure why this happen - TBA

EDIT: after dig in a little it seems delay is due to the onChange even of input file control that take time to be fired - so nothing that can be done because - may be a bug in Preact ?

next step are:

luc-github commented 3 years ago

Machine settings is now done for Marlin as POC MachineSettings

Next steps:

luc-github commented 3 years ago

Devt and Build scripts have been added, as well as backbone code based on Marlin and need to be updated accordingly per target and sub target accordingly: It is based on 3 main applications CNC / 3DPrinter / Sand Table, CNC may be splitted to laser / milling if necessary but as UI is not defined yet : https://github.com/luc-github/ESP3D-WEBUI/issues/144 - TBD

image

GRBL_ESP32 won't be supported but the new evolution will be : FluidNC Reprap connected to serial and reprap connected so spi (embedded) have been added as futur support

I removed MarlinKimbra (https://github.com/MKFirmware/MK4duo) because sadly it seems project is now no more sustained 😞, I did not followed what's happened, it was a nice Fw with unique features.

So it will give more time for Reprap support addition.

If any target is missing - feel free to highlight it 😸

luc-github commented 3 years ago

3D printer settings are on going: Smoothieware:

Marlin: image

Repetier: image

So far so good, but I have forgotten some features present in current 3.0 😞

luc-github commented 3 years ago

GRBL and FluidNC settings Tabs are now implemented, the FLuidNC yaml support is pretty basic and need more test - it follows wiki description https://github.com/bdring/FluidNC/wiki/FluidNC-Config-File-Overview It has some additional assumptions:

fluidNC

Here size of current version:

The Direct SD is not yet implemented and need to be added - but it should be same as localFS because it use same web query format and rendering

import / export of settings are not implemented but for FluidNC which use explicitly a File and it need to be uploaded, when other targets do not but use commands, so in spirit of saving space I won't implement them for the moment, as I am not sure there is a real nee for it, if you disagree - feel free to expose your reasons - I am open to discussion :smile_cat:

RepRap support is not yet done and will be later, same for SandTable because they are a sub of CNC, so just need a copy and past files.

Now I will work on Jog Panel with also position display in Notification area (desktop view) / Notification page (Mobile view), and eventually panel itself, because if my first wish to have a permanent notification area was the new rendering feature in UI 3.0- it seems no one care, and even it generate complain... - so it is may be a false good idea - TBC if I remove this feature and just use panel view as classic UIs

I am still locked in quarantine for a week and more and no system in hand but just dev boards - so my testing are limited to emulation server

EDIT: I will also to add some new features to extension like upload/download files , be able to target specific extension queries, and some variables definition support as suggested by @cotepat

luc-github commented 3 years ago

Extensions missing features are now implemented, documentation is available here: https://github.com/luc-github/ESP3D-WEBUI/blob/3.0-rewrite/extensions_sample/API.md and sample codes are also here: https://github.com/luc-github/ESP3D-WEBUI/blob/3.0-rewrite/extensions_sample I need to add a script that will minify and compress any extension in command line - should be done tomorrow

Edit: It is completed with additional fixes

luc-github commented 2 years ago

Jog Jog for Marlin is now implemented for Marlin printer3d I still need to add the positions control to information page but should be done today

Now POC is almost done for Jog, I will do it for all Printer3D and do the UI for CNC / Sandtable which is slightly different: no Pronterface UI = just buttons, support up to 6 axis, WCO and MCO, Zero buttons for each axis, no disable motor, no parking command seems requested in https://github.com/luc-github/ESP3D-WEBUI/issues/144, so it won't be added neither.

Edit: adding Jog Panel for Marlin version = 61.2 KB, with previous version calculation 58.8KB+9.4KB = 68.1KB, so benefit is there because more features and smaller size

luc-github commented 2 years ago

Jog Panel for GRBL is now done : image

The number of axis detection is dynamic and the axis homing command is set in settings, disable and stop command are configurable in settings, real time commands are usable in this configurable commands and are also show in terminal 

0x18 (ctrl-x) : Soft-Reset : #SOFTRESET#
0x84 : Safety Door #SAFETYDOOR#
0x85 : Jog Cancel #JOGCANCEL#

Feed Overrides
0x90 : Set 100% of programmed rate. #FO100#
0x91 : Increase 10% #FO+10#
0x92 : Decrease 10% #FO-10#
0x93 : Increase 1%  #FO+1#
0x94 : Decrease 1%  #FO-1#

Rapid Overrides
0x95 : Set to 100% full rapid rate. #RO100#
0x96 : Set to 50% of rapid rate.    #RO50#  
0x97 : Set to 25% of rapid rate.    #RO25#

Spindle Speed Overrides 10%->200%
0x99 : Set 100% of programmed spindle speed #SSO100#
0x9A : Increase 10% #SSO+10#
0x9B : Decrease 10% #SSO-10#
0x9C : Increase 1%  #SSO+1#
0x9D : Decrease 1%  #SSO-1#

0x9E : Toggle Spindle Stop #T-SPINDLESTOP#
0xA0 : Toggle Flood Coolant #T-FLOODCOOLANT#
0xA1 : Toggle Mist Coolant #T-MISTCOOLANT#

Next steps:

luc-github commented 2 years ago

I have added some control dependency API, so now in settings, to show/hide a control depending another control value is possible, by just adding the id of the reference control in UI json I have implemented only for input control based on boolean value of reference control and applied the filter only to feedrate of GRBL

{
        "id": "feedrate",
        "type": "group",
        "label": "CN9",
        "value": [
      {
        "id": "xyfeedrate",
        "depend":[
            {"id":"showx",
            "value": true},
            {"id":"showy",
              "value": true}
          ],
        "type": "number",
        "label": "XY",
        "min": "1",
        "append": "CN1",
        "value": "1000"
      },
      {
        "id": "zfeedrate",
        "depend":[
          {"id":"showz",
            "value": true}
          ],
        "type": "number",
        "label": "Z",
        "append": "CN1",
        "min": "1",
        "value": "1000"
      },

This will open possibility to more contextual UI in settings depedency

Next:

luc-github commented 2 years ago

3D printers / CNC / Sand Table are now at same level of code, variable POC in macro is ready - need to define the needed variables, for CNC / and Sand table the real time commands are part of variable List already next step is

add sensors display and add specifics panels:

luc-github commented 2 years ago

Project was not updated since end of November due to personal issue and travelling back to France, now I am back to Taiwan and ramp up to recover the delay. But as I am in quarantine hotel with limited test environment so please be patient.

As raised in discord, the direct SD (Marlin+ESP3DLIB /FluidNC) and Shared SD (all FW with proper HW ) support is not yet implemented, I will work on it. I do not plan to support M28/M29 file upload as it is only a source of complain and disappointment due to extremely slow speed for upload

then I will continue on specific panel implementation

ETE-Design commented 2 years ago

@luc-github If you use the "MKS Protocol" and of cause have an MKS Mainboard then the Transfer speed would be faster... Could you mabye just implement M28/M29 for us with that board - already use it for uploading and works fine here... There is a lot of MKS Mainboards around, and they still use that method on newest board with support for addon Wifi Module.