Implement AsyncWebServer instead of SyncWebServer

luc-github commented 8 years ago

Using the great library of me-no-dev https://github.com/me-no-dev/ESPAsyncWebServer - this would allow to have better experience instead of using one html file to split css and javascript files and decrease template file interpretation workload parsing only part of html

mkeyno commented 8 years ago

good luck @luc-github this would be best feature of ESP3D

luc-github commented 8 years ago

created a branch for this purpose : https://github.com/luc-github/ESP3D/tree/asyncwebserver will take opportunity to rewrite webinterface.cpp to use addHandler method instead of on method to split file in several classes/files for clarity

luc-github commented 8 years ago

Easier way is to do https://github.com/luc-github/ESP3D/issues/114 first to simplify coding

luc-github commented 8 years ago

Will use embedded authentication method - this should simplify the code also
As I can see page handles need to be rewrited because use AsyncWebServerRequest class

luc-github commented 7 years ago

Ok after testing (a lot) and rewriting/porting part of ESP3D code for async (https://github.com/luc-github/ESP3D/tree/asyncwebserver) I will put on hold for a while. Looks like library is not yet fully stable and because maintainer is busy with ESP32, so no idea when fixes would come, and async concept is not clear from code point of view for me, so I won't dig in

What I have found/observed so far : 1 - Multiple files upload is not working (issue open) mitigation would be to allow only 1 file upload 2 - Embedded authentication is not working under window 7 / IE 11 - but seems ok on chrome/firefox/windows safari, I did not tested android (issue open) mitigation would be to use my authentication method which do not rely on browser but IP 3 - Webserver is definitly faster but I have also randomly no display page / no answer, I cannot define reproductible steps to report issue properly, and looking at others open issues it is not a priority. 4 - SSDP seems not working - but I did not dig in yet, Netbios looks fine.

So Async library may not fully mature yet for ESP3D but very promising - it also gave me opportunity to redo some code with another approach that I will use in sync server, waiting for update on async

luc-github commented 7 years ago

Update 1 - Multiple files upload is not working (issue open) 2 - Embedded authentication is not working under window 7 / IE 11 - issue fixed by @me-no-dev - thank you 3 - Webserver is definitly faster but I have also randomly no display page / no answer, - this may be due to 5 connections limits and the number of files linked in html pages (2 js and one css) - seems better to combine them in one gzip files to limite the issue 4 - SSDP seems not working - This one is on me (:shame:) I forgot to enable the /description.xml with new request method

me-no-dev commented 7 years ago

if you have static files you might consider gzipping and putting them in progmem. That will speed up the interface immensely :) I convert gzipped file to progmem using https://gist.github.com/me-no-dev/f137a950ce6dedb641d427d8db6355d2

me-no-dev commented 7 years ago

some browsers seem to make more than one request at a time sometimes which grows connections to more than actually needed. Could also be the case.

me-no-dev commented 7 years ago

this method deals with uploads and detecting when one ends and another starts. In some cases with new lines at the end of the file you might confuse the method at some point and not detect the end of the file. If you find a bug there please report as you seem to have a good test case

luc-github commented 7 years ago

@me-no-dev thanks - actually the goal is to let user to customize the html pages without diging in FW, I have another request to be able to serve smoothie files (https://github.com/luc-github/ESP3D/issues/112) so in any way I may not control the files update / content but I can set some limitations

Thanks a lot for pointing me this function, I will dig in and report to you - I should confess I have started to looks at async code but I am ~~little bit~~ fully lost, even read me is very good.

Thanks a lot for taking time to come here, I know you are very busy with ESP32 ^_^ ( i cannot wait to play with it)

me-no-dev commented 7 years ago

btw I have Flashforge Dreamer 3D printer that I would like to test with your project :) Some commands are different but I have them somewhere here. It has WiFi but it sucks... will be nice to get a working one

luc-github commented 7 years ago

Yes Flashforge dreamer has wifi, and it sucks ? I though was good printer, I like enclosure and removable bed, I remember it is based on makerbot but should be easy to adapt (S3d can handle it so should be easy)- I have a Beaver II (also a makerbot clone) but I put a wifi SD on this one 😉

me-no-dev commented 7 years ago

the printer itself is amazing, but some combination of the softwares is not and the printer drops the connection at some point and you can no longer connect to it if WiFi. I have spent many weeks communicating with flashforge. They even sent me a different WiFi... still the same. But if I am in control, I can do it ;) I have written libusb things for it and could control it but never got to do wifi with interface

luc-github commented 7 years ago

Working on multifile upload is still in my TODO list but was busy on other topic will go back on it ASAP

luc-github commented 7 years ago

Ok issue of multiple upload is now fixed - the second issue I meet is: randomly the test page display...nothing I suspect the 5 connections limit as the page has one html, 2 js (bootstrap and Jquery) one css (bootstrap) and do one call to list all files on SPIFFS and there some hidden call like favicon, etc...

As recently I have discovered GULP, thanks to smoothie imrahil project, that I have ported to ESP3D (https://github.com/luc-github/smoothieware-webui-for-ESP3D) to concat files , minify file and gzip file at once, so I think keep the monolitic pages as for static web server but using original js to avoid waste time in optimizationis the way to go and was suggested several times by the dev guru @me-no-dev

the next step is to find an easy mechanism to select which file need authentication and which do not : may be add a config file which list all protected files and which level of authentication: guest, user, admin

luc-github commented 7 years ago

New FW and New UI are very fast with sync web server so implementation will be postponed until the need grow

luc-github commented 6 years ago

Status as of today about Async port : on ESP8266 :

UI is loaded :smile:
xxx.xxx.xxx.xxx:forcefallback=yes to force embeded page is working :smile:
SPIFFS file management is working :smile:
web update is working :smile:
SDDP is working :smile:
serial upload on SD is working on repetier :smile: but not all boards and got issue on marlin :cry: I suspect timing issue as delay usage is prohibited I may fine tune timing after M28 command before upload
web command is working partially ( ESP queries seems ok) on Serial small queries look fine - but M205 on repetier which dump EEPROM (4K) randomly fire Soft WDT did not found why :cry: :exclamation: :exclamation: :exclamation: this is critcal part.
Adding WDT functions did not helped:
```
Soft WDT reset
```

ctx: sys sp: 3ffffb80 end: 3fffffb0 offset: 01b0

stack>>> 3ffffd30: 3a525045 34322033 2e302036 20303030
3ffffd40: 72747845 6120312e 00000000 4021a7d9
3ffffd50: 3d305b20 5d66666f 3fff59ac 4020cf89
3ffffd60: 00000000 00000000 00000000 00000000
3ffffd70: 00000000 00000000 3fff50a4 0000003f
3ffffd80: 00000028 3fff4fe4 0000007f 00000000
3ffffd90: 3fff463c 0000000f 00000004 4010020c
3ffffda0: 3fff4eb4 00020fc1 3ffffd60 0000002a
3ffffdb0: 3ffed2fa 3ffffdd0 3fff4e8c 00000002
3ffffdc0: 3ffed2fa 3fff59ac 3fff59ac 4021d304
3ffffdd0: 00000000 00000000 00000000 40214c04
3ffffde0: 3fff59bc 3fff59ac 3fff3dec 40214f60
3ffffdf0: 00000001 3fff59ac 3fff59d0 00000002
3ffffe00: 3ffed2fa 00000002 3fff59ac 402129e4
3ffffe10: 3ffed2fa 00000002 3fff59d0 4021b818
3ffffe20: 3ffed2fa 3ffee3b0 3fff5c30 00000002
3ffffe30: 3ffed2fa 00000002 3fff59ac 40212c42
3ffffe40: 3ffeb1dc 00000000 00000000 4010068c
3ffffe50: 00007fff 0801adbb 401021ee 00000100
3ffffe60: 3fff59d0 00000000 3ffeb1dc 00000001
3ffffe70: 00000001 00004a88 0000046b 3fff5c08
3ffffe80: 00000001 3fff4624 3fff5bb4 40212c8d
3ffffe90: 3fffc278 40101f88 3fff5bb4 4021779d
3ffffea0: 3fff53bc 3fff365c 3fff53bc 4023cdbd
3ffffeb0: 3fff2160 3fff216c 00000006 3fff20bc
3ffffec0: 3fff53bc 3fff20c4 3fff20c0 402177e8
3ffffed0: 3fff53bc 3fff20c4 3fff20c0 40242588
3ffffee0: 0000015a 2601a8c0 00000018 0000015a
3ffffef0: 00000000 00000018 3fff0a7c 40106858
3fffff00: 00000000 3ffebef4 3ffebf10 3fff2164
3fffff10: 3ffed17a 3fff216c 3fff4624 402409a5
3fffff20: 3fff1fe8 3fff288c 3fff288c 3ffef538
3fffff30: 00000000 3fff4624 0000001c 3fff288c
3fffff40: 3ffed16c 00000000 3fff4624 4023fda5
3fffff50: 1601a8c0 000001b8 00000000 00000024
3fffff60: 00000002 0000001a 40235c9f 3ffee3b0
3fffff70: 3ffed144 3fffdcc0 3ffea988 3ffea988
3fffff80: 40235c12 3ffee3b0 00000000 3fff3b24
3fffff90: 3fffdc80 00000000 3fff4624 402439ef
3fffffa0: 40000f49 3fffdab0 3fffdab0 40000f49
<<<stack<<<

ets Jan 8 2013,rst cause:2, boot mode:(3,7)

Decoding 26 results 0x4021a78d: HardwareSerial::available() at C:\Users\user\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.4.0-nightly+20170222\cores\esp8266/HardwareSerial.cpp line 180 0x4020d0a8: handle_web_command(AsyncWebServerRequest) at C:\Users\user\AppData\Local\Temp\arduino_build_393520\sketch/webinterface.cpp line 1711 0x4010020c: _umm_free at C:\Users\user\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.4.0-nightly+20170222\cores\esp8266\umm_malloc/umm_malloc.c line 1291 0x4021d2b8: std::_Function_handler ::_M_invoke(std::_Any_data const&, AsyncWebServerRequest) at C:\Users\user\AppData\Local\Temp\arduino_build_393520\sketch/webinterface.cpp line 1711 0x40214bb8: std::function ::operator()(AsyncWebServerRequest) const at I:\Documents\Arduino\libraries\ESPAsyncWebServer\src/WebServer.cpp line 79 : (inlined by) AsyncCallbackWebHandler::handleRequest(AsyncWebServerRequest) at I:\Documents\Arduino\libraries\ESPAsyncWebServer\src/WebHandlerImpl.h line 95 0x40214f14: AsyncWebServer::_attachHandler(AsyncWebServerRequest) at I:\Documents\Arduino\libraries\ESPAsyncWebServer\src/WebServer.cpp line 79 0x40212998: AsyncWebServerRequest::_parseLine() at I:\Documents\Arduino\libraries\ESPAsyncWebServer\src/WebRequest.cpp line 777 0x4021b7cc: String::concat(char const) at C:\Users\user\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.4.0-nightly+20170222\cores\esp8266/WString.cpp line 719 0x40212bf6: AsyncWebServerRequest::_onData(void, unsigned int) at I:\Documents\Arduino\libraries\ESPAsyncWebServer\src/WebRequest.cpp line 777 0x4010068c: free at C:\Users\user\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.4.0-nightly+20170222\cores\esp8266\umm_malloc/umm_malloc.c line 1737 0x40106868: vPortFree at C:\Users\user\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.4.0-nightly+20170222\cores\esp8266/heap.c line 47 0x4010020c: _umm_free at C:\Users\user\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.4.0-nightly+20170222\cores\esp8266\umm_malloc/umm_malloc.c line 1291 0x40212c41: _M_invoke at I:\Documents\Arduino\libraries\ESPAsyncWebServer\src/WebRequest.cpp line 777 0x40244005: system_pp_recycle_rx_pkt at ?? line ? 0x40217751: AsyncClient::_recv(tcp_pcb, pbuf, long) at I:\Documents\Arduino\libraries\ESPAsyncTCP\src/ESPAsyncTCP.cpp line 659 0x4023cd71: pbuf_free at core/pbuf.c line 758 0x4021779c: AsyncClient::_s_recv(void, tcp_pcb, pbuf, long) at I:\Documents\Arduino\libraries\ESPAsyncTCP\src/ESPAsyncTCP.cpp line 659 0x4024253c: tcp_input at core/tcp_in.c line 394 (discriminator 1) 0x40106858: pvPortMalloc at C:\Users\user\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.4.0-nightly+20170222\cores\esp8266/heap.c line 40 0x40240959: ip_input at core/ipv4/ip.c line 559 0x4023fd59: ethernet_input at netif/etharp.c line 1379 0x40235c53: pp_tx_idle_timeout at ?? line ? 0x40235bc6: pp_tx_idle_timeout at ?? line ? 0x402439a3: ets_snprintf at ?? line ?


I need to test the config.txt on smoothieware which is around 20K but I expect same issue  as repetier EEPROM dump and SD File listing...
- Authentication - not yet ported
- wifi scan is not working but  sample code is available on readme so need to be ported also
- TCPIP/Serial bridge is separated from async as not compatible - so no debug anymore using telnet on async - this can be mitigate using websocket when implemented 

-on ESP32 : 
- nothing tested / done yet (finger crossed it works after ESP8266 is ported)

luc-github commented 6 years ago

Update :

WDT issue on ESP8266 is now fixed (I hope...) 22k of Smoothieware config can be downloaded without error /data lost, 4k of Repetier EEPROM also. need to use ESP.wdtFeed(); and to use beginChunkedResponse instead of beginResponseStream is mandatory for long flow
Wifi Scan is now implemented

luc-github commented 6 years ago

The branch https://github.com/luc-github/ESP3D/tree/asyncwebserver is updated

luc-github commented 6 years ago

I close issue as now code is available and async is no more a blocking feature thanks @me-no-dev :smile:

me-no-dev commented 6 years ago

will be in a couple of days in CZ, will give it a go :)

github-actions[bot] commented 4 years ago

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

luc-github / ESP3D

Implement AsyncWebServer instead of SyncWebServer #110