Update dependencies, rework WebUI, new features

[TrainzLuvr]

I pulled the sync branch in and trying to do menuconfig, getting an error:

CMake Error at CMakeLists.txt:135 (message):
  WiFi SSID & Password are required when WiFi mode is STATION or
  STATION+SoftAP.

Aside, I'm using my existing sdkconfig because it is configured to my custom pinout.

[atanisoft]

In your sdkconfig rename CONFIG_WIFI_SSID / CONFIG_WIFI_PASSWORD to CONFIG_WIFI_STATION_SSID / CONFIG_WIFI_STATION_PASSWORD ... or SOFTAP...

[TrainzLuvr]

More errors now building:

../components/StatusDisplay/StatusDisplay.cpp: In lambda function:
../components/StatusDisplay/StatusDisplay.cpp:215:60: error: 'class esp32cs::LCCWiFiManager' has no member named 'get_ssid'
          , Singleton<esp32cs::LCCWiFiManager>::instance()->get_ssid().c_str());
                                                            ^~~~~~~~

and

../components/Configuration/LCCWiFiManager.cpp: In constructor 'esp32cs::LCCWiFiManager::LCCWiFiManager(openlcb::SimpleStackBase*, const esp32cs::Esp32ConfigDef&)':
../components/Configuration/LCCWiFiManager.cpp:147:26: error: 'CONFIG_WIFI_SOFTAP_CHANNEL' was not declared in this scope
                        , CONFIG_WIFI_SOFTAP_CHANNEL
                          ^~~~~~~~~~~~~~~~~~~~~~~~~~
../components/Configuration/LCCWiFiManager.cpp:147:26: note: suggested alternative: 'CONFIG_WIFI_SOFTAP_SSID'
                        , CONFIG_WIFI_SOFTAP_CHANNEL
                          ^~~~~~~~~~~~~~~~~~~~~~~~~~
                          CONFIG_WIFI_SOFTAP_SSID

[atanisoft]

More errors now building:

I'd suggest clean up your local copy of the code and start fresh from the sync branch, you are missing a number of changes that are on the branch.

[TrainzLuvr]

I did do idf.py clean.

Do you mean checkout sync into another folder?

[atanisoft]

Idf.py clean will only clear the build folder. You can skip it by deleting that folder entirely (forces a recompile of everything).

But it appears you have a mix of master, uplink2 and sync branches. Save your sdkconfig file somewhere outside the CS tree and run: git reset --hard; git checkout sync.

[TrainzLuvr]

github search shows no CONFIG_WIFI_SOFTAP_CHANNEL in this repository

[atanisoft]

Github search is not very accurate when changes are on branches...

[atanisoft]

Also note that it will show up without the "CONFIG_" prefix as part of this PR...

[TrainzLuvr]

>git grep CONFIG_WIFI_SOFTAP_CHANNEL
components/Configuration/LCCWiFiManager.cpp:                       , CONFIG_WIFI_SOFTAP_CHANNEL

There's no definition of CONFIG_WIFI_SOFTAP_CHANNEL

EDIT: so then it's not reading my Kconfig.projbuild, it's in there but yet...

[atanisoft]

Ahh, I'll add that. You can workaround it by enabling the SoftAP in config.

[TrainzLuvr]

Still this error:

../components/StatusDisplay/StatusDisplay.cpp: In lambda function:
../components/StatusDisplay/StatusDisplay.cpp:215:60: error: 'class esp32cs::LCCWiFiManager' has no member named 'get_ssid'
          , Singleton<esp32cs::LCCWiFiManager>::instance()->get_ssid().c_str());
                                                            ^~~~~~~~

EDIT: I don't understand why git is not syncing the sync branch correctly.

[atanisoft]

Just pushed an update for the get_ssid() reference.

[TrainzLuvr]

Not working at all here - crash upon startup:

assertion "instance_ != nullptr" failed: file "../components/OpenMRNLite/src/utils/Singleton.hxx", line 78, function: static T* Singleton<T>::instance() [with T = StatusDisplay]

abort() was called at PC 0x4019663b on core 0
0x4019663b: __assert_func at /builds/idf/crosstool-NG/.build/HOST-x86_64-w64-mingw32/xtensa-esp32-elf/src/newlib/newlib/libc/stdlib/assert.c:62 (discriminator 8)

Backtrace:0x40088f8e:0x3ffbe250 0x400895a1:0x3ffbe270 0x40090109:0x3ffbe290 0x4019663b:0x3ffbe310 0x400f0aae:0x3ffbe340 0x400f1aa1:0x3ffbe380 0x400d721a:0x3ffbe4b0 0x400d49d4:0x3ffbee00

stack:
0:
  0x40088f8e: panic_abort at C:/espressif/esp-idf/components/esp_system/panic.c:341
0:
  0x400895a1: esp_system_abort at C:/espressif/esp-idf/components/esp_system/system_api.c:106
0:
  0x40090109: abort at C:/espressif/esp-idf/components/newlib/abort.c:46
0:
  0x4019663b: __assert_func at /builds/idf/crosstool-NG/.build/HOST-x86_64-w64-mingw32/xtensa-esp32-elf/src/newlib/newlib/libc/stdlib/assert.c:62 (discriminator 8)
0:
  0x400f0aae: esp32cs::update_status_display() at r:\code\esp32commandstation\build/../components/OpenMRNLite/src/utils/Singleton.hxx:78
  \-> inlined by: update_status_display at r:\code\esp32commandstation\build/../components/DCCSignalGenerator/DCCSignalVFS.cpp:212
0:
  0x400f1aa1: esp32cs::init_dcc(openlcb::Node*, Service*, esp32cs::TrackOutputConfig const&, esp32cs::TrackOutputConfig const&) at r:\code\esp32commandstation\build/../components/DCCSignalGenerator/DCCSignalVFS.cpp:512
0:
  0x400d721a: app_main at r:\code\esp32commandstation\build/../main/ESP32CommandStation.cpp:536 (discriminator 1)
0:
  0x400d49d4: main_task at C:/espressif/esp-idf/components/esp32/cpu_start.c:600

[atanisoft]

I've just pushed a fix that should fix that. I need to review the dependencies between the modules and see if it is possible to remove any from the critical path.

[TrainzLuvr]

It did fix it, now getting

URI /index.html was not found.

when I go to WebUI.

Also, the last LED does not light up green anymore when it connects to my home WiFi but stays blue. It used to go green when it connects, which was a good indicator to me that things are somewhat working.

[atanisoft]

It did fix it, now getting

URI /index.html was not found.

when I go to WebUI.

Do you have a cached copy of the link or are you trying to directly access /index.html? It should now just be /.

Also, the last LED does not light up green anymore when it connects to my home WiFi but stays blue. It used to go green when it connects, which was a good indicator to me that things are somewhat working.

I'll check it out. It was working when I looked last but perhaps something broke as part of the restructure of WiFi code.

[TrainzLuvr]

Hmm, it's Chrome..."likes" to screw around with me showing me cached pages.

Well it loaded and I clicked on the LCC tab and it bombed. Trying to refresh the UI just made it hang more.

Those tabs need to be different than everything else, they do not stand-out from other buttons, but that's just styling. There are more pressing issues atm.

[atanisoft]

Did the CS crash or was it browser side? (There is a bit of debug logging on the browser side as well)

[TrainzLuvr]

CS crashed - the browser hung and I tried refreshing a few times, closed the tab, tried to open a new but it would not connect.

Here's the crash:

accept: error (23) Too many open files in system

abort() was called at PC 0x40125e14 on core 1
0x40125e14: print_errno_and_exit at r:\code\esp32commandstation\build/../components/OpenMRNLite/src/utils/errno_exit.c:52

Backtrace:0x40088f8e:0x3ffe8070 0x400895a1:0x3ffe8090 0x40090109:0x3ffe80b0 0x40125e14:0x3ffe8130 0x40126802:0x3ffe8160 0x40126981:0x3ffe81b0 0x40122fe7:0x3ffe81d0
0x40088f8e: panic_abort at C:/espressif/esp-idf/components/esp_system/panic.c:341

0x400895a1: esp_system_abort at C:/espressif/esp-idf/components/esp_system/system_api.c:106

0x40090109: abort at C:/espressif/esp-idf/components/newlib/abort.c:46

0x40125e14: print_errno_and_exit at r:\code\esp32commandstation\build/../components/OpenMRNLite/src/utils/errno_exit.c:52

0x40126802: SocketListener::AcceptThreadBody() at r:\code\esp32commandstation\build/../components/OpenMRNLite/src/utils/socket_listener.cpp:164

0x40126981: accept_thread_start(void*) at r:\code\esp32commandstation\build/../components/OpenMRNLite/src/utils/socket_listener.cpp:63

0x40122fe7: os_thread_start at r:\code\esp32commandstation\build/../components/OpenMRNLite/src/os/os.c:391

[atanisoft]

Interesting, it shouldn't have opened too many files at once. Http should be limited to only a couple sockets concurrently. I'll see if I can reproduce

[TrainzLuvr]

Trying to open the LCC/OpenLCB Configuration tab hangs the browser.

After it displays

(index):1253 unnamed segment: 251
(index):1253 WiFi Configuration segment: 253
(index):1253 H-Bridge Configuration segment: 253
(index):1253 Thermal Configuration segment: 253

I get an error in the console:

Uncaught (in promise) DOMException: The user aborted a request.

Also, the bus errors keep piling up, 297531 and going up.

I have no CAN connected - this is just pure USB powered WiFi connection.

[atanisoft]

Ignore bus errors since you don't have the can transceiver or terminator present.

The browser will abort the fetch of data after 10sec for a single request. Check the network tab to see if it was successful in fetching any cdi data. The timeout can be increased in the index.html file from 10000 to 25000 or similar.

[TrainzLuvr]

Right now it's not responding, I've tried refreshing the tab a few times. There should be something in place to ignore too many consequent requests for page refresh, since ESP32 seems shite for anything web related. So much about IoT platform if it can't handle a few simultaneous requests.

[TrainzLuvr]

/config?offs=1&size=63&type=string request fails to respond with data.

[atanisoft]

That offset is one of the first requests. It sounds like the http stack got locked up. Are you using the single executor config option? If so, disable it. The CDI data load is currently using a blocking call to the main executor to load the data. Once I move to full async with payload coming back via WS it will be usable

[TrainzLuvr]

Nope, Use one Executor for OpenMRN and Httpd is unchecked in my config.

[TrainzLuvr]

Why are you calling offs=1 with size=63 and type=string?

If I call offs=1 by itself, I get the same response, but if I include type=string it hangs.

If I call it with size=63 it appears to hang for a few moments but eventually responds with the same response.

{"resp-type":"OK"}

[atanisoft]

It's based on the cdi entry for user info node name. The default response of OK is when it doesn't lookup the CDI data. It checks for size and offs parameters.

It may just be better to shift to the WS rather than serialized requests. I'll see if I can implement that.

[TrainzLuvr]

How come you are dynamically creating cdi.xml, it looks to me like those values do not change, or they are defaults?

I think it should be a static file in the data/ folder so that the web server can send it as fast as possible, rather than waiting on CPU to dynamically compile it.

At least all the text definitions and default values should be static. The only dynamic thing that should come across to the client side is the new/different values than the defaults.

[atanisoft]

The CDI.xml is static and downloaded from the CS. What you are seeing as requests are the data fetch requests

[TrainzLuvr]

All I see is a hang when I click on the LCC/OpenLCB Configuration tab. :)

[atanisoft]

It's hanging while trying to load the CDI dynamic data. It sounds a lot like single executor still. It shouldn't be hanging otherwise.

[atanisoft]

@TrainzLuvr Give the latest a test run. It moves almost all operations to JSON based WS and should be faster and more reliable.

[TrainzLuvr]

The icons in the upper right (WiFi, eSTOP, OPS Power, Active Locomotive) are not intutive because some are "hot" and can be clicked on, and others are not.

I suggest making them just as indicators and making it obvious they are just indicators, or turning them into some button like design with a clear illumination "on/off" indicator.

The Configuration tab now loads, but if I change to another tab then go back to Configuration, the two Refresh buttons next to User Name and User Description fields are spinning.

Going back and forth between the Configuration tab and some other a few times, caused a crash (seems I DDoSed WS connection):

0x400d5d87: __cxa_end_catch at C:/espressif/esp-idf/components/cxx/cxx_exception_stubs.cpp:13

Backtrace:0x40088f8e:0x3ffd3510 0x400895a1:0x3ffd3530 0x40090109:0x3ffd3550 0x400d5d87:0x3ffd35d0 0x40195b89:0x3ffd35f0 0x400e7ad6:0x3ffd3610 0x401a7815:0x3ffd3760 0x4010a1da:0x3ffd3780 0x4010a3c2:0x3ffd37b0 0x40116819:0x3ffd3810 0x40116621:0x3ffd3840 0x401a8819:0x3ffd3870 0x40123fab:0x3ffd3890
0x40088f8e: panic_abort at C:/espressif/esp-idf/components/esp_system/panic.c:341

0x400895a1: esp_system_abort at C:/espressif/esp-idf/components/esp_system/system_api.c:106

0x40090109: abort at C:/espressif/esp-idf/components/newlib/abort.c:46

0x400d5d87: __cxa_end_catch at C:/espressif/esp-idf/components/cxx/cxx_exception_stubs.cpp:13

0x40195b89: operator new(unsigned int) at /builds/idf/crosstool-NG/.build/HOST-x86_64-w64-mingw32/xtensa-esp32-elf/src/gcc/libstdc++-v3/libsupc++/new_op.cc:54

0x400e7ad6: process_wsjson(http::WebSocketFlow*, http::WebSocketEvent, unsigned char const*, unsigned int) at r:\code\esp32commandstation\build/../main/WebServer.cpp:501 (discriminator 6)

0x401a7815: std::_Function_handler<void (http::WebSocketFlow*, http::WebSocketEvent, unsigned char*, unsigned int), void (*)(http::WebSocketFlow*, http::WebSocketEvent, unsigned char const*, unsigned int)>::_M_invoke(std::_Any_data const&, http::WebSocketFlow*&&, http::WebSocketEvent&&, unsigned char*&&, unsigned int&&) at c:\espressif\tools\xtensa-esp32-elf\esp-2020r3-8.4.0\xtensa-esp32-elf\xtensa-esp32-elf\include\c++\8.4.0\bits/std_function.h:297

0x4010a1da: std::function<void (http::WebSocketFlow*, http::WebSocketEvent, unsigned char*, unsigned int)>::operator()(http::WebSocketFlow*, http::WebSocketEvent, unsigned char*, unsigned int) const at c:\espressif\tools\xtensa-esp32-elf\esp-2020r3-8.4.0\xtensa-esp32-elf\xtensa-esp32-elf\include\c++\8.4.0\bits/std_function.h:687

0x4010a3c2: http::WebSocketFlow::recv_frame_data() at r:\code\esp32commandstation\build/../components/HttpServer/src/HttpRequestWebSocket.cpp:375

0x40116819: StateFlowBase::run() at r:\code\esp32commandstation\build/../components/OpenMRNLite/src/executor/StateFlow.cpp:63 (discriminator 4)

0x40116621: ExecutorBase::entry() at r:\code\esp32commandstation\build/../components/OpenMRNLite/src/executor/Executor.cpp:324

0x401a8819: OSThread::start(void*) at r:\code\esp32commandstation\build/../components/OpenMRNLite/src/os/OS.hxx:193

0x40123fab: os_thread_start at r:\code\esp32commandstation\build/../components/OpenMRNLite/src/os/os.c:391

[atanisoft]

The icons in the upper right (WiFi, eSTOP, OPS Power, Active Locomotive) are not intutive because some are "hot" and can be clicked on, and others are not.

They are all clickable. OPS power when clicked will change color to reflect the status etc. I'm still searching for a good icon to use for eStop as the current one is not very intuitive for sure, similarly the coloring.

The Configuration tab now loads, but if I change to another tab then go back to Configuration, the two Refresh buttons next to User Name and User Description fields are spinning.

The refresh button will spin until the field has been refreshed (triggered via tab change after the first load). Most likely the exception you posted is the reason why it stayed spinning. I logged https://github.com/atanisoft/esp32olcbio/issues/1 previously to have a queuing mechanism rather than dynamically creating/processing/deleting the CDIRequestProcessor. It looks like that will need to be tackled as part of this PR.

[TrainzLuvr]

Also appears that User Name/Description fields do not want to save back, it just spins.

For the icons, if they are buttons they should look like some kind of a button (pressable item). Perhaps adding a rounded border via CSS would make them look more buttony. :)

[atanisoft]

Also appears that User Name/Description fields do not want to save back, it just spins.

Check the browser console or network tab to check for errors. Be sure to press the update-complete button as well when done (I'll add a timer to trigger it automatically after 20sec or so).

Perhaps adding a rounded border via CSS would make them look more buttony. :)

I'll see what I can add. Initial tests with the CSS classes "btn bg-dark s-circle" added are not very promising.

[TrainzLuvr]

Shouldn't the button stop spinning first before clicking Update Complete?

Nothing in the console at all.

[atanisoft]

Shouldn't the button stop spinning first before clicking Update Complete?

Under normal circumstances yes. Check the /wsjson endpoint in the network tab for the messages being sent/received. There should be one message sent for each field being loaded (or reloaded/saved) and a corresponding reply with the same "target=xxx" field.

[atanisoft]

I was able to reproduce the spinning behavior but only if I flipped between tabs rapidly without allowing the queues to drain. On the CS console the following is seen:

Incoming connection from 10.0.0.221, fd 52.
[TaskMon] uptime: 00:00:47 freeHeap: 142504, largest free block: 62948, tasks: 17, mainBufferPool: 4.94kB
[WebSocket fd:51] Discarding text frame
[WebSocket fd:51] Discarding text frame
[WebSocket fd:51] Discarding text frame
[WebSocket fd:51] Discarding text frame
[WebSocket fd:51] Discarding text frame
....

This is indicating that the WS TX queue is not draining fast enough, I'm not certain why this is the case but it does appear to work when flipping between tabs but not in a rapid fashion.

[TrainzLuvr]

I have not seen those messages on my CS console ever.

Just the usual Incoming Connection... and [TWAI] stats. I don't see [TaskMon] either.

[atanisoft]

I don't see [TaskMon] either.

You should be seeing it every ~45sec. It is enabled by default.

One other oddity:

[TaskMon] uptime: 00:02:17 freeHeap: 140688, largest free block: 62156, tasks: 17, mainBufferPool: 4.72kB
[TaskMon] uptime: 00:03:02 freeHeap: 154388, largest free block: 71984, tasks: 17, mainBufferPool: 0.67kB

I've never seen the mainBufferPool size go down without a restart.

[TrainzLuvr]

Maybe because I have the Status OLED connected, so all the TaskMon messages are redirected there?

But that's still weird, because serial console is sort of a debug output, it should be displayed there as well.

[atanisoft]

Maybe because I have the Status OLED connected, so all the TaskMon messages are redirected there?

TaskMon knows nothing of the OLED so that is unlikely.

[TrainzLuvr]

maybe try this for the .tooltip class

border-radius: 13px;
border: 3px solid #ccddff;
margin: 0 3px;
padding: 6px;

[TrainzLuvr]

Web server still hangs and buttons spinning.

All I did was to update the User Name in the Configuration which caused it to stop responding.

No errors anywhere.

[atanisoft]

Verify the WS messages, there should be a TX for "cdi-set" with the value you set and it should have a corresponding RX soon there after.

I can't reproduce the hangs you are seeing.

[TrainzLuvr]

I only see "req" and not "res".

[atanisoft]

Something is definitely off then, this is the expected output for setting the "User Name:" field:

{"req":"cdi-set","offs":1,"size":63,"type":"string","target":"cdi_0_006602631391999347","value":"ESP32 Command Station2"} | 121 | 14:52:20.363 |  
-- | -- | -- | --
{"res":"saved","target":"cdi_0_006602631391999347"} | 52 | 14:52:20.404 |  
{"req":"cdi-set","offs":1,"size":63,"type":"string","target":"cdi_0_006602631391999347","value":"ESP32 Command Station"} | 120 | 14:52:38.650 |  
{"res":"saved","target":"cdi_0_006602631391999347"}

Note that the "target" value is randomly generated by the browser.