mikee47
commented
1 month ago Is it possible to run on Host? This sort of thing is much easier to debug there. Can also run sanitisers, etc.
Closed pljakobs closed 3 weeks ago
mikee47
commented
1 month ago Is it possible to run on Host? This sort of thing is much easier to debug there. Can also run sanitisers, etc.
pljakobs
commented
1 month ago I have not cecked what I need to run on host, right now, it won't compile. let me see if it's easy.
mikee47
commented
1 month ago grep LmacRxBlk $SMING_HOME/Arch/Esp8266/Components/esp8266/sdk -r it's in the SDK. A web search indicates this has something to do with NAT table but I'd place bets on memory corruption somewhere... hence sanitisers.
pljakobs
commented
1 month ago hmm? I think that might be a red herring, we don't do NAT here I would tend to think it's memory corruption or timing
(compiling for host, let's see how hard it will be - I've been mostly focussed on fixing the Esp8266 base first
okay, this will take a bit longer... I think what I will try is to take all the OTA code out and build then. Just need to do it cleanly.
mikee47
commented
1 month ago pljakobs
commented
1 month ago so I have the code running on Host but am seeing "can't open file" errors from ConfigDB
2002049 mounting host file system
digitalRead: 16, 0
2002070 CLR button low - resetting settings
2002076 AppWIFI::forget_wifi
2002084 SSID '' not found
2002095 AppWIFI::startAp
2002101 Enabling AP
2002106 AppWIFI:: WifiAP enable
2002112 AP enabled
2002118 [CFGDB] LoadStore 'network'
2002129 [CFGDB] Store() network 881446c 1
2002137 [CFGDB] incUpdate() network 881446c 1
2002260 attached file: '' (379 bytes) #0xFFAFED80
2002384 [CFGDB] decUpdate() network 881446c 0
2002400 UDP listen port 53
2002407 application init => checking ConfigDB
2002418 [CFGDB] ~Store() network 881446c 0
2002425 [CFGDB] LoadStore 'general'
2002431 [CFGDB] Store() general 881446c 1
2002442 [CFGDB] incUpdate() general 881446c 1
2002528 attached file: '' (305 bytes) #0xFFAFEEA0
2002589 [CFGDB] decUpdate() general 881446c 0
2002601 ConfigDB already initialized. starting
** Stream **
{2002625 [CFGDB] ~Store() general 881446c 0
2002635 [CFGDB] LoadStore ''
2002642 [CFGDB] Store() 881446c 1
2002648 [CFGDB] incUpdate() 881446c 1
2002667 File 'app-config/_root.json' open error: NotFound
2002678 [CFGDB] decUpdate() 881446c 0$ls app-config
general.json network.jsonAs I am using the pointer to ConfigDB across multiple Classes, I have defined it as
class Application{
public:
std::shared_ptr<AppConfig> cfg;
}and in the application initialization
cfg = std::make_shared<AppConfig>(configDB_PATH);General is the first store I'm accessing, I guess Network is the second.
longer log snippet:
2003347 mounting host file system
digitalRead: 16, 0
2003368 CLR button low - resetting settings
2003375 AppWIFI::forget_wifi
2003384 SSID '' not found
2003396 AppWIFI::startAp
2003402 Enabling AP
2003407 AppWIFI:: WifiAP enable
2003413 AP enabled
2003421 [CFGDB] LoadStore 'network'
2003433 [CFGDB] Store() network 93a646c 1
2003441 [CFGDB] incUpdate() network 93a646c 1
2003578 attached file: '' (379 bytes) #0xFF972F60
2003719 [CFGDB] decUpdate() network 93a646c 0
2003738 UDP listen port 53
2003746 application init => checking ConfigDB
2003760 [CFGDB] ~Store() network 93a646c 0
2003768 [CFGDB] LoadStore 'general'
2003775 [CFGDB] Store() general 93a646c 1
2003787 [CFGDB] incUpdate() general 93a646c 1
2003870 attached file: '' (305 bytes) #0xFF973080
2003935 [CFGDB] decUpdate() general 93a646c 0
2003948 ConfigDB already initialized. starting
** Stream **
{2003975 [CFGDB] ~Store() general 93a646c 0
2003987 [CFGDB] LoadStore ''
2003993 [CFGDB] Store() 93a646c 1
2004001 [CFGDB] incUpdate() 93a646c 1
2004021 File 'app-config/_root.json' open error: NotFound
2004034 [CFGDB] decUpdate() 93a646c 0
"security":{"api_secured":false,"api_password":""},"ota":{"url":""},"events":{"color_interval_ms":500,"color_mininterval_ms":500,"transfin_interval_ms":1000,"server_enabled":true},2004261 [CFGDB] ~Store() 93a646c 0
2004268 [CFGDB] LoadStore 'general'
2004275 [CFGDB] Store() general 93a646c 1
2004286 [CFGDB] incUpdate() general 93a646c 1
2004359 attached file: '' (305 bytes) #0xFF972E30
2004424 [CFGDB] decUpdate() general 93a646c 0
"general":{"channels":[],"supported_color_models":[],"is_initialized":true,"device_name":"mrpj","buttons_debounce_ms":50,"pin_config":"13,12,14,5,4","buttons_config":"","pin_config_url":"https://raw.githubusercontent.com/pljakobs/esp_rgb_webapp2/devel/public/config/pinconfig.json","current_pin_config_name":"mrpj"},2004679 [CFGDB] ~Store() general 93a646c 0
2004692 [CFGDB] LoadStore 'color'
2004699 [CFGDB] Store() color 93a646c 1
2004706 [CFGDB] incUpdate() color 93a646c 1
2004727 File 'app-config/color.json' open error: NotFound
2004739 [CFGDB] decUpdate() color 93a646c 0
"color":{"brightness":{"red":100,"ww":100,"green":100,"blue":100,"cw":100},"colortemp":{"ww":2700,"cw":6000},"hsv":{"red":0,"magenta":0,"green":0,"blue":0,"yellow":0,"model":0,"cyan":0},"startup_color":"last","outputmode":0},2005152 [CFGDB] ~Store() color 93a646c 0
2005159 [CFGDB] LoadStore 'sync'
2005171 [CFGDB] Store() sync 93a646c 1
2005178 [CFGDB] incUpdate() sync 93a646c 1
2005196 File 'app-config/sync.json' open error: NotFound
2005208 [CFGDB] decUpdate() sync 93a646c 0
"sync":{"cmd_master_enabled":false,"color_slave_enabled":false,"color_slave_topic":"home/led1/command","clock_master_enabled":false,"color_master_interval_ms":0,"clock_slave_enabled":false,"cmd_slave_enabled":false,"clock_master_interval":30,"clock_slave_topic":"home/led1/clock","cmd_slave_topic":"home/led/command","color_master_enabled":false},2005478 [CFGDB] ~Store() sync 93a646c 0
2005485 [CFGDB] LoadStore 'network'
2005491 [CFGDB] Store() network 93a646c 1
2005503 [CFGDB] incUpdate() network 93a646c 1
2005577 attached file: '' (379 bytes) #0xFF972E30
2005796 [CFGDB] decUpdate() network 93a646c 0
"network":{"mqtt":{"server":"mqtt.local","password":"","port":1883,"topic_base":"home/","enabled":false,"username":""},"ntp":{"enabled":false,"server":"pool.ntp.org","interval":3600},"connection":{"netmask":"255.255.255.0","ip":"192.168.1.1","dhcp":true,"gateway":"192.168.1.255"},"ap":{"password":"configesp","secured":true,"ssid":"RGBWW3221339823"},"mdns":{"enabled":true,"name":"mrpj"}}}2006290 [CFGDB] ~Store() network 93a646c 0
2006298 [CFGDB] LoadStore 'general'
2006309 [CFGDB] Store() general 93a646c 1
2006316 [CFGDB] incUpdate() general 93a646c 1
2006400 attached file: '' (305 bytes) #0xFF972FE0
2006467 [CFGDB] decUpdate() general 93a646c 0
2006475 AppMqttClient::init: building MQTT ID from device name: 'mrpj'
2006488 APPLedCtrl::init
2006504 APPLedCtrl::setup
2006511 [CFGDB] ~Store() general 93a646c 0
2006518 [CFGDB] LoadStore 'color'
2006529 [CFGDB] Store() color 93a646c 1
2006536 [CFGDB] incUpdate() color 93a646c 1
2006557 File 'app-config/color.json' open error: NotFound
2006568 [CFGDB] decUpdate() color 93a646c 0
2006578 COLORMODE 0
2006584 HSVMODE 0
2006674 attached file: '' (29 bytes) #0xFF972FC0
2006845 H: 0 | s: 0 | v: 0 | ct: 2700
2006861 [CFGDB] ~Store() color 93a646c 0
2006869 [CFGDB] LoadStore 'general'
2006880 [CFGDB] Store() general 93a646c 1
2006887 [CFGDB] incUpdate() general 93a646c 1
2006962 attached file: '' (305 bytes) #0xFF972FC0
2007027 [CFGDB] decUpdate() general 93a646c 0
2007041 [CFGDB] ~Store() general 93a646c 0
2007053 [CFGDB] LoadStore 'network'
2007059 [CFGDB] Store() network 93a646c 1
2007066 [CFGDB] incUpdate() network 93a646c 1
2007142 attached file: '' (379 bytes) #0xFF972F20
2007221 [CFGDB] decUpdate() network 93a646c 0
2007229 AppWIFI::init dhcp
(pulled the current ConfigDB develop and rebuilt - same result, the first two stores are created, after that, I get "NotFound" errors)
mikee47
commented
1 month ago mikee47
commented
1 month ago The NotFound errors are fine, that just means you'll get the default store values.
mikee47
commented
1 month ago I note you have some customised LWIP configuration, could that be implicated?
mikee47
commented
1 month ago Also, you can remove the ARDUINO_LIBRARIES line from component.mk to fix the duplicate build target warnings.
pljakobs
commented
1 month ago The NotFound errors are fine, that just means you'll get the default store values.
understood - but should the file not be crated then?
I note you have some customised LWIP configuration, could that be implicated?
I have removed them now, I had used them when I was troubleshooting what I thought was a faulty tcp fragmentation issue - turned out that looking at the request.bodyStream was what I needed to do (instead of request.body)
Also, you can remove the ARDUINO_LIBRARIES line from component.mk to fix the duplicate build target warnings.
The RGBWWLed library is a required componet, I removed ArduinoJSON and OtaNetwork, I guess they were still unchanged from the original version.
I think the result is still much the same:
I see NotFound on all but the first two stores and they are not created on the Host file system.
Am I right thinking that fileSetFileSystem(&IFS::Host::getFileSystem()); is all I need to enable the host filesystem?
2002344 mounting host file system
digitalRead: 16, 0
2002365 CLR button low - resetting settings
2002372 AppWIFI::forget_wifi
2002380 SSID '' not found
2002392 AppWIFI::startAp
2002398 Enabling AP
2002404 AppWIFI:: WifiAP enable
2002410 AP enabled
2002417 [CFGDB] LoadStore 'network'
2002429 [CFGDB] Store() network 93ed3fc 1
2002438 [CFGDB] incUpdate() network 93ed3fc 1
2002566 attached file: '' (379 bytes) #0xFFA9A710
2002711 [CFGDB] decUpdate() network 93ed3fc 0
2002729 UDP listen port 53
2002737 application init => checking ConfigDB
2002748 [CFGDB] ~Store() network 93ed3fc 0
2002756 [CFGDB] LoadStore 'general'
2002763 [CFGDB] Store() general 93ed3fc 1
2002774 [CFGDB] incUpdate() general 93ed3fc 1
2002855 attached file: '' (305 bytes) #0xFFA9A830
2002917 [CFGDB] decUpdate() general 93ed3fc 0
2002930 ConfigDB already initialized. starting
** Stream **
{2002956 [CFGDB] ~Store() general 93ed3fc 0
2002968 [CFGDB] LoadStore ''
2002974 [CFGDB] Store() 93ed3fc 1
2002981 [CFGDB] incUpdate() 93ed3fc 1
2003002 File 'app-config/_root.json' open error: NotFound
2003014 [CFGDB] open 'app-config/_root.json' failed
2003021 [CFGDB] Load defaults
2003033 [CFGDB] decUpdate() 93ed3fc 0
"security":{"api_secured":false,"api_password":null},"ota":{"url":null},"events":{"color_interval_ms":500,"color_mininterval_ms":500,"transfin_interval_ms":1000,"server_enabled":true},2003271 [CFGDB] ~Store() 93ed3fc 0
2003283 [CFGDB] LoadStore 'general'
2003289 [CFGDB] Store() general 93ed3fc 1
2003296 [CFGDB] incUpdate() general 93ed3fc 1
2003376 attached file: '' (305 bytes) #0xFFA9A5E0
2003434 [CFGDB] decUpdate() general 93ed3fc 0
"general":{"channels":[],"supported_color_models":[],"is_initialized":true,"device_name":"mrpj","buttons_debounce_ms":50,"pin_config":"13,12,14,5,4","buttons_config":"","pin_config_url":"https://raw.githubusercontent.com/pljakobs/esp_rgb_webapp2/devel/public/config/pinconfig.json","current_pin_config_name":"mrpj"},2003673 [CFGDB] ~Store() general 93ed3fc 0
2003680 [CFGDB] LoadStore 'color'
2003687 [CFGDB] Store() color 93ed3fc 1
2003698 [CFGDB] incUpdate() color 93ed3fc 1
2003718 File 'app-config/color.json' open error: NotFound
2003730 [CFGDB] open 'app-config/color.json' failed
2003736 [CFGDB] Load defaults
2003744 [CFGDB] decUpdate() color 93ed3fc 0
"color":{"brightness":{"red":100,"ww":100,"green":100,"blue":100,"cw":100},"colortemp":{"ww":2700,"cw":6000},"hsv":{"red":0,"magenta":0,"green":0,"blue":0,"yellow":0,"model":0,"cyan":0},"startup_color":"last","outputmode":0},2004172 [CFGDB] ~Store() color 93ed3fc 0
2004183 [CFGDB] LoadStore 'sync'
2004190 [CFGDB] Store() sync 93ed3fc 1
2004197 [CFGDB] incUpdate() sync 93ed3fc 1
2004221 File 'app-config/sync.json' open error: NotFound
2004228 [CFGDB] open 'app-config/sync.json' failed
2004234 [CFGDB] Load defaults
2004241 [CFGDB] decUpdate() sync 93ed3fc 0
"sync":{"cmd_master_enabled":false,"color_slave_enabled":false,"color_slave_topic":"home/led1/command","clock_master_enabled":false,"color_master_interval_ms":0,"clock_slave_enabled":false,"cmd_slave_enabled":false,"clock_master_interval":30,"clock_slave_topic":"home/led1/clock","cmd_slave_topic":"home/led/command","color_master_enabled":false},2004508 [CFGDB] ~Store() sync 93ed3fc 0
2004515 [CFGDB] LoadStore 'network'
2004522 [CFGDB] Store() network 93ed3fc 1
2004533 [CFGDB] incUpdate() network 93ed3fc 1
2004609 attached file: '' (379 bytes) #0xFFA9A5E0
2004693 [CFGDB] decUpdate() network 93ed3fc 0
"network":{"mqtt":{"server":"mqtt.local","password":"","port":1883,"topic_base":"home/","enabled":false,"username":""},"ntp":{"enabled":false,"server":"pool.ntp.org","interval":3600},"connection":{"netmask":"255.255.255.0","ip":"192.168.1.1","dhcp":true,"gateway":"192.168.1.255"},"ap":{"password":"configesp","secured":true,"ssid":"RGBWW3221339823"},"mdns":{"enabled":true,"name":"mrpj"}}}2005360 [CFGDB] ~Store() network 93ed3fc 0
2005368 [CFGDB] LoadStore 'general'
2005379 [CFGDB] Store() general 93ed3fc 1
2005386 [CFGDB] incUpdate() general 93ed3fc 1
2005476 attached file: '' (305 bytes) #0xFFA9A790
2005545 [CFGDB] decUpdate() general 93ed3fc 0
2005554 AppMqttClient::init: building MQTT ID from device name: 'mrpj'
2005566 APPLedCtrl::init
2005583 APPLedCtrl::setup
2005589 [CFGDB] ~Store() general 93ed3fc 0
2005597 [CFGDB] LoadStore 'color'
2005608 [CFGDB] Store() color 93ed3fc 1
2005615 [CFGDB] incUpdate() color 93ed3fc 1
2005635 File 'app-config/color.json' open error: NotFound
2005647 [CFGDB] open 'app-config/color.json' failed
2005653 [CFGDB] Load defaults
2005660 [CFGDB] decUpdate() color 93ed3fc 0
2005674 COLORMODE 0
2005680 HSVMODE 0
2005756 attached file: '' (29 bytes) #0xFFA9A760
2005917 H: 0 | s: 0 | v: 0 | ct: 2700
2005937 [CFGDB] ~Store() color 93ed3fc 0
2005944 [CFGDB] LoadStore 'general'
2005950 [CFGDB] Store() general 93ed3fc 1
2005962 [CFGDB] incUpdate() general 93ed3fc 1
2006032 attached file: '' (305 bytes) #0xFFA9A770
2006098 [CFGDB] decUpdate() general 93ed3fc 0
2006116 [CFGDB] ~Store() general 93ed3fc 0
2006123 [CFGDB] LoadStore 'network'
2006129 [CFGDB] Store() network 93ed3fc 1
2006158 [CFGDB] incUpdate() network 93ed3fc 1
2006232 attached file: '' (379 bytes) #0xFFA9A6D0
2006315 [CFGDB] decUpdate() network 93ed3fc 0
2006323 AppWIFI::init dhcp
2006330 '*' registered
2006339 '/' registered
2006353 '/webapp' registered
2006361 '/config' registered
2006368 '/info' registered
2006375 '/color' registered
2006383 '/networks' registered
2006397 '/scan_networks' registered
2006405 '/system' registered
2006413 '/update' registered
2006422 '/connect' registered
2006435 '/ping' registered
2006443 '/hosts' registered
2006451 '/object' registered
2006459 '/stop' registered
2006472 '/skip' registered
2006480 '/pause' registered
2006489 '/continue' registered
2006497 '/blink' registered
2006510 '/toggle' registered
IF_UP, AP: Host WiFi
2006539 AppWIFI::_STAConnected SSID - Host WiFi
2006546 [CFGDB] ~Store() network 93ed3fc 0
2006554 [CFGDB] LoadStore 'general'
2006560 [CFGDB] Store() general 93ed3fc 1
2006567 [CFGDB] incUpdate() general 93ed3fc 1
2006641 attached file: '' (305 bytes) #0xFFA9A6D0
2006716 [CFGDB] decUpdate() general 93ed3fc 0
2006724 AppWIFI::connect setting hostname to mrpj
2006736 [CFGDB] LoadStore 'network'
2006743 [CFGDB] Store() network 93eedec 2
2006750 [CFGDB] incUpdate() network 93eedec 1
2006828 attached file: '' (379 bytes) #0xFFA9A6A0
2007014 [CFGDB] decUpdate() network 93eedec 0
2007035 [CFGDB] lockStore() app-config 93ed32c
2007048 [CFGDB] incUpdate() network 93eedec 1
2007055 [CFGDB] incUpdate() network 93eedec 2
2007067 [CFGDB] decUpdate() network 93eedec 1
2007075 [CFGDB] decUpdate() network 93eedec 0
2007082 [CFGDB] Save 'network'
2007320 attached file: '' (0 bytes) #0xFFA9A498
2007509 [CFGDB] Object saved to 'app-config/network.json'
2007556 [CFGDB] ~Store() general 93ed3fc 1
IP_CHANGE, ip: 192.168.13.10, netmask: 255.255.255.0, gateway: 192.168.13.1
2007581 AppWIFI::_STAGotIP
2007587 [CFGDB] LoadStore 'general'
2007594 [CFGDB] Store() general 93ed3fc 2
2007606 [CFGDB] incUpdate() general 93ed3fc 1
2007772 attached file: '' (305 bytes) #0xFFA9A730
2007896 [CFGDB] decUpdate() general 93ed3fc 0
2007917 AppWIFI::_STAGotIP - device_name mrpj mdnshostname mrpj
2007924 AppWIFI::_STAGotIP - setting mdns hostname to mrpj
2007936 [CFGDB] ~Store() general 93ed3fc 1
2008007 UDP listen port 5353
2008015 MDNS initialised for 'mrpj'
2008029 [MDNS] addService 'esprgbwwAPI._http._tcp.local'
2008038 [MDNS] addService 'esprgbwwWebApp._http._tcp.local'
2008046 [MDNS] addService 'esprgbwwWS._http._tcp.local'
2008060 setting searchName to esprgbwwAPI._http._tcp.local
2008105 Application::wsBroadcast
2008117 === Websocket Broadcast ===
{"jsonrpc":"2.0","method":"new_host","params":{"hostname":"mrpj","ip_address":"dhcp","ttl":-1}}
2008159 ===>nr of tcpConnections: 0
2008165 =========================================
2008172 ===>nr of websockets: 0
2008229 rpc: root ={"station":{"connected":true,"ssid":"Host WiFi","dhcp":true,"ip":"192.168.13.10","netmask":"255.255.255.0","gateway":"192.168.13.1","mac":"52854b8022b0"},"ap":{"enabled":false,"ssid":"","ip":"0.0.0.0"}}
2008265 rpc: msg ={"jsonrpc":"2.0","method":"wifi_status","params":{"station":{"connected":true,"ssid":"Host WiFi","dhcp":true,"ip":"192.168.13.10","netmask":"255.255.255.0","gateway":"192.168.13.1","mac":"52854b8022b0"},"ap":{"enabled":false,"ssid":"","iactually, and I may have missed that - do you only create the storage file once it's first written to and otherwise it's just a read from defaults? Thinking about it, that would be sensible
pljakobs
commented
1 month ago now I'm wondering if I broke something with my re-factor to be able to build for Host (and Esp32 at the same time) as I am now getting the open {filename} failed errors on Esp8266, too
** Stream **
{[Components/ConfigDB/src/Object.cpp:447] [CFGDB] open 'app-config/_root.json' failed
"security":{"api_secured":false,"api_password":null},"ota":{"url":null},"events":{"color_interval_ms":500,"color_mininterval_ms":500,"transfin_interval_ms":1000,"server_enabled":true},"general":{"channels":[],"supported_color_models":["RGB","RGBW","RGBWW","RAW"],"is_initialized":true,"device_name":"mrpj","buttons_debounce_ms":50,"pin_config":"13,12,14,5,4","buttons_config":null,"pin_config_url":"https://raw.githubusercontent.com/pljakobs/esp_rgb_webapp2/devel/public/config/pinconfig.json","current_pin_config_name":"mrpj"},[Components/ConfigDB/src/Object.cpp:447] [CFGDB] open 'app-config/color.json' failed
"color":{"brightness":{"red":100,"ww":100,"green":100,"blue":100,"cw":100},"colortemp":{"ww":2700,"cw":6000},"hsv":{"red":0,"magenta":0,"green":0,"blue":0,"yellow":0,"model":0,"cyan":0},"startup_color":"last","outputmode":0},[Components/ConfigDB/src/Object.cpp:447] [CFGDB] open 'app-config/sync.json' failed
"sync":{"cmd_master_enabled":false,"color_slave_enabled":false,"color_slave_topic":"home/led1/command","clock_master_enabled":false,"color_master_interval_ms":0,"clock_slave_enabled":false,"cmd_slave_enabled":false,"clock_master_interval":30,"clock_slave_topic":"home/led1/clock","cmd_slave_topic":"home/led/command","color_master_enabled":false},"network":{"mqtt":{"server":"mqtt.local","password":null,"port":1883,"topic_base":"home/","enabled":false,"username":null},"ntp":{"enabled":false,"server":"pool.ntp.org","interval":3600},"connection":{"netmask":"255.255.255.0","ip":"192.168.1.1","dhcp":true,"gateway":"192.168.1.255"},"ap":{"password":"configesp","secured":true,"ssid":"RGBWW2237526"},"mdns":{"enabled":true,"name":"RGBWW2237526"}}}[/home/pjakobs/devel/esp_rgbww_firmware/app/mqtt.cpp:93] AppMqttClient::init: building MQTT ID from device name: 'mrpj'
[/home/pjakobs/devel/esp_rgbww_firmware/app/ledctrl.cpp:106] APPLedCtrl::init
[/home/pjakobs/devel/esp_rgbww_firmware/app/ledctrl.cpp:164] APPLedCtrl::setup
[Components/ConfigDB/src/Object.cpp:447] [CFGDB] open 'app-config/color.json' failed
[/home/pjakobs/devel/esp_rgbww_firmware/app/ledctrl.cpp:142] H: 0 | s: 0 | v: 0 | ct: 2700
sleep disable
[Components/Network/Arch/Esp8266/Platform/StationImpl.cpp:239] SSID: ''
[/home/pjakobs/devel/esp_rgbww_firmware/app/networking.cpp:157] AppWIFI::init no AP to connect to - start own AP
[/home/pjakobs/devel/esp_rgbww_firmware/app/networking.cpp:375] AppWIFI::startAp
[/home/pjakobs/devel/esp_rgbww_firmware/app/networking.cpp:376] Enabling AP
[/home/pjakobs/devel/esp_rgbww_firmware/app/networking.cpp:378] AppWIFI:: WifiAP enable
[/home/pjakobs/devel/esp_rgbww_firmware/app/networking.cpp:380] AP enabled
[Components/Network/Arch/Esp8266/Platform/AccessPointImpl.cpp:88] AP configuration loaded
[Components/Network/Arch/Esp8266/Platform/StationImpl.cpp:239] SSID: ''
[Components/Network/Arch/Esp8266/Platform/AccessPointImpl.cpp:166] SSID: RGBWW2237526
[/home/pjakobs/devel/esp_rgbww_firmware/app/networking.cpp:432] rpc: root ={"message":"AP started","station":{"connected":false,"ssid":"","dhcp":true,"ip":"0.0.0.0","netmask":"0.0.0.0","gateway":"0.0.0.0","mac":"84f3eb222456"},"ap":{"enabled":true,"ssid":[/home/pjakobs/devel/esp_rgbww_firmware/app/networking.cpp:433] rpc: msg ={"jsonrpc":"2.0","method":"wifi_status","params":{"message":"AP started","station":{"connected":false,"ssid":"","dhcp":true,"ip":"0.0.0.0","netmask":"0.0.0.0","gateway":"0.0.0.0","m[/home/pjakobs/devel/esp_rgbww_firmware/app/application.cpp:554] Application::wsBroadcast
[/home/pjakobs/devel/esp_rgbww_firmware/app/webserver.cpp:137] === Websocket Broadcast ===
{"jsonrpc":"2.0","method":"wifi_status","params":{"message":"AP started","station":{"connected":false,"ssid":"","dhcp":true,"ip":"0.0.0.0","netmask":"0.0.0.0","gate[/home/pjakobs/devel/esp_rgbww_firmware/app/webserver.cpp:138] ===>nr of tcpConnections: 0
[/home/pjakobs/devel/esp_rgbww_firmware/app/webserver.cpp:143] =========================================
[/home/pjakobs/devel/esp_rgbww_firmware/app/webserver.cpp:144] ===>nr of websockets: 0
[Components/Network/src/Network/Http/HttpResourceTree.cpp:50] '*' registered
[Components/Network/src/Network/Http/HttpResourceTree.cpp:50] '/' registered
[Components/Network/src/Network/Http/HttpResourceTree.cpp:50] '/webapp' registered
[Components/Network/src/Network/Http/HttpResourceTree.cpp:50] '/config' registered
[Components/Network/src/Network/Http/HttpResourceTree.cpp:50] '/info' registered
[Components/Network/src/Network/Http/HttpResourceTree.cpp:50] '/color' registered
[Components/Network/src/Network/Http/HttpResourceTree.cpp:50] '/networks' registered
[Components/Network/src/Network/Http/HttpResourceTree.cpp:50] '/scan_networks' registered
[Components/Network/src/Network/Http/HttpResourceTree.cpp:50] '/system' registered
[Components/Network/src/Network/Http/HttpResourceTree.cpp:50] '/update' registered
[Components/Network/src/Network/Http/HttpResourceTree.cpp:50] '/connect' registered
[Components/Network/src/Network/Http/HttpResourceTree.cpp:50] '/ping' registered
[Components/Network/src/Network/Http/HttpResourceTree.cpp:50] '/hosts' registered
[Components/Network/src/Network/Http/HttpResourceTree.cpp:50] '/object' registered
[Components/Network/src/Network/Http/HttpResourceTree.cpp:50] '/stop' registered
[Components/Network/src/Network/Http/HttpResourceTree.cpp:50] '/skip' registered
[Components/Network/src/Network/Http/HttpResourceTree.cpp:50] '/pause' registered
[Components/Network/src/Network/Http/HttpResourceTree.cpp:50] '/continue' registered
[Components/Network/src/Network/Http/HttpResourceTree.cpp:50] '/blink' registered
[Components/Network/src/Network/Http/HttpResourceTree.cpp:50] '/toggle' registered
mode : sta(84:f3:eb:22:24:56) + softAP(86:f3:eb:22:24:56)
add if0
add if1
dhcp server start:(ip:192.168.4.1,mask:255.255.255.0,gw:192.168.4.1)
bcn 100
[/home/pjakobs/devel/esp_rgbww_firmware/app/application.cpp:162] Free heap: 25624
[/home/pjakobs/devel/esp_rgbww_firmware/app/application.cpp:359] Application::startServices
[/home/pjakobs/devel/esp_rgbww_firmware/app/ledctrl.cpp:406] APPLedCtrl::start
[Components/ConfigDB/src/Object.cpp:447] [CFGDB] open 'app-config/_root.json' failed
[/home/pjakobs/devel/esp_rgbww_firmware/app/eventserver.cpp:25] Starting event server with webserver referal
[Components/ConfigDB/src/Object.cpp:447] [CFGDB] open 'app-config/sync.json' failed
[Components/ConfigDB/src/Object.cpp:447] [CFGDB] open 'app-config/_root.json' failed
[Components/ConfigDB/src/Object.cpp:447] [CFGDB] open 'app-config/sync.json' failed
[Components/ConfigDB/src/Object.cpp:447] [CFGDB] open 'app-config/color.json' failed
[Components/ConfigDB/src/Object.cpp:447] [CFGDB] open 'app-config/_root.json' failed
[Components/ConfigDB/src/Object.cpp:447] [CFGDB] open 'app-config/sync.json' failedThe NotFound errors are fine, that just means you'll get the default store values.
understood - but should the file not be crated then?
Only on commit.
Also, you can remove the ARDUINO_LIBRARIES line from component.mk to fix the duplicate build target warnings.
The RGBWWLed library is a required componet, I removed ArduinoJSON and OtaNetwork, I guess they were still unchanged from the original version.
Libraries should be given in either COMPONENT_DEPENDS or ARDUINO_LIBARIES, not both: these two lists get merged by the build system.
I think the result is still much the same: I see
NotFoundon all but the first two stores and they are not created on the Host file system.
2007082 [CFGDB] Save 'network'
2007320 attached file: '' (0 bytes) #0xFFA9A498
2007509 [CFGDB] Object saved to 'app-config/network.json'Are you sure?
Am I right thinking that
fileSetFileSystem(&IFS::Host::getFileSystem());is all I need to enable the host filesystem?
Yes. Directories get created automatically (in Object::exportToFile).
mikee47
commented
1 month ago I made a change in #37 to Object::importFromFile which returns all file errors, including 'not found', so the caller could make a better decision about loading defaults, etc. Previously this silently succeeded, but the message is, while perhaps a little confusing, useful for debugging, and the 'not found' can be considered a warning or notification rather than an actual error.
pljakobs
commented
1 month ago ok, I understand now, just wonder why I didn't see those messages before.
mikee47
commented
1 month ago DEBUG_VERBOSE_LEVEL=3 ?
pljakobs
commented
1 month ago okay, that was user error - in a way.
I had, as a first pass, just mindlessly replaced every access to the app.cfg struct with an access to the corresponding CofigDB, not really thinking about call frequency much. Turns out that in the LED control loop, a set of seven properties located in three different stores are read ever 20ms.
I believe this is not a good use for ConfigDB at any rate, so I have created local variales that are read from ConfigDB upon initialization, not every 20ms.
Problem solved.
This does again make a case for a hook on updates to a specific property (or maybe a list of properties) as I would now need to call the AppLedCtrl::reconfigure() function when one of those values was changed
void APPLedCtrl::reconfigure() {
debug_i("APPLedCtrl::reconfigure");
{
AppConfig::Sync sync(*app.cfg);
clockMaster = sync.getColorMasterEnabled();
clockMasterInterval = sync.getColorMasterIntervalMs();
colorMaster = sync.getColorMasterEnabled();
colorMasterInterval = sync.getColorMasterIntervalMs();
} // end AppConfig::Sync context
{
AppConfig::Root config(*app.cfg);
transFinInterval = config.events.getTransFinIntervalMs();
colorMinInterval = config.events.getColorMinIntervalMs();
} //close configdb root context
{
AppConfig::Color color(*app.cfg);
startupColorLast=(color.getStartupColor() == "last");
} //close configdb context for color
}From a performance point of view: would it make sense to keep a map of existing ConfigDB stores in RAM? It feels a bit wasteful to check if the file exists only to then return the default if it does not. A bitmap for the defined stores would in most cases only take a byte, maybe two. That would not really fix my bad code here, as if the parameter was changed from default, it would have to be read from storage if it was not cached and since I was opening different stores, it would probably have been evicted quite often.
mikee47
commented
1 month ago I had, as a first pass, just mindlessly replaced every access to the app.cfg struct with an access to the corresponding CofigDB, not really thinking about call frequency much. Turns out that in the LED control loop, a set of seven properties located in three different stores are read ever 20ms. I believe this is not a good use for ConfigDB at any rate, so I have created local variales that are read from ConfigDB upon initialization, not every 20ms.
On the one hand yes, caching those variables locally makes sense, but on the other hand the application has to take additional steps to keep them in sync with the database.
If this data was all in the same store, and if ConfigDB could satisfy the request from cache, then that may well be good enough.
So if the database could be improved so that frequent accesses do not incur such a large hit then IMHO that would be useful.
From a performance point of view: would it make sense to keep a map of existing ConfigDB stores in RAM? It feels a bit wasteful to check if the file exists only to then return the default if it does not. A bitmap for the defined stores would in most cases only take a byte, maybe two. That would not really fix my bad code here, as if the parameter was changed from default, it would have to be read from storage if it was not cached and since I was opening different stores, it would probably have been evicted quite often.
Yes, a fairly simple optimisation to implement but as you point out the benefit of this would disappear once the configuration is customised... which I would expect in most cases.
So some revisions to read caching might be beneficial, such as:
Let's consider how much RAM does it requires to load your entire database into RAM?
root 18 general 19 color 64 sync 20 network 35
Total 156 bytes
This doesn't include the string data, or the general.channels and general.supportedColorModels arrays.
Still, we're not looking at massive amounts of RAM usage so keeping stores around for longer is probably not going to be an issue.
pljakobs
commented
1 month ago I think it would be great to have a setting to allows to configure how many stores are kept in RAM at any given time or evenn to pin stores in RAM, if so desired
pljakobs
commented
1 month ago also, I've managed to run the code on Host now (thank you for pushing me to finally do this, Esp32 should now also work) - and I believe there is a memory leak, given that the free heap goes down over time. Now I need to learn to interpret the valgrind output. Most of the errors it shows right now seem to be destructors upon terminating the application - which I don't care much about - if I terminate the application, I restart the chip. But I will need to troubleshoot this another day.
mikee47
commented
1 month ago
(I never know whether to open an issue or a discussion, this seems to be more an issue)
I have fully ported my application to use ConfigDB (which was a lot easier than I anticipated and makes the code super clean!)
However, I have two issues of which I do not know if they're configdb related:
a) I get frequent
LmacRxBlk:0messages - I was not able to figure out where they originate or what they mean (grepping for LmacRxBlk in /opt didn't yield any hit) b) after flashing the app for the first time I can access the UI for a while but then it stack dumps. After the restart, it keeps running into a stack dump over and over again. It's always a Watchdog Reset and the stack trace is, in those cases I checkd, always similar to the following: