Extending WiFiManager To Be A Generalist Webserver

[kentaylor]

This is in response to posts on Issue #154 which had drifted off topic.

@datacute I haven't tested whether the "The existing support for additional parameters" still works after my changes. It would be great if you did that.

@batika and @datacute the solution written by John Lassen is built on the WebServerExample in a similar way to WiFiManager. Major differences being it is a sketch rather than a library and the web server is intended to be run continuously.

I don't think serving web pages is the right way to use these devices. I think they should be minimalist and all interactions, even configuration should be through a cloud based server which updates the device. Ultimately even the firmware should be generated on the server and uploaded over the air. An early version of this is https://www.kickstarter.com/projects/seeed/wio-link-3-steps-5-minutes-build-your-iot-applicat . I realise others will not agree.

However, cloud based server interaction can only happen after connectivity is established and I envisage WiFiManager to be a tool to enable that connectivity and not much more. It should be minimalist, only take up memory when required and disappear without trace once it's job is done. That is why @cjamin the config portal is not always open. Currently when it is open it blocks the rest of the sketch so must be closed to return control to the sketch. If a web server is required I would start it again in the sketch and use the Arduino WebServerExample as a template. You probably only want to see this server on the WiFi Network to which the ESP8266 is connected, not on it's own network as well. The only gotcha is that for some reason the web server can only be started once per boot up. The work around is to reboot after configuring the network, which takes seconds.

@cjamin "Access point becomes flaky if set in dual mode and not successfully connected to network so change to access point mode only when there is no network connection" is unrelated to keeping the config portal open. The web server still works but becomes very slow in these circumstances causing long delays and the browser to sometimes time out.

The web server even when running well is not great. It will handle only a single request and queue subsequent requests. If you were to send it a wifi scan request then send a few more because it was taking too long it does them one after the other, but your browser is only waiting for the last request it made so will be not listening for the web pages returned by earlier requests. This can take so long to complete that the browser times out and it looks to a user like the web server failed. I saw this in user testing which is why it doesn't go directly from the root page to the page that does WiFi scanning anymore. People would click on the WiFi configure page then when the response was a bit slow click on the same button over and over. That would queue the requests and eventually the browser timed out. These are the sort of issues you have to think about when you use the ESP8266 as a web server. Better to use a normal, high capacity, cloud based web server in my view.

[cjamin]

Thank you for your detailed answer. So, to sum up, the ESP8266 is not powerful enough to properly run a web-server while doing other things in parallel.

I agree that this library should stick to WiFi configuration to be as light-weight as possible.

I like your "button" solution when one wants to edit the WiFi configuration that is stored. Another option would be to run the ESP8266 in dual mode just after booting, only for a limited period of time (e.g. 1 minute). That would allow for reconfiguration at startup. Of course, this should only be an option in the API. Does that seem reasonable?

PS: I have just received my ESP8266 recently and did not have time to test it yet. But I'm willing to use this library to configure the WiFi.

[datacute]

@kentaylor Additional parameters still work.

I'm coming around to your point of view too, with regard to using a cloud based server for the things you'd otherwise be tempted to use a web server on an ESP8266 for.

The reason I started using WiFiManager was twofold:

1. So that I could run the same code on multiple devices (in different locations, using different WiFi networks), and configure/individualise them (such as naming the location).
2. So that I could publish my code, and other people would be able to make use of it without having to replace all of my custom settings (such as my MQTT server).

If I was creating a product, I'd have a static location I could use to manage everything apart from the WiFi details, but when publishing code for others to use, that needs to be configurable too.

That's all possible now, it's just that there's a lot of code to write/copy to get the additional parameters working. I'm now keen to shift my focus to making that easier, while I get some of my devices updated and start testing long term connectivity behaviour.

With the additional parameters working, it still makes sense to me to use them for all the other non-WiFi related settings that I'd want to set up roughly once for each device, which doesn't need a web-server running all the time, so this issue is no longer an issue for me.

Going off track again... I also have to find some more buttons and free gpio ports, and so like @cjamin I've been trying to come up with some other way of launching the config. I've been toying with the idea of using multiple reboots within a time period to trigger configuration mode. So at t=0 check eeprom/FS for a flag, if not set then config mode with a timeout, else clear flag, normal mode, and 10 seconds later set the flag again... which doesn't work if your device goes into deep sleep mode long before then, and resets itself out of it... but in that case you also need a way of telling the device to not go into deep sleep mode anyway, and a button seems like the best option for that. (You could also check a web service.) If I come up with something that works, I'll create an example.

[datacute]

(Double posting since I remembered the other reason I was looking for WiFiManager to be extended as generalist web server.)

One of my use cases is for an internet enabled clock.

I have not yet set up an interrupt to drive the 'tick', but have my loop method checking WiFi connectivity, MQTT connectivity, button presses to control brightness, and then seeing whether enough time has passed that I should update the clock display.

I want the clock to turn on and start ticking even while it is being configured with WiFiManager.

One hack I implemented was to supply another callback to the WiFiManager for it to call within its busy loop, but I was thinking it would work better with WiFiManager being able to be setup, then called by my loop, and then closed down when no longer needed.

@kentaylor I think you are saying that there's no point checking WiFi connectivity, and trying to re-establish connectivity, as the ESP8266 does that automatically anyway, and that I really shouldn't be firing up WiFiManager whenever WiFi drops out? If so then I think a callback to enable some background work to be performed (at an appropriate time) while WiFiManager is running, is probably a reasonable solution to keep my clock ticking instead of appearing hung.

[kentaylor]

@cjamin I've created an example using the scenario you suggested. It creates a configuration portal for 1 minute after startup then closes the portal and returns to the sketch. See https://github.com/kentaylor/WiFiManager/blob/dev/examples/ConfigOnStartup/ConfigOnStartup.ino . This is a development branch and you will need to replace the whole library to test it. Please test and provide feedback.

Also @cjamin the ESP8266 is powerful enough to run a web server, depending on what you need it for. It couldn't serve up Facebook but it might do for your purposes. I don't think it is a good scenario though for several reasons. Another reason is that many sensors and control loops require precise timing. Multitasking with a web server can stuff that up and definitely complicates things. I like the way you have total control of the processor and can choose when to yield to the WiFi currently. #158 mentions another web server that can support concurrent connections which might suit your purposes.

@datacute There is now a good example for capturing parameters at https://github.com/kentaylor/WiFiManager/blob/dev/examples/ConfigOnSwitchFS/ConfigOnSwitchFS.ino developed by @battica.

[cjamin]

Thank you for the example! Sure, I will test it and provide feedback. From what I understand, the call to startConfigPortal is blocking. Would there be a way to make this asynchronous?

[kentaylor]

@cjamin asks "Would there be a way to make this asynchronous?"

We are not running an operating system so there is no thread or process scheduling. This is a good thing in this sort of device in my view. It makes it better than a Raspberry Pi for instrumentation and control, again in my view. Others don't agree and Espressif distributes an RTOS SDK for the ESP8236. However, a downside of not having an operating system is that making WiFiManager asynchronous would be hard to do. It also makes it hard to provide a good web server.

@elieyal in #104 wants this too.

[cjamin]

It doesn't have to use threads, it could be something based on interrupt handlers or something like that. Of course, it could jeopardize the real-time guarantees, but it would be the programmer's choice to accept it or not.

I'm not familiar yet with the ESP8266, so maybe I'll first experiment with it and see what would make sense or not when I am...

Thanks for the answers anyway!

[kentaylor]

@cjamin Yes I agree, which is why I said "hard to do" rather than can not be done. Losing the real time guarantee is why I think it is also not desirable.

The WiFi needs regular attention. If the WiFi is left alone too long a non overrideable timer will reset the device, which annoys a lot of people. In Arduino the WiFi gets attention when you call delay(some number) or yield(). A lot of people don't like this but I think, for instrumentation and control, it is clever because you get total control. You could do something similar with WiFi manager which is what the web server does within WiFiManager. It gets serviced when called. I think a blocking WiFiManager is better.

Using the button to initiate scenario, WiFiManager will only ever run when human initiated and you don't swap WiFi networks very often so disabling an application during WiFi configuration seems not too inconvenient. This is what Google Chromecast does, the Withings scale and the Fitbit scale. I also wondered whether non blocking would be good but came down on the side of blocking for all these reasons. Others may not agree and for some usage scenarios I can see that blocking isn't ideal.

[cjamin]

I agree that the best solution is using the button, when possible. Thanks for your work.

[kentaylor]

@datacute says "I think you are saying that there's no point checking WiFi connectivity, and trying to re-establish connectivity, as the ESP8266 does that automatically anyway, and that I really shouldn't be firing up WiFiManager whenever WiFi drops out?"

Yes, as long as the ESP8266 is set to autoconnect which it is by default. If in doubt about autoconnect check with WiFi.printDiag(Serial);

[datacute]

I created a library for detecting a double reset, saving the need for a button to trigger entering "config mode": https://github.com/datacute/DoubleResetDetector

I'm using it with the changes to WiFiManger made by @kentaylor except with one further change to fix a scenario, that I think will be of interest... I had been using a version prior to the changes to allow timeout again. When entering config mode, the SSID & password were cleared. This meant that you couldn't return to normal operation without re-entering the config details. The changes to allow timeout also resulted in running in dual STA & AP mode, but half the time I couldn't connect to the configuration server. I wondered if it was only listening on one of the two IP addresses. The behaviour that I'm currently happy with is to not wait for a connection result, but to call: WiFi.persistent(false); WiFi.disconnect(true); WiFi.persistent(true);

This results in clearing the config, but not forgetting it, so if I accidentally get into config mode, I can just reset it again, and it'll reconnect to its last WiFi network.

[pieman64]

@kentaylor I'm not quite sure where to post issues with your branch so I will post them here. I have added 8 new parameters prior to your parameters and I hadn't realised that you have to click the Exit Portal link for the details to be saved. So it failed first time but appears to have worked second time around.

Not sure if I removed anything that I shouldn't, unlikely as I was just adding extra parameters, but the tick box for the temperature sensor in the portal appears to have disappeared and been replaced with the box that has the "T" in it. I didn't put a T anywhere in the portal. See screenshot below:

Any ideas what has happened to the tick box? Edit: Having looked more closely at your sample sketch I can see where the "T" comes from.

[pieman64]

I will just add that I have had the 192.168.244.1 address appear already and I have only used the portal 3 or 4 times. So I guess this is not fixed. What we have done in our sketches is flash the on board LED 3 times if normal 192.168.4.1 address comes up, 6 times for ...244.1 and no times if no address comes up (meaning reset device and try again).

[kentaylor]

@pieman64 glad you got the checkbox sorted.

According to https://github.com/esp8266/Arduino/blob/c52088c774bc570507485f2ac6d19f4ebdf6da25/libraries/ESP8266WiFi/src/ESP8266WiFiAP.cpp#L162 the IP is set to 192.168.244.1 when the result of the first effort to set the IP was "[AP] IP config Invalid ".

You could change this line to set it to 192.168.4.1 instead of 192.168.244.1. Then you wouldn't have to detect which address it is.

[kentaylor]

I didn't understand the explanation at https://github.com/tzapu/WiFiManager/issues/160#issuecomment-241237648 but running in dual STA & AP mode is a problem when not connected to an access point for the reasons explained at http://bbs.espressif.com/viewtopic.php?t=671#p2531 . Could it be that "couldn't connect to the configuration server" occurred when the ESP8266 was not connected to an access point but was trying to do so? The described fix causes it not to try and connect to an access point which would stop the issues described at http://bbs.espressif.com/viewtopic.php?t=671#p2531 .