OTGW Web server slow with MQTT enabled v0.9.0

[mihsu81]

Hi @rvdbreemen,

After an OTA upgrade from v0.8.5 to v0.9.0 the Web server became very slow. After disabling MQTT it got fast again. Re-enabling MQTT makes it slow again. I've downgraded from v0.9.0 to v0.8.6 and didn't see web server slowness anymore with MQTT enabled. In v0.9.0 MQTT appears disconnected and in Mosquitto broker i see successful connection attempts and disconnections every few seconds. I've also noticed that in v0.9.0 Board type changed from _ESP8266NODEMCU to _ESP8266NODEMCUUnknownboard

P.S. There's a typo as well, should be Thermostat instead of Thermostate.

Thanks in advance for your help in getting this issue solved. :)

[rvdbreemen]

@mihsu81 thanks for reporting this issue. First of all, thanks for pointing out the typo. I just fixed it in the dev branch. So in a future release, so that should be gone. Also any reference to Dutch language should be fixed. If you ever find one, let me know.

Now on the MQTT connect issue in v0.9.0 I am stumped for now. Would you be willing to do some testing to find out that it could be? That would require you to update to v0.9.0 again, and then capture some logging on port 23 (telnet). Maybe the debug log would give us more information about the issue.

Hope you are willing to invest some time in this.

[mihsu81]

Hi @rvdbreemen,

Here's a capture from the device on port 23. Hopefully will help. debug_log.txt

[rvdbreemen]

@mihsu81 from the log I can tell you that you are running the MQTT and an OTGW native integration at the same time over port 25238. As far as I can see this causes a reboot from time to time, I have never tested this particular scenario.

The firmware was designed to support only one option at a time, either you use serial over network OR MQTT.

To test my suspision here would you be so kind to run the following test:

1. Turn off native OTGW integration in HA, and check to see if you can find the MQTT device for OTGW
2. Turn off the MQTT option in the firmware, and turn back on the native integration in HA.

In either case you should see the OTGW device working. You might need to restart HA in between tests.

Please let me know your results.

[mihsu81]

Hi @rvdbreemen ,

I've been running both integrations at the same time since i first installed your firmware (v0.8.5) and never had these issues or errors/warnings in HASSIO.

Case 1 Removing the native OTGW integration and restarting HA maybe improved things by a small margin but the web server is still very slow. Much slower than on v0.8.6 with both integrations enabled. To the point where it would fail to display even Device Info.

Here are more logs and my Mosquitto broker settings (which are pretty basic). debug_logs.txt

I forgot to mention, but it still publishes data to MQTT even with both integrations enabled. Just randomly displays "MQTT connected" false or true. And still tries to reconnect every few seconds.

Case 2 When disabling MQTT and using the native OTGW integration the web server is fast and no issues are encountered. P.S. I have to modify settings.ini, reupload it and restart the NodeMCU in order to disable MQTT. Saving settings from the Web Server seems to time out, or maybe I'm not waiting long enough. 😁

[rvdbreemen]

Thanks for the testing and sharing the results.

First off. Well good to hear you can run both integrations at the same time. Didn't know that.

Will look into this more over the weekend.

[rvdbreemen]

I just updates to release 0.9.1 that tried to solve some of the issues we saw with the 0.9.0 release. Not sure if I fixed the MQTT reconnect good enough. Could you try the latest release and let me know.

[rvdbreemen]

In the current beta I have been working on improving "slowness" of the webserver, by improving the whole handling of web, mqtt, serial in the main loop. Hopefully this will fix your issue. I will leave it open as part of 0.9.2 beta phase.

[mihsu81]

Hi @rvdbreemen, Sorry for the late reply. Version 0.9.1 did improve things a bit but still didn't reach the responsiveness of v0.8.x. I'll try v0.9.2 when released. Thanks.

[rvdbreemen]

No problem. We keep moving the project forward. The v0.9.2 beta release is much more responsive as some change were made to improve that. If you join the community the you can help test the beta.

[mihsu81]

I've compiled the main branch, and flashed it, but it appears to be as slow as v0.9.1. At this point I'm beginning to wonder if there's something wrong on my side. 😁 I'll do a factory reset and see if it helps at all.

[rvdbreemen]

Well,if you compile the mail branch you will get v0.9.1 ;-) You can simply join the discord channel and pick up the binary buids from the beta tester channel. But if you building your own binary, please switch to the dev branch, that's where the beta v0.9.2 lives at this point.

[mihsu81]

Yeah, just realized 15 minutes ago that it was still v0.9.1. 🤣 I should have payed better attention. 😜 Sorry. I've switched to the dev branch and i can see now that the build is 0.9.2-beta+491dd26. But i'm seeing the same issue. I'll try the builds from your discord channel. If those don't help either I'll flash it over UART and erase it before. It's crazy that even when only MQTT traffic is enabled the web server is still slow (and I cannot save settings).

[rvdbreemen]

Just let me know when you get to the discord, because I can add you to the betatester channel.

[rvdbreemen]

Slowness what caused by incoming topics repeatedly firing with home assistant online. This should not happen, but happened anyway, this was fixed by correctly detecting a restart cycle, you need to see HA go offline, and then come back online.