Horrible performance and frequency disconnections with Pico W? Disable power management

[Jibun-no-Kage]

Post this as a feature request, here, so that the following can be integrated to the mqtt_as base class of object init scope as a configuration option, i.e. power management enabled or disabled.

Rationale, for this request, I was having random disconnects (although reconnect worked well) and very odd performance of MQTT subscribed messages and publishing that was significantly delayed. Took a bit of testing and researching to find that the Pico W was the key source of the issue, not my environment. Moreover, default power up configuration of a Pico W is with WiFi power management enabled. This really impacted MQTT communication of course, as I discovered.

theWiFi=network.WLAN(network.STA_IF)
theWiFi.active(True)
theWiFi.config(pm=0xa11140)
Echo('WiFi! Power Management {0}'.format(thePower))
theWiFi.active(False)

The above code disables power management for WiFi interface. Unfortunately I don't see a way, supported or otherwise, to query the power management setting. But by doing the above just before creating the MQTT client object (via mqtt_as) radically improved the performance of MQTT communication. removing a 5 to 8 delay when MQTT subscriptions accepted by Pico W.

True, routine pinging of the broker might avoid this issue as well, but again that would have to be added to mqtt_as, right? But I see that as generating additional network traffic, which IMHO is not preferred, no?

Setting the keepalive configuration option DID NOT address the 5 to 8 second lag. I did try that as well, but did not expect that to address the issue, just tested it as a sanity check.

[peterhinch]

routine pinging of the broker might avoid this issue as well, but again that would have to be added to mqtt_as, right?

Regularly issuing MQTT Ping packets is part of the MQTT protocol and is how the Last Will mechanism works. If the broker does not receive any packets from the client within the keepalive period it assumes the client has died and sends the Last Will. The mqtt_as library keeps track of when the last message was sent to the broker. If nothing was sent in 1/4 of the keepalive time it sends a Ping. So, if you set your keepalive time to 60s there is a guarantee that something is sent to the broker at least every 15s.

Re the power management I'm not keen on magic code :) Please could you point me at some documentation which explains and justifies your power management code. I will try to replicate your issue - in my testing so far the Pico W works fine but I'll try a long running test.

[peterhinch]

I've had the range_ex.py demo running for four hours without issue. This test keeps the interface busy - I run the pubtest script on a PC so the target sends and receives constantly.

I'm guessing that power saving kicks in when the interface is lightly used - which relates to the keepalive time and the resultant ping interval.

I don't think I can do anything further or comment meaningfully without documentation on the power save mode.

[Jibun-no-Kage]

Right, here is the documentation source where I found the dlue to what was happening... https://datasheets.raspberrypi.com/picow/connecting-to-the-internet-with-pico-w.pdf

Reference section 3.6.3 has a code snippet that turns off power management for WiFi, not much detail, but it does work. Maybe you can reference it as a known issue, or such, and add an option at MQTT class init to turn off power management if desired (a suggestion).

Since I am also doing a range_ex like thing in my code, I turn the power management off before my scan routine, then I pass the selected network bssid of the selected scan result to connect and test internet access, i.e. wan_ok like, as well as query NTP.

Once all the house keeping is done, I let mqtt_as logic take over. It is a bit hockey because mqtt_as re-establishes the connection but that is ok, because to change mqtt_as to accept an existing connection would change it core goal/function, and I figure what I am doing is unique use case scenario.

[peterhinch]

I have pushed an update to disable auto sleep by default.

We already do something similar here for ESP8266. Some users have applications which subscribe only and expect a reasonably quick response from rare messages, so this would undoubtedly have come up again as an issue.

Thank you for raising this and pointing me to the docs.

[ebolisa]

@peterhinch

I have pushed an update to disable auto sleep by default.

Hey Peter, can the option of able/disable the auto-sleep be added to the config.py file so that if the lib is modified it would not be overwritten by a future updates? Thx.

[Jibun-no-Kage]

The interesting thing is, I discovered the lag (power management lag) when I was generated message to the Pico will within the keepalive and Mosquitto time limit for disconnect, I was sending the messages via manual injection via Node Red, noticed the lag, then setup a message every 15 seconds, and still the first message would not be bounced back via publish until 5 to 8 seconds delta because the subscribe took that long, i.e. for the Pico W to wake up. That is not long and should not have cause the broker to disconnect the connection, as I understand it. No other device I have, and I have many (over 25) that interact with the same broker exhibited this subscribe receive lag, only the Pico W had the issue as noted. The minute I disabled the power management on Pico W explicitly (per the code/example I provided) the lag disappeared and everything since has been working quite well. No more lag as before.

My solution or use case could have lived with the lag, but it just did not make sense, hence I started digging. I really would suggest that you make it a config option, if you agree to do so, maybe set the config to power management enabled as the default, so users of the module can disable power management as needed? I have both use cases applicable so such flexibility would be appreciated, by me if not many others?

In a few days I will have an original Pico with an ESP01 for WiFi setup, and will test that as well, not expecting it to have any issues, but maybe saying that, I jinxed it? LOL.

[peterhinch]

not long and should not have cause the broker to disconnect the connection

It certainly shouldn't trigger the keepalive. If you use qos==1 possibly the broker has a timeout on receiving the ack.

I really would suggest that you make it a config option

I'm not keen. The primary aim of mqtt_as is reliability and, as you have demonstrated, power management causes breakages. The library has a lot of users some of whom are inexperienced. I can see power management being a support nightmare. On ESP8266 power management caused various hard to diagnose behaviours. My strong feeling is to disable it.

[Jibun-no-Kage]

I can see and appreciate your point. Moreover, I have yet to find a way to re-enable it. It is really odd that the documentation clearly illustrates how to disable it, but lacks information on how to enable it. Begs the question that someone thought it being enabled would generate issues and questions, does it not? :) Ok, it gets disabled. If someone really wants it enabled, they would have to have the skill and knowledge to customize mqtt_as. I could, I may, create a personal customized copy, at some point, but since I don't plan to run an Pico as yet on battery, not in the near future.

[peterhinch]

For battery operation of mqtt_as see the docs. In general uasyncio does not play well with low power usage.