Cannot configure uasyncio to support micropython-mqtt

[cefn]

Hi there,

Was excited to see this library and was moving towards using as a substitute for https://github.com/micropython/micropython-lib/tree/master/umqtt.simple with more robustness. However, I can't seem to get it to connect and respond at all. This seems to do with not having the right uasyncio setup but it's not obvious where this is documented.

I added the mqtt_as.py file in the ports/esp8266/modules folder and then built an image from source, taking care to put the uasyncio directory from micropython-lib in there as a frozen module too.

After flashing a Wemos D1 Mini with this new image like... esptool.py --port /dev/ttyUSB0 --baud 1500000 write_flash --flash_size=32m 0 build/firmware-combined.bin ...the library successfully imports as expected. I then used ampy to put a suitable main.py in place.

However, I couldn't initially recreate similar topic-publish-acknowledgement behaviour as my successful test case... https://github.com/cefn/retrotextual/blob/master/gist/mqtttest.py using the reference code https://github.com/peterhinch/micropython-mqtt/blob/master/mqtt_as/range.py

My modified version of range.py is mqttpulse.py and it simply changes the server and topic to match the test case I am already successfully running for umqtt.simple (which also relies on the implicit cached connection to Wifi, and is running in exactly the same deployed configuration).

The umqtt.simple mqtttest.py version responds by echoing the topic and message to serial, but the range.py version is non-responsive after boot, regardless of trying to trigger its message acknowledgement by calling...

mosquitto_pub --topic 'hello' --message 'world' --retain

...which successfully triggers a response from umqtt.simple

On boot my mqtttest.py initially reported the following...

ets_task(40100130, 3, 3fff837c, 4)
scandone
state: 0 -> 2 (b0)
WebREPL daemon started on ws://192.168.4.1:8266
WebREPL daemon started on ws://0.0.0.0:8266
Started webrepl in manual override mode
Traceback (most recent call last):
  File "main.py", line 16, in <module>
ImportError: no module named 'config'

MicroPython v1.9.3 on 2017-12-01; ESP module with ESP8266
Type "help()" for more information.
>>> state: 2 -> 0 (2)
reconnect
scandone
state: 0 -> 2 (b0)
state: 2 -> 3 (0)
state: 3 -> 5 (10)
add 0
aid 7
cnt 

connected with Kitchen2, channel 11
dhcp client start...
ip:192.168.43.201,mask:255.255.255.0,gw:192.168.43.1

I then realised that the config.py file needed to be placed in the modules directory too.

After putting that in place, I received a boot error (from uasyncio?), extracted from the boot sequence as below...

  File "main.py", line 24, in <module>
AttributeError: 'module' object has no attribute 'get_event_loop'

To fix this, I populated the path uasyncio/core.py in modules with the file from...

https://github.com/micropython/micropython-lib/blob/master/uasyncio.core/uasyncio/core.py

However, it doesn't change the error.

Switching to import uasyncio.core as asyncio fixes the error superficially, but just creates a new one as uasyncio.core doesn't have an implementation of the sleep() coroutine, so I get...

AttributeError: 'module' object has no attribute 'sleep'

...instead.

What's the way of unpicking this, as there doesn't seem to be any guidance at...

https://github.com/micropython/micropython-lib/tree/master/uasyncio

...and the top level guidance for micropython-mqtt simply says...

The only dependency is uasyncio from the MicroPython library. Ensure this is installed on the device.

[cefn]

Incidentally, I tried building the unix port and getting reference information that way about the module structure for uasyncio, but after successfully building installing and running (unix) micropython, I get results like...

>>> import upip
>>> upip.install('uasyncio')
Installing to: /home/cefn/.micropython/lib/
Warning: pypi.python.org SSL certificate is not validated
Error installing 'uasyncio': Package not found, packages may be partially installed

...which looks a lot like https://github.com/micropython/micropython/issues/2589 then...

>>> upip.install('uasyncio.core')
Installing to: /home/cefn/.micropython/lib/
Error installing 'uasyncio.core': Package not found, packages may be partially installed

Please note I am building against the version of micropython by checking out git tag '1.9.3' which I understand to be the latest stable.

$ git log
commit fe45d78b1edd6d2202c3544797885cb0b12d4f03 (HEAD -> retrotextual, tag: v1.9.3)

[cefn]

OK, so the Warning regarding the SSL certificate is not an Error, so it is not responsible for the issue. In fact the same warning appears when I successfully installed it with this different invocation after fixing my stupid...

micropython -m upip install micropython-uasyncio

This of course reflects the fact that the micropython modules are in python's own pip, so have to be separately named with the micropython- prefix.

[cefn]

Finally I have...

ip:192.168.1.64,mask:255.255.255.0,gw:192.168.1.254
Checking WiFi integrity.
Got reliable connection
Connecting to broker.
Connected to broker.
We are connected to broker.
(b'hello', b'world')

So the key thing was to build the unix port by cding into ports/unix in the micropython repository as follows in my case...

make clean && make axtls && make
sudo make install

After successfully installing the unix port of micropython I then ran...

micropython -m upip install micropython-uasyncio

...and inspecting the folder .micropython/lib/uasyncio I found there was an additional file sync.py ...

$ ls .micropython/lib/uasyncio/
core.py      __init__.py  sync.py

I then changed to the unix library uasyncio directory and put everything from there overwriting the files in the ports/esp8266/modules/uasyncio folder (the mqtt_as.py and config.py remained in the modules folder from before). In my case this looked like...

~/.micropython/lib/uasyncio$ cp *.py ~/Documents/shrimping/git/micropython-image-retrotextual/ports/esp8266/modules/uasyncio/

After that I changed into the ports/esp8266 folder and re-ran...

export PATH=/home/cefn/Documents/shrimping/git/esp-open-sdk/xtensa-lx106-elf/bin:$PATH
make clean && make axtls && make
esptool.py --port /dev/ttyUSB0 --baud 1500000 write_flash --flash_size=32m 0 build/firmware-combined.bin

And after rebooting and installing my mqttpulse.py as main.py by running...

ampy --port /dev/ttyUSB0 put ~/Documents/shrimping/git/retrotextual/code/cockle/mqttpulse.py main.py

I had success!

[peterhinch]

The official documentation could use improvement for the case where a library must be frozen. You ended up with the same method I use, except that installing uasyncio on Unix also installs errno.py and logging.py which I also copy across.

[cefn]

Surprisingly there is no errno.py or logging.py or at least I don't know where they would show up. The only contents of my .micropython folder at this time...

$ find .micropython/ -type f
.micropython/lib/uasyncio/core.py
.micropython/lib/uasyncio/__init__.py
.micropython/lib/uasyncio/sync.py

[peterhinch]

I've just checked by doing a fresh upip install to an empty directory and you are correct: it no longer includes those files. So ignore my remarks about them :)