mDNS capability

[goatchurchprime]

This library is practically the whole justification of the async system, because you can make a sensor with an oled display do its stuff as soon as you turn it on, while all the nonsense of wifi connections and mqtt-broker finding is carried out in the background.

But could it also be upgraded to implement mDNS as well, as a further stage in its login? I've done my first hack of at this (sending and receiving UDP packets and decoding them) here:

https://github.com/micropython/micropython-esp32/issues/201#issuecomment-488094507

This would be really useful because I have a lot of these ESP32s running this asynchronous mqtt code, and it's nice to be able to connect them to different networks (whose ssid and passwords I can set) where they can find the mqtt.local broker reliably without each individually needing to be reprogrammed with the new ipnumber.

[peterhinch]

I'm not clear why this merits any change to my code. You use mDNS to get the server IP address, issue

config['server'] = the_IP_address

then instantiate the client.

Further, your mDNS solution appears to be synchronous. If I were to integrate mDNS it would really need to be asynchronous. But, unless I'm missing something, it seems to me that the issues of mDNS and asynchronous MQTT are orthogonal.

[goatchurchprime]

When I set up mqtt things on a local network, I've often got the hostname of the mqtt-broker, but not its ip-number, since this is dynamically allocated by the router. Sometimes I try to get this set up on someone else's router, or I have to use my phone hotspot as a router to the mqtt-broker (which is often on a RaspberryPI). It would be great if I didn't need to reflash the config file onto all my ESP32 devices with the new ip-number.

Now, the MQTTClient.connect() function asynchronously connects to the wifi and then connects to the mqtt-broker (and then it creates the tasks keep_connected and keep_alive task to redo these connections when they drop).

Obviously I cannot do any mDNS lookups (to convert the hostname of the broker into its ip-number) until after the wifi is connected, so this feature would need to be embedded into the mqtt_as system in order to take advantage of the fact that it's expertly handling both the wifi and broker connections in the background.

Now, there's a comment in the code worrying about the fact that the DNS lookup function is blocking:

            # Note this blocks if DNS lookup occurs. Do it once to prevent
            # blocking during later internet outage:
            self._addr = socket.getaddrinfo(self.server, self.port)[0][-1]

In other words, the DNS lookup is considered a thing.

In the case where the hostname ends with the letters ".local" there's a convention that it should use mDNS.

Fortunately, the necessary code is pretty similar to what is implemented by the MQTT_base.wan_ok() function. Here it sends the UDP packet:

b'$\x1a\x01\x00\x00\x01\x00\x00\x00\x00\x00\x00\x03www\x06google\x03com\x00\x00\x01\x00\x01'

to 8.8.8.8:53 and waits for a response that's 32 bytes long to confirm that the internet is working.

Accordingly, to make mDNS work it would need to pack together a similar string based on the hostname of the broker,

eg 'b\x00\x01\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x04mqtt\x05local\x00\x00\x01\x00\x01'

and instead send it to 224.0.0.251:5353, and then unpack the response

eg b'\x00\x01\x84\x00\x00\x01\x00\x01\x00\x00\x00\x00\x04mqtt\x05local\x00\x00\x01\x00\x01\xc0\x0c\x00\x01\x00\x01\x00\x00\x00\n\x00\x04\n\x00\x1e\xc2' which says that the ip-number for "mqtt.local" is 10.0.30.194

Now, I wouldn't trust my code on this; it was heavily hacked down from SLIMdns as a learning exercise to find out exactly what was going on behind the scenes. But the technicalities make it seem like a plausible addition to this library.

[peterhinch]

OK, I see what you want to do. Alas my experience of network coding is limited; I have zero experience of mDNS. It sounds as if you don't have a lot of confidence in this either so it's not clear how the task would be accomplished.

DNS lookup is indeed blocking. Paul (@pfalcon) has said that a nonblocking solution would be good but so far as I know nobody has written one. So the code does block for the getaddrinfo call. This is regrettable and I certainly don't want to introduce further blocking methods.

[goatchurchprime]

I'll make an attempt to implement it at some point.

But if you look at createmdnsrequestpacket() function you'll see it makes a byte string pretty similar to the one in wan_ok.

It might be simple enough to work out how to decode the 32 byte UDP packet it gets back to find the ip-number (instead of discarding it), and thus produce a non-blocking getaddrinfo call implementation in pure python.

The mDNS implementation (from the point of view of looking things up) seems to use compatible formats to the DNS, except it goes through port 5353 instead of 53.

We may need to look deeper than the micropython interpreter to find the actual implementation and see what's happening.

[goatchurchprime]

mDNS has now been included in the base micropython build https://github.com/micropython/micropython-esp32/issues/201