peterhinch / micropython-mqtt

A 'resilient' asynchronous MQTT driver. Recovers from WiFi and broker outages.
MIT License
575 stars 126 forks source link

mDNS capability #14

Closed goatchurchprime closed 5 years ago

goatchurchprime commented 5 years ago

This would fix https://github.com/peterhinch/micropython-mqtt/issues/11

Note that the getmdnsaddr() function only gets called when the (server[-6:] == '.local')

This code calls the asynchronous function _as_mdns_send_read_udp() to handle the UDP send and receive, which is based on wan_ok().

Unfortunately I was unable to use sock.connect(); _as_write() because the sending only worked with sock.sendto(), and the connect() function didn't work. This might be a UDP thing, so I don't know how it operated in the wan_ok() function.

Also, I was unable to use _as_read() function because I don't know the length of the returning packet. (In any case, Datagrams are not streams, so you only get the packet.) Instead I had to use sock.recvfrom() which can be set to non-blocking, and it uses the same trick of await asyncio.sleep_ms(_SOCKET_POLL_DELAY) in a busy loop.

Additionally, I needed 4 mDNS packet parsing functions: mdns_createrequestpacket(), mdns_extractpackedname(), mdns_lenpackedname(), mdns_decoderesponsepacket()

My scrap-work area is here: https://github.com/goatchurchprime/jupyter_micropython_developer_notebooks/blob/master/basicsockets/mdnstest.ipynb

I don't expect this to be fully up to standard with the real micropython-mqtt code, so treat this as a reference implementation. (It works for me for now, though.)

The ESP8266 Arduino implementation of mDNS is utterly vast.
https://github.com/esp8266/Arduino/tree/master/libraries/ESP8266mDNS I can't quite find where in it is the equivalent of the code I have here. I based my code on an editing down of https://github.com/nickovs/slimDNS

peterhinch commented 5 years ago

I'm not really in a position to support mDNS. The point of this driver is resilience: proving that an mDNS implementation meets this requirement is frankly outside of my skill set. Further I'd struggle to deal with any mDNS issues which might arise.

Your best way forward may be to maintain and support your own fork.

goatchurchprime commented 5 years ago

Internally the library is very resilient. However, if we have to use hard-coded ip-numbers whenever we are interacting with a local MQTT-broker (because we can't resolve a ".local" hostname), then the total system becomes quite unstable. Unless the router has been configured to provide a fixed ip-number to the MQTT-broker device, this ip-number is liable to change, which breaks everything. Every single MQTT-client needs to be taken down and reprogrammed when the broker changes. Configuring router to give out fixed ip-numbers is also not easy.

Can you verify that my code does no harm, by inspection? This would just be as a second opinion, not for the purpose of actually pulling it in.

All it does is some asynchronous UDP sends and receives. The worst that could happen (from the standpoint of no knowledge of mDNS) is that the some random UDP packets are sent out to a port and get lost by the network, or the mDNS unpacking code is buggy and it doesn't resolve the ".local" name into ipnumbers in all cases.

At the moment the driver fails completely if you give it a server string that ends in ".local" (which by default are the only hostnames that mDNS resolves), because you are relying on socket.getaddrinfo(), which does not presently implement mDNS.

When mDNS eventually becomes part of Micropython, (Issue#201 ) then the issue will go away. I've verified that on CPython it is implemented, because socket.getaddrinfo("something.local") does work on my laptop, so it is reasonable to expect that it will eventually happen, and the problem will disappear.

But for now, this feature is essential when connecting devices to a local MQTT-broker without the ability to configure the router to make fixed ip-numbers. (MQTT-brokers out on the internet that can be found with a conventional DNS have always been fine.)

goatchurchprime commented 5 years ago

Thanks for taking a look.

According to this comment: https://stackoverflow.com/questions/13317532/receiving-a-part-of-packet-via-recvfrom-udp/13319158#13319158

UDP operates on messages, not streams like TCP does. There is a 1-to-1 relationship between sendto() and recvfrom() when using UDP. There is no option to receive partial data in UDP, it is an all-or-nothing type of transport.

I was looking for another source and discovered that the 'D' which stands for Datagram is a contraction of the phrase "Data Telegram", because it is like an old fashioned Telegram message where you either get the whole thing, or it simply goes missing.

As stated above, I tried very hard to make it the same as the wan_ok() method, but couldn't get those TCP-centric functions to work for UDP, as you somehow managed to.

peterhinch commented 5 years ago

Given that UDP operates on messages, so you'll either get all or nothing, it looks OK to me.

goatchurchprime commented 5 years ago

No longer required as it feature included in the base micropython system within the socket.getaddrinfo() function