search

peterhinch / micropython-mqtt

A 'resilient' asynchronous MQTT driver. Recovers from WiFi and broker outages.
MIT License
549 stars 116 forks source link

access to the socket instead of the queue? #124

Closed karlp closed 8 months ago

karlp commented 8 months ago

I realize it won't work with any sort of reasonable queue length, but the current behaviour pretty much precludes receiving large messages, because it forces reading the entire message into memory before putting it into the queue and then out to the client. (note, this applies to the callback method as well, as the message is read before deciding whether to use the callbacks or the queue, but I've only used the queue api)

It would be nice to be given the unread socket, along with the decoded payload size, so that we could chunk reads out of it.

I've currently resorted to having my MQ layer sending http URLs, as the urequests library lets me chunk out the request, 1k at a time...

karlp commented 8 months ago

I mean, this would probably only be feasible with the callback API, but would it be worth it?

peterhinch commented 8 months ago

It's probably not stated in the docs but the design does assume short messages. Long messages have other implications. A purpose of MQTT is to achieve reliable communication over an unreliable medium. The longer the message, the higher the probability of failure and, if qos==1, the greater the likelihood of retransmission. As a general rule it's better to break a long message into shorter parts.

That said, the MQTT spec places no limit on message length.

As for passing the socket as part of the API I don't think this has legs. Firstly the socket is nonblocking: the user would need to understand how to use this in asynchronous code. Secondly the API would need a way to report successful reception to enable the qos==1 mechanism to work, and would need to be able to handle failure.