Asynchronous reading from and writing to the network

[johanstokking]

I need to be able to publish to the MQTT broker directly when an event is available in my application. This is not possible in the two current implementations of using this library:

• MQTTYield() only returns when a message has been received or when there is a time-out. I cannot cancel the wait so that I can publish
• Using the worker thread, there is a read time-out of 500 ms while a mutex is locked. This same mutex is locked when publishing a message

I propose using threads or socket callbacks to separate read and write operations.

Using Threads

The user thread is the only thread that performs write operations. The read thread spawned by the library is the only thread that performs read operations. The read thread is the producer of incoming messages, and the user thread is the consumer of them.

In the case when a response is required after a write operation, the user thread waits with a time-out on the read thread to put the response message in a shared variable and release a lock on it (produce) so that the user thread can read and process it (consume). If the read thread reads a non-response message (e.g. a publish from the broker), then it should call back to the user application.

So what the read thread does is:

while (true) {
    msg_t m = receive();
    if (is_response_type(m)) {
        lock(message_mutex);
        message = m;
        unlock(message_mutex);
    } else {
        message_received(m);
    }
}

Using Callbacks

This would work similar to threads, but there instead of having a thread that reads continuously, there is a callback set that gets called when data is available on the socket. This may be a more efficient way of working with sockets as they don't need a thread, but it requires the operating system's sockets to support these callbacks. Since this library is intended for embedded use, I would not require this to be supported by the network interface.

[icraggs]

Hello Johan. Thanks for opening this issue. Last year I started adding threading into the client for the FreeRTOS port. I think that continuing and optimizing this approach for Linux and FreeRTOS is worth exploring, which I will try to do this week.

On the callback option, can you explain what you mean by "I would not require this to be supported by the network interface"? I'm not sure I understand.

[johanstokking]

Thanks, I'm looking forward. If you need testing or anything, let me know.

I mean that we shouldn't require socket callback support in the network interface (like NetworkInit()), because it may not be supported. I think there's better support for threading than for socket callbacks, but I'm not an embedded operating system's expert.

[johanstokking]

BTW, in my particular use case, I can work with FreeRTOS.

What can I do to make this working using threading?

[icraggs]

Hi Johan. Sorry I've been distracted by other work, including the Paho C and embedded MQTT-SN clients. I'll try to get to this issue this week. I can work with Linux in the first place - that would be easier for me anyway. I intend to make it work the same way on Linux and FreeRTOS. You did say that you use another OS - was it OpenWRT? Are you using the Linux interface at the moment?

[johanstokking]

@icraggs would be great. I'm very much looking forward. We're using a PIC32 with Microchip Harmony but turns out that it runs FreeRTOS. I'm currently using Linux indeed; it should there as well in my use case.

[icraggs]

Can you tell me the version of FreeRTOS used? And what TCP stack?

[johanstokking]

We're using FreeRTOS v8.x, and a TCP/IP library from Microchip Harmony. The latter is kind of exotic but we've got it working. It's doable to implement the network abstraction in there and I'll contribute it to this project.

[ammaree]

@icraggs any progress on the async client? I agree with @johanstokking that the thread solution (as opposed to callbacks) is better supported in the RTOS space. We use FreeRTOS v9.0.0 with 2 different TCP/IP stacks, SimpleLink on CC3200 and Cyclone/TCP on BCM43362. Threads can work in both environments, callbacks not.

In a current CoAP based implementation we have used slightly different roles for the 2 threads.

Receive/read thread: receives all incoming requests (GET/PUT/POST/DELETE) and responses (ACK etc) and completely act on these up to the point (if required) of producing a response packet. At this stage an entry is put into a message queue for the transmit thread.

Transmit/write thread: takes as input entries from it message queue and transmits these. Handles all retransmission (if required) and correctly frees up (if any) dynamically allocated memory buffers.

In this scenario the configuration can be tuned based on resources (available memory), performance (thread priorities for receiving and transmitting volumes) and various other considerations.

We would like to start a migration from CoAP to MQTT (with paho mqt) but having asynchronous client communications is a must.

Thanks

[johanstokking]

@ammaree I got it working, see https://github.com/thethingsnetwork/paho.mqtt.embedded-c

I'm planning a pull request. We changed the code formatting though, which was very inconsistent, so loads of diffs, but you'll get the point.

[icraggs]

@johanstokking As we discussed, please make a pull request. I'm happy to take some time to figure out how to merge.