Multi-threaded or Single-threaded network interactions for federated programs

[erlingrj]

A design decision on this question must be taken soon because it is blocking further development of federated reactor-uc programs. The question is whether we should use a dedicated thread for each network input which does blocking receive on the network API (underneath it is Sockets, BLE, etc), or if we should have a single runtime thread which polls the network inputs in a non-blocking way at certain points.

Advantages of multithreaded

• Simplifies the implementation
• Trivial to let main runtime thread go to sleep and be awoken when there is new data on a network input.

Advantages of single-threaded

• No reliance on an OS
• Less memory overhead
• Easier to prioritize between network inputs and handle local events
• Avoid risk of concurrency bugs
• No need to expand the platform API to include threads, mutexe, cond vars/

Clearly there are several advantages of doing everything in a single thread. But we don't yet know if it is even feasible. We need the runtime-thread to go to sleep waiting for the next event on the event queue but be awoken if it arrived data on a network input. How can we ensure this wake-up?

A potential possibility is using a timed wait on a mutex. When data arrives, surely there will be interrupts firing and if we can hook a function into the ISR that releases a mutex which the runtime thread is doing a timed wait on, then it would work. But is there a generic portable way of specifying this? We want to build on Unix Sockets, Ethernet frames, BLE, LoraWAN, Zephy, RIOT,.

If we weren't targeting OSes as Zephyr and RIOT we could have used the wait for event or wait for interrupt pseudo-instructions available on arm and risc-v, but this gets in the way when we are trying to build on Zephyr and RIOT which don't expose a sleeping function which returns if there occurs any interrupt.

Because of these considerations I am inclined to go for the multithreaded variant.

Happy for any feedback on this

[erlingrj]

We concluded to go for a callback-oriented API where we defer the question of threads to each NetworkBundle implementaton