please support interleaved send/recv

[adferguson]

hi all,

while doing some throughput testing, I noticed that LWT lacks fairness between competing light weight threads. because OCaml lacks support for hardware parallelism, this situation can easily lead to starvation.

the scenario is as follows:

1) we have an OpenFlow switch which is sending many packets to the controller (eg, ARP requests) 2) for every packet received by the controller, the controller emits a packet (eg, ARP replies)

this is the design of a basic ARP cache.

in OpenFlow0x01_Switch.ml, we have two light weight threads: recv_thread and send_thread which should be responsible for parts 1 and 2 above. each is implemented as an infinite recursion.

the problem is that LWT appears by default to only yield to a different thread when a system call doesn't return right away. therefore, because there is always something in the receive buffer, recv_thread hogs the sole OCaml hardware thread, and send_thread starves.

it would be a big improvement if recv_thread and send_thread were interleaved to prevent starvation in such scenarios. of course, this only solves the problem for a single switch, and for as many light-weight threads as the authors see fit to interleave -- the proper solution is to have a language with hardware parallelism. :-(

thank you, Andrew

PS -- to see / reproduce this, I have two xterms on the same host in mininet. in the first xterm, I do arping -c 200 -w 500000 h2 while in the second, I have tcpdump -i h1-eth0 ... from the tcpdump, we can clearly see that all 200 arp requests are received before the 200 replies are sent (even though our my logging emits an arp reply for every arp request incrementally). hence, arping reports that 100% of the arp queries are unanswered (since they arrive after the 200 requests have all been sent).

[seliopou]

Closing and marking as wontfix, as this is lwt-related.