iperf not working on top of TAS

[ash995]

I am not able to run iperf on top of TAS. When ever I try and run iperf on top of TAS, the program stalls and there is no output on the screen. Although, I am able to run the benchmarking repo code just fine.

[vsag96]

Not associated with the repository. It seems that not all socket options are offered out of the box(https://github.com/tcp-acceleration-service/tas/blob/b72710c7c96d233493e138c03b556ce47da0c0e0/lib/sockets/control.c#L757) I think this is something that should be fixed from iperf's end. You cannot run iperf I am afraid(Unless I am completely wrong) with my analysis.

[FreakyPenguin]

Sorry for the delay, just digging myself out of pile of things that accumulated over the OSDI deadline.

@vsag96 is probably right, this likely has to do with a missing socket emulation feature. TAS has sufficient sockets support to run quite a few unmodified apps, including memcached, redis, and even some java apps. But every once in a while we stumble across another missing feature with a new application.

The setsockopt warnings you might or might not be able to ignore (they seem to be for SO_SNDTIMEO, SO_RCVTIMEO, and SO_TIMESTAMP). Not sure which of these will be fatal for iperf. Might be worth just starting with a backtrace on both ends to see where things hang. You can also try running linux on one end and then seeing if it's just on the server or just on the client side to narrow it down. After things are stuck you can also run sudo tools/statetool to get a dump of the fast path connection state on both sides. That shows among other things how much pending data is in send and receive buffers for each connection as well as the sequence numbers, which at least with TAS on both sides will start at 1 or 2, so that might also tell you if something is stuck and if anything has been sent and received.

I'll try to have a quick look when I get the chance to see if I can find something. But if this is something trickier then I probably won't get around to any fixes before the hotnets deadline on the 15.

If you just want some basic numbers in the meantime to compare linux and TAS, you might want to start with the micro_rpc and micro_unidir microbenchmarks here: https://github.com/tcp-acceleration-service/benchmarks

[ash995]

@FreakyPenguin The problem seems to be in receiving the a response from the server. The tx sequence number on the server end is 2, and the rx sequence number on the client side is 2. So, the client is able to send out some data which is received by the server, but the server is not able to write back to the client. A similar thing happens when I run a simple client server application on top of TAS. This is the link to client code, and server code.

[kuretru]

--- a/src/iperf_server_api.c
+++ b/src/iperf_server_api.c
@@ -550,7 +550,7 @@ iperf_run_server(struct iperf_test *test)

    if (result > 0) {
             iperf_time_now(&last_receive_time);
-            if (FD_ISSET(test->listener, &read_set)) {
+            if ((test->state == IPERF_START || test->state == IPERF_DONE) && FD_ISSET(test->listener, &read_set)) {
                 if (test->state != CREATE_STREAMS) {
                     if (iperf_accept(test) < 0) {
            cleanup_server(test);

or

diff --git a/context.c b/context.c
index 35af511d..4da5eb65 100644
--- a/context.c
+++ b/context.c
@@ -xxx,y +xxx,y @@ static inline void ev_listen_accept(struct flextcp_context *ctx,
     sl = s->data.connection.listener;
     assert(sl != NULL);

-    flextcp_epoll_set(sl, EPOLLIN);
+    if ((sl->flags & SOF_NONBLOCK) == SOF_NONBLOCK) {
+        flextcp_epoll_set(sl, EPOLLIN);
+    }

     if (ev->ev.listen_accept.status == 0) {
         s->data.connection.status = SOC_CONNECTED;