TCP sends excessive amount of TCP window update messages

[psavol]

TCP sends a window update message every time application calls ofp_recv(). Window update is meant to be sent when there's a long delay between TCP receiving the data and application receiving it from the socket. In our case, the TCP performance test application does nothing else but calls ofp_recv() in a tight loop. Also according to Wireshark the receive window size is always between 450 ... 470 kB, so not even close of running out.

The window update messages are generated here: src/ofp_uipc_socket.c:

` /*

• If ofp_soreceive() is being done from the socket callback,
• then don't need to generate ACK to peer to update window,
• since ACK will be generated on return to TCP. / if (!(flags & OFP_MSG_SOCALLBCK) && (pr->pr_flags & PR_WANTRCVD)) { SOCKBUF_UNLOCK(&so->so_rcv); (pr->pr_usrreqs->pru_rcvd)(so, flags); SOCKBUF_LOCK(&so->so_rcv); }`

When we commented out this hunk, TCP receive throughput improved about 25%, as iperf/kernel stack did not any more receive these confusing / extra messages.

Before: [ ID] Interval Transfer Bandwidth [ 3] 0.0-10.0 sec 6.96 GBytes 5.98 Gbits/sec [ 3] 10.0-20.0 sec 6.55 GBytes 5.63 Gbits/sec [ 3] 20.0-30.0 sec 7.06 GBytes 6.06 Gbits/sec [ 3] 30.0-40.0 sec 8.90 GBytes 7.64 Gbits/sec [ 3] 40.0-50.0 sec 7.03 GBytes 6.04 Gbits/sec [ 3] 50.0-60.0 sec 7.07 GBytes 6.07 Gbits/sec

After: [ ID] Interval Transfer Bandwidth [ 3] 0.0-10.0 sec 8.24 GBytes 7.08 Gbits/sec [ 3] 10.0-20.0 sec 9.31 GBytes 8.00 Gbits/sec [ 3] 20.0-30.0 sec 9.91 GBytes 8.52 Gbits/sec [ 3] 30.0-40.0 sec 8.55 GBytes 7.34 Gbits/sec [ 3] 40.0-50.0 sec 8.68 GBytes 7.45 Gbits/sec [ 3] 50.0-60.0 sec 8.73 GBytes 7.50 Gbits/sec

[JereLeppanen]

On the current master HEAD, this change improves throughput (as reported by iperf/tcpperf) by only 1-1.5 percent in a basic test case with tcpperf as the server. And there are other cases where this change has a significant negative impact on throughput.

Perhaps things have changed since this issue was opened.

I'm thinking of closing this as wontfix, unless someone has better ideas.

[RadulescuValentin]

Hi, We did measurements using webbench and webserver2: Case 1. TCP Window updates are sent: webbench -c 2000 -t 10 http://192.168.1.4:2048/ Benchmarking: GET http://192.168.1.4:2048/ 2000 clients, running 10 sec. Speed=2244984 pages/min, 710907 bytes/sec. Requests: 374164 susceed, 0 failed. Case 2. Window updates not sent: webbench -c 2000 -t 10 http://192.168.1.4:2048/ Webbench - Simple Web Benchmark 1.5 Copyright (c) Radim Kolar 1997-2004, GPL Open Source Software. Benchmarking: GET http://192.168.1.4:2048/ 2000 clients, running 10 sec. Speed=2249670 pages/min, 712393 bytes/sec. Requests: 374945 susceed, 0 failed. So, we could notice an 0,2 % increase without the window update messages. Also, run same tests against lighttpd and there were not any TCP window updates coming from the server.

[smititelu]

Hi,

Today I have sent a patch that fixes this issue, in order to be reviewed. In my tests I've used tcpperf as a server, compiled with #define SOCKET_RX_BUF_LEN (4 * 1024), and iperf as a client. I have obtained the following test results:

1. Before the patch:

   > iperf -c 192.168.20.11
   ------------------------------------------------------------
   Client connecting to 192.168.20.11, TCP port 5001
   TCP window size: 85.0 KByte (default)
   ------------------------------------------------------------
   [  3] local 192.168.20.10 port 43296 connected with 192.168.20.11 port 5001
   [ ID] Interval       Transfer     Bandwidth
   [  3]  0.0-10.0 sec  7.86 GBytes  6.75 Gbits/sec

2. After the patch:

   > iperf -c 192.168.20.11
   ------------------------------------------------------------
   Client connecting to 192.168.20.11, TCP port 5001
   TCP window size: 85.0 KByte (default)
   ------------------------------------------------------------
   [  3] local 192.168.20.10 port 43306 connected with 192.168.20.11 port 5001
   [ ID] Interval       Transfer     Bandwidth
   [  3]  0.0-10.0 sec  9.24 GBytes  7.94 Gbits/sec

The improvement is maintained on each run of the test.

The improvement is maintained when testing tcpperf compiled with #define SOCKET_RX_BUF_LEN (32 * 1024), which is the default value.

Thank you, Stefan