search

OpenFastPath / ofp

OpenFastPath project
BSD 3-Clause "New" or "Revised" License
349 stars 126 forks source link

TCP transmission failed to send packets #219

Closed bestgen closed 5 years ago

bestgen commented 5 years ago

I built a simple three-point network to use odp&ofp (both are master version) to forward packets. The central point runs ofp&odp as a router, and the other two runs iperf as client and server, without odp$ofp. UDP transmission is good. When using iperf through ofp & odp to transmit TCP packets, the client can only send several packets after connection established and then it nearly failed to continue. The speed is very slow. I use wireshark to capture the packets and do some tests, and find:

  1. If TCP packets is larger than the MSS(1460 Bytes), it will trigger TCP retransmission. Each time the ACK from server expects a 1448 length increasing, while the packet length from the client increasing fast. After several packets, TCP flow will be out of order. In other words, the ACK from receiver limits the TCP packets increasing and leads to transmission fail.
  2. If I set TCP packets no larger than MSS, the receiving port can receive expected packets, but the packets captured from transmitting port are still larger than MSS. It seems very possible that the ODP or OFP reassemble the packets. (wireshark shows IPv4 fragmentation flag is not set)

Is there anyone have some ideas about this problem, or have got similar situation before? Thanks.

JereLeppanen commented 5 years ago

Regarding large packets, if segmentation offloading is enabled, captured packets may be the unsegmented packets, not the packets on the wire. Please check ethtool -k to see whether offloads are enabled.

On the client side, please ascertain that offloads are disabled, then capture from connection establishment to the slowdown or failure, and send the capture file.

bestgen commented 5 years ago

Thank you for your help @JereLeppanen . You are right, the packets captured are not the real packets on the wire. The TCP transmission problem is caused by TCP and IP segmentation functions of system. After I closed the GSO and TSO, it works well. Thanks again.