on performance of packet rate

[GoogleCodeExporter]

Hi 
I use a powerful server with 10G eth port to generate traffics,but the max 
packet rate is about 400k packets/s, the packet length is fixed to 512bypes, 
that is about 200Mbps.
Are there any limitations on the rate?

Henry

Original issue reported on code.google.com by zhanghen...@yahoo.com.cn on 1 Apr 2011 at 2:53

[GoogleCodeExporter]

Henry,

There is certainly a lot of scope to optimize the code to improve the max 
packet rate that Ostinato can generate, but till that happens, you can browse 
through the following conversation on the mailing list for some tips to improve 
your rates with the existing code -

http://groups.google.com/group/ostinato/browse_thread/thread/1b75f4c83c2acae0#

I will keep this issue open as a placeholder for the time when code 
optimizations are done.

Srivats

Original comment by pstav...@gmail.com on 1 Apr 2011 at 1:56

• Changed state: Accepted

[GoogleCodeExporter]

Strivats,
I can't open the link, and could you directly send the related info to me?
thanks very much!

Henry

Original comment by zhanghen...@yahoo.com.cn on 2 Apr 2011 at 5:47

[GoogleCodeExporter]

Hi Strivats,
according to your info, I tried it again, and it can reach 83Kpps, see detail 
below:
1. Xeon 5500 8 cores, when test runing, idle 73%
2. OS is Fedora 10
3. the eth is 10Gbps port
4. memory 8G

no matter what kind of packetsize, the packet rate can't exceed 80-90Kpps.if 
the packetsize is 512bypes, the throuput is about 400Mbps. if it is 1518 bytes, 
the throuput can reach 1Gbps. but the port is 10G port. Are there some limits 
in codes?

Henry

Original comment by zhanghen...@yahoo.com.cn on 6 Apr 2011 at 8:14

[GoogleCodeExporter]

TBC
using tcpreplayer, under the same case conditions, the throughput can reach 
3-4Gbps.

Original comment by zhanghen...@yahoo.com.cn on 6 Apr 2011 at 8:18

[GoogleCodeExporter]

Henry,

I will try and take out some time this weekend to investigate this. Someone 
else on the mailing list also reported a problem with respect to performance of 
smaller size packets.

Meanwhile can you please clarify the following -

 - In the original report, you said you were able to generate 400Kpps of 512byte packets, but in your latest update you say for any packet size you get only 80-90Kpps?
 - With tcpreplay you report 3-4 Gbps => this is with what packet size? Can you tell me the max Kpps for 64byte packets that you can get with tcpreplay?

Regards,
Srivats

Original comment by pstav...@gmail.com on 6 Apr 2011 at 5:03

[GoogleCodeExporter]

Strivats,
it is 40kpps, not400k, sorry for the mistake.
the packet size is 1518bytes for tcpreplay 3-4Gbps.
for 64byte packet rate, i will try it.

Original comment by zhanghen...@yahoo.com.cn on 7 Apr 2011 at 6:59

[GoogleCodeExporter]

for 288bytes packet, the rate is about 735kpps

Original comment by zhanghen...@yahoo.com.cn on 7 Apr 2011 at 7:31

[GoogleCodeExporter]

Strivats,
what's the status of the issue?
in tcpreplay, there is a 'cache' option. if using the option, the performance 
improved significantly.

Henry

Original comment by zhanghen...@yahoo.com.cn on 11 Apr 2011 at 8:08

[GoogleCodeExporter]

Henry,

I looked at the tcpreplay and also libpcap, Qt code.

The major differences I found between tcpreplay and Ostinato in terms of 
sending packets were -

1. tcpreplay has several possible compile-time options to send packets on the 
interface - on Linux it typically will use PF_PACKET raw sockets; Ostinato uses 
pcap_sendpacket(). Looking at the libpcap code, on Linux, pcap_sendpacket() 
will most likely also use PF_PACKET raw socket only. So the only extra work 
that Ostinato is doing over tcpreplay is function call overheads. I don't think 
this will account for the performance difference between tcpreplay and Ostinato.

2. tcpreplay has several methods (-T option) for inter packet timing - by 
default it uses gettimeofday(). Ostinato, on the other hand, uses Qt's 
QThread::usleep() which in turn on linux uses pthread_cond_wait(). I didn't get 
a chance to look into pthread_cond_wait() yet - but it is very likely that 
pthread_cond_wait() does not use a hi-precision timer and also probably 
deschedules the host thread. 

3. tcpreplay uses "weighted averages" to achieve the specified rates at the 
risk of making the traffic bursty (see 
http://tcpreplay.synfin.net/wiki/tcpreplay#ChoosingaTimingMethod). This is 
planned but not yet implemented in Ostinato.

4. The tcpreplay 'cache' option (-K) preloads the pcap file into memory to 
avoid disk I/O. Ostinato already uses memory only.

If you are Ok with playing around with code, I suggest that you build Ostinato 
from source (see the Wiki for instructions) and try changing the Ostinato code 
to use gettimeofday() instead of QThread::usleep() and see if it helps.

The Ostinato code to change is in server/pcapport.cpp 
PcapPort::Transmitter:udelay()
The gettimeofday() timing code can be taken from 
http://tcpreplay.synfin.net/browser/trunk/src/sleep.h gettimeofday_sleep()

Please ensure that while comparing tcpreplay and Ostinato performance you are 
using the same/similar streams (at the very least the packet size should be the 
same)

I'm currently occupied with finishing the next Ostinato release 0.4 which is 
due later this month. I plan to look at packet rate performance and related 
issues in release 0.5.

Let me know the results of any experiments you do after changing the code - it 
will be useful for me when I look into these issues.

Regards,
Srivats

Original comment by pstav...@gmail.com on 11 Apr 2011 at 2:00

[GoogleCodeExporter]

Ok, I will try it.
thanks very much for your suggestion.

Henry

Original comment by zhanghen...@yahoo.com.cn on 12 Apr 2011 at 2:13

[GoogleCodeExporter]

This issue was closed by revision ebfb116c22a9.

Original comment by pstav...@gmail.com on 16 Oct 2011 at 2:44

• Changed state: Fixed