search

esnet / iperf

iperf3: A TCP, UDP, and SCTP network bandwidth measurement tool
Other
6.93k stars 1.28k forks source link

Add RTT measurement to iperf3? #635

Open AndyGermain opened 7 years ago

AndyGermain commented 7 years ago

NOTE: The iperf3 issue tracker is for registering bugs, enhancement requests, or submissions of code. It is not a means for asking questions about building or using iperf3. Those are best directed towards the iperf3 mailing list at iperf-dev@google-groups.com or question sites such as Stack Overflow (http://www.stackoverflow.com/). A list of frequently-asked questions regarding iperf3 can be found at http://software.es.net/iperf/faq.html.

Context

Please note: iperf3 is supported on Linux, FreeBSD, and macOS. Support may be provided on a best-effort basis to other UNIX-like platforms. We cannot provide support for building and/or running iperf3 on Windows, iOS, or Android.

Please fill out one of the "Bug Report" or "Enhancement Request" sections, as appropriate.

Bug Report

Please submit patches or code changes as a pull request.

Enhancement Request

If submitting a proposed implementation of an enhancement request, please use the pull request mechanism.

bmah888 commented 7 years ago

A few thoughts on this...

I'm interpreting this as a request for an RTT estimate for TCP tests (as opposed to UDP tests, which I kind of talked about recently in a response to #630). It looks like nuttcp does its RTT estimate by timing the length of time for a TCP connection establishment to complete (the first one?). That's one way to do this, though I have a few doubts about how represenative this single data point might be. It's not real easy for iperf3 to otherwise measure this during the data transfer because of TCP segmentation, queueing, etc., also the fact that the tests run are unidirectional (at the application layer anyway). Another possible (but non-portable) way to figure this out might be to grab something out of the struct tcpinfo structure.

As I commented in #630, there are other tools already in existence for measuring one-way and two-way delays.

AndyGermain commented 7 years ago

Yes, definitely TCP. I note that maybe there are multiple rtts. There is the "minimum rtt", which would occur when the network is otherwise idle, and all routers have the route cached. So my first choice would be this value, which would be the shortest definitive response observed in a session (example: packet n sent to the corresponding ack).

I think this "minimim" rtt is the "inherent" rtt (for that route, anyway -- the route can change during a tcp session.) Note that if the rtt starts increasing above that value, it's a sign of buffering in intermediate routers, which is a sign of network congestion- occurring earlier than dropping packets. I do not know if current tcp implementations make use of this concept -- maybe they should.

Anyway, timing the syn to syn-ack is MUCH better than nothing.

Thanks!

Andy Germain 301-801-4498 Sent from my iPhone6

On Sep 18, 2017, at 1:20 PM, Bruce A. Mah notifications@github.com<mailto:notifications@github.com> wrote:

A few thoughts on this...

I'm interpreting this as a request for an RTT estimate for TCP tests (as opposed to UDP tests, which I kind of talked about recently in a response to #630https://github.com/esnet/iperf/issues/630). It looks like nuttcp does its RTT estimate by timing the length of time for a TCP connection establishment to complete (the first one?). That's one way to do this, though I have a few doubts about how represenative this single data point might be. It's not real easy for iperf3 to otherwise measure this during the data transfer because of TCP segmentation, queueing, etc., also the fact that the tests run are unidirectional (at the application layer anyway). Another possible (but non-portable) way to figure this out might be to grab something out of the struct tcpinfo structure.

As I commented in #630https://github.com/esnet/iperf/issues/630, there are other tools already in existence for measuring one-way and two-way delays.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/esnet/iperf/issues/635#issuecomment-330292301, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AeEZj0pq9zRQ79esgbuBm3FbfskVjVN2ks5sjqY8gaJpZM4PJr6h.

rarylson commented 3 years ago

Hi guys,

It's worth to share that, currently (version 3.10.1+), iperf3 already exposes rrt and rttvar metrics for each time interval, and reports max_rtt, min_rtt and mean_rtt in the final/end report.

Note: I also tested in 3.7 (available in Ubuntu 20.04 apt), and rtt-metrics seems to be implemented in this version as well.

It's important to mention that these metrics are only available if you use the --json flag (JSON output mode).

So, in theory, this issue is already implemented and I guess @bmah888 (or any other committer) can safely resolve it.

Side notes:

jerrylususu commented 3 years ago

Just noting: RTT values reported in the JSON output is in usec (1e-6 sec). Reference: https://github.com/esnet/iperf/blob/332c31ee6512514c216077407a725b5b958b1582/src/tcp_info.c#L168

myers-dev commented 2 years ago

Just noting: RTT values reported in the JSON output is in usec (1e-6 sec). Reference:

https://github.com/esnet/iperf/blob/332c31ee6512514c216077407a725b5b958b1582/src/tcp_info.c#L168

Trying to understand why reported RTT is about 10x larger (if converted from usec to milisec) vs time reported by "ping" ?

                                        "max_rtt":      102051,
                                        "min_rtt":      53389,
                                        "mean_rtt":     85359,

PING 10.1.0.6 (10.1.0.6) 56(84) bytes of data.
64 bytes from 10.1.0.6: icmp_seq=1 ttl=63 time=7.82 ms
64 bytes from 10.1.0.6: icmp_seq=2 ttl=63 time=4.52 ms
robbertb commented 2 years ago

This is due to the standard package size in iperf3, in TCP this is 8KB, 1470 bytes for UDP. When using ping it is 56 bytes by default. Try specifying the --length flag in iperf3 and you will see similar RTT times. Note that you will never get exactly the same result because ping uses ICMP (which is a network layer protocol), whereas iperf3 is on the transport layer.

HenrikWahsner commented 2 years ago

@robbertb thanks for figure this out.

This lets the RTT shine in a complete different way!

We may have to discuss how this result is named, because in my understanding, Time to send a Block != RTT

Ask 100 people what RTT stands for, 99 will say: it is the "Round Trip Time" and mostly is an understanding for the single Paket forward and reverse time of any size. (compare ICMP 56bytes, or TCP SYN/ACK Time or even a ICMP Frame with 1500 Byte).

But RTT in iperf stand for the time a complete Block is being send and ACKed. If the Block is may 130K, the time is to send and ACK the complete Block, which is far away from what most People understand under RTT!

In short, the word RTT using for this measurement is just wrong and MUST be renamed to some better word, like "Block Transfer Time". Additionally, I again kindly asking to add the RTT measurement, wich was the initial request by the first person (AndyGermain). which should measure a singe packet forward and return time, may taken from init SYN/SYN-ACK (if possible). This would be great!

basil369 commented 2 years ago

This is due to the standard package size in iperf3, in TCP this is 8KB, 1470 bytes for UDP. When using ping it is 56 bytes by default. Try specifying the --length flag in iperf3 and you will see similar RTT times. Note that you will never get exactly the same result because ping uses ICMP (which is a network layer protocol), whereas iperf3 is on the transport layer.

How to use the --length flag to get values similar to ping? I used it like this ' -J -l 64 '. But it is still sending more than 64 bytes

davidBar-On commented 2 years ago

How to use the --length flag to get values similar to ping? I used it like this ' -J -l 64 '. But it is still sending more than 64 bytes

Per the ping man page -s option, the 64 bytes includes the 8 bytes of the ICMP header. Therefore, probably for TCP -l 44 should be used (TCP header size is 20 bytes).

Alternatively, if the intention is to compare RTT, then the ping packet size can be increased using -s packetsize.

mstovenour commented 2 years ago

It looks like there is some misunderstanding in this thread regarding the nature of RTT measurements. Ping does not measure true network RTT. Ping measures the time required to send the payload in both directions because it sends the full payload data in both directions (echo). That is not the true network RTT but approaches the network RTT for very small payload sizes. A single iperf3 socket is sending the full payload data in only one direction so ping and iperf3 will never report the same rtt values even though they are both measuring the data transit time. Measuring the syn/ack RTT, as nuttcp does, is even closer to the true network RTT because it is the smallest IP payload size achievable within a TCP socket. Effectively there is nothing wrong with the RTT measurement if iperf3, but it is important to understand the conditions it is measuring.

If iperf3 were to try to report something close to the nuttcp value, it could record the TCP socket structure "min_rtt" value. This value is normally latched just after socket open (unless there is high variability and the syn/syn-ack/ack transactions were unlucky). You can see this value for yourself by using the "ss -it" Linux command while an iperf3 test is underway.

Delivering this enhancement request could be as easy as recording the TCP socket structure min_rtt after opening the socket. That value could be recorded in the JSON "start.connected[n]" structure so it is not confused with the min_rtt in the data transfer statistics. IMO, reporting this would remove the confusion over the other rtt measurements by making it clear that this is the min_rtt at connection vs. the other measurements being conducted during data transfers.