search

perfsonar / pscheduler

The perfSONAR Scheduler
Apache License 2.0
55 stars 34 forks source link

option --mss=1500 fails for iperf2 #1268

Closed rhclopes closed 2 years ago

rhclopes commented 2 years ago

Setting maximmum segment size for iperf2 seems to fail for all values. Also fails if II use '-m 1500'. It works nicely if the tool is iperf3.

pscheduler task  --slip PT10S --debug  --tool iperf2 throughput  --source x.y.net --dest a.b.edu  -t 60 -i 60   --congestion=htcp -P 6 -w $((256 * 1024 * 1024)) -m 1500
Diagnostics:
  /usr/bin/iperf -p 5001 -c a.b.edu -t 60 -m -i 60 -P 6 -w 268435456 -M 1500 -Z htcp

Limit system diagnostics for this run:
  Hints:
    requester: 194.81.18.227
    server: 145.146.0.6
  Identified as everybody
  Classified as default
  Application: Defaults applied to non-friendly hosts
    Group 1: Limit 'innocuous-tests' failed: Passed but inverted
    Group 1: Limit 'throughput-default-tcp' passed
    Group 1: Limit 'throughput-default-udp' failed: UDP testing not allowed
    Group 1: Limit 'idleex-default' failed: Test is not 'idleex'
    Group 1: Want any, 1/4 passed, 3/4 failed: PASS
    Application PASSES
  Proposal meets limits
  Priority set to default of 0

Error from ps2.netherlight.net:
  iperf2 returned an error: WARNING: attempt to set TCP maximum segment size to 1500, but got 536
  WARNING: attempt to set TCP maximum segment size to 1500, but got 536
  WARNING: attempt to set TCP maximum segment size to 1500, but got 536
  WARNING: attempt to set TCP maximum segment size to 1500, but got 536
  WARNING: attempt to set TCP maximum segment size to 1500, but got 536
  WARNING: attempt to set TCP maximum segment size to 1500, but got 536

MTU setting works for iperf 2 without pscheduler. On a local test be

[raul@setebos ~]$ iperf -c 192.168.220.1 -t 5 -i 1 -M 1500 -e
WARNING: attempt to set TCP maximum segment size to 1500, but got 536
------------------------------------------------------------
Client connecting to 192.168.220.1, TCP port 5001 with pid 240017 (1 flows)
Write buffer size: 131072 Byte
TCP window size: 85.0 KByte (default)
------------------------------------------------------------
[  1] local 192.168.220.4%ens2f1np1 port 48816 connected with 192.168.220.1 port 5001 (MSS=1488) (sock=3) (ct=0.41 ms) on 2022-02-19 16:26:53 (GMT)
[ ID] Interval        Transfer    Bandwidth       Write/Err  Rtry     Cwnd/RTT        NetPwr
[  1] 0.00-1.00 sec  1.46 GBytes  12.6 Gbits/sec  11986/0         73     1195K/232 us  6771112
^Bz[  1] 1.00-2.00 sec  1.53 GBytes  13.1 Gbits/sec  12502/0         16     1114K/230 us  7124618
[  1] 2.00-3.00 sec  1.54 GBytes  13.2 Gbits/sec  12624/0          0     1413K/233 us  7101515
[  1] 3.00-4.00 sec  1.32 GBytes  11.4 Gbits/sec  10850/0         23     1406K/225 us  6320583
[  1] 4.00-5.00 sec  1.50 GBytes  12.8 Gbits/sec  12253/0         22     1256K/244 us  6582071
[  1] 0.00-5.01 sec  7.35 GBytes  12.6 Gbits/sec  60217/0        134     1256K/244 us  6456139

Thanks, Raul

mfeit-internet2 commented 2 years ago

Iperf2 sets the MSS with a call to setsockopt() and doesn't appear to be getting the value it asked for, which is between it and the kernel. We don't maintain either.

Iperf2 doesn't consistently set a non-zero exit code in the face of certain errors and the only reliable way we have to know if it encountered a problem is the presence of something on the standard error, which is the case here. That said, if the the test spec wants a MSS of 1500 and the tool was unable to set it, any data we get back from iperf2 is based on the wrong parameters and should be considered invalid and therefore a measurement failure.

Other than reporting the problem to the iperf2 maintainers, there's not a whole lot the perfSONAR team can do to fix it.

rhclopes commented 2 years ago

iperf issues an unfortunate warning, but it seems to run with the requested mss.

[raul@callisto ~]$ iperf -c 192.168.220.4 -w $((256 * 1024 *1024))  -e --mss=1500
WARNING: attempt to set TCP maximum segment size to 1500, but got 536
------------------------------------------------------------
Client connecting to 192.168.220.4, TCP port 5001 with pid 438485 (1 flows)
Write buffer size: 131072 Byte
TCP window size:  512 MByte (WARNING: requested  256 MByte)
------------------------------------------------------------
[  1] local 192.168.220.1%enp175s0f0np0 port 43340 connected with 192.168.220.4 port 5001 (MSS=1488) (sock=3) (ct=0.49 ms) on 2022-02-22 19:23:19 (GMT)
[ ID] Interval        Transfer    Bandwidth       Write/Err  Rtry     Cwnd/RTT        NetPwr
[  1] 0.00-10.37 sec  11.2 GBytes  9.30 Gbits/sec  92009/0         86     1662K/373 us  3118028
[raul@callisto ~]$ iperf -c 192.168.220.4 -w $((256 * 1024 *1024))  -e --mss=2000
WARNING: attempt to set TCP maximum segment size to 2000, but got 536
------------------------------------------------------------
Client connecting to 192.168.220.4, TCP port 5001 with pid 438491 (1 flows)
Write buffer size: 131072 Byte
TCP window size:  512 MByte (WARNING: requested  256 MByte)
------------------------------------------------------------
[  1] local 192.168.220.1%enp175s0f0np0 port 43342 connected with 192.168.220.4 port 5001 (MSS=1988) (sock=3) (ct=0.38 ms) on 2022-02-22 19:23:59 (GMT)
[ ID] Interval        Transfer    Bandwidth       Write/Err  Rtry     Cwnd/RTT        NetPwr
[  1] 0.00-10.32 sec  14.3 GBytes  11.9 Gbits/sec  117122/0         43     2079K/622 us  2390548
[raul@callisto ~]$ iperf --version 
iperf version 2.1.4 (18 August 2021) pthreads

When I run through pscheduler, I get an error, butthe monitoring system shows the test running. The error I get

[raul@ps-slough-10g iperf2-streams]$  pscheduler task --slip PT3M --tool iperf2  throughput  --source $src --dest $bnl -t60 -i 60  --congestion=cubic  -w $((256 * 1024 * 1024)) -P 4 --mss=4000
Submitting task...
Task URL:
https://source.net/pscheduler/tasks/253e227c-e243-4dc1-bf27-1b5059af6014
Running with tool 'iperf2'
Fetching first run...

Next scheduled run:
https://source.net/pscheduler/tasks/253e227c-e243-4dc1-bf27-1b5059af6014/runs/03dc427e-8d74-4d14-b156-515b72e6d0b9
Starts 2022-02-22T20:26:37+01 (~4 seconds)
Ends   2022-02-22T20:27:43+01 (~65 seconds)
Waiting for result...

Run failed.

Error:
  iperf2 returned an error: WARNING: attempt to set TCP maximum segment size to 4000, but got 536
  WARNING: attempt to set TCP maximum segment size to 4000, but got 536
  WARNING: attempt to set TCP maximum segment size to 4000, but got 536
  WARNING: attempt to set TCP maximum segment size to 4000, but got 536
mfeit-internet2 commented 2 years ago

I looked at the code in iperf2 before writing my earlier comment and everything I said there still applies. The example you pasted in has the same problem:

[raul@setebos ~]$ iperf -c 192.168.220.1 -t 5 -i 1 -M 1500 -e
WARNING: attempt to set TCP maximum segment size to 1500, but got 536

Iperf2 should fail right there, but it does a measurement anyway and produces a result that, according to its own code, doesn't reflect the requested MSS. I can't see my clear to making the iperf2 plugin ignore that.

When I run through pscheduler, I get an error, butthe monitoring system shows the test running.

That's the correct behavior. pScheduler hands the work to the plugin and considers the test running until the plugin returns a result. Tool plugins that run an external program let it run to completion and make a success/failure decision based on the output. The iperf2 plugin treats this situation as a failure and pScheduler marks the run as failed after getting the plugin's result:

Run failed.
mfeit-internet2 commented 2 years ago

After consultation with other developers, we've decided to drop it since there's not much we can do with iperf2.

Release Notes:

Support for the mss throughput parameter has been dropped from the iperf2 plugin because iperf2 doesn't produce accurate measurements with it in effect.