"iperf3: error - unable to read from stream socket: Resource temporarily unavailable" after running PTP4l

[NicoEspla]

Context

• Version of iperf3: Iperf 3.1.1

• Hardware: macB cadence, Power PC E550

• Operating system (and distribution, if any): Linux 4.1.38

Bug Report

• Expected Behavior I am trying to make iperf3 work with PTP4l at the same time, I expect to have clock synchronization while testing the network bandwidth.

• Actual Behavior When running iperf3 without running ptp4l I get a successful network test:

Then, If I run PTP4l, I can have the clocks synchronization working:

If I try to run iperf3 again, I got "iperf3: error - unable to read from stream socket: Resource temporarily unavailable":

Running tcpdump I see one pattern:

This is the server side, on the left the capture before running PTP4l, on the rigth after running PTP4l

This is the client side, on the left the capture before running PTP4l, on the rigth after running PTP4l. In this capture I see: -In the left side a message is sent from the client to the server, then appear the "Connecting to host..." msg, and finally a second message is sent from the client to the server. -In the rigth side a message is sent from the client to the server, then a second message is sent from the client to the server, and finally appear the "Connecting to host..." msg.

I always see this pattern when running ptp4l.

• Steps to Reproduce -Run Iperf3 -Run PTP4l -Run again Iperf3

Can someone help me to find the reason for this? or maybe give me some guidance to try to figure out why I am getting this issue?

Best regards

[davidBar-On]

The tcpdumps info (which is very useful) shows that the client successfully sent on the control stream (TCP) the cookie (seq 1:38) and the parameters (seq 42-140) and that the server acked receiving them. The server also received the first 4 bytes UDP message - the Connect message, but for some reason did not reply to it.

The following information may help further evaluation:

1. Run the client and server with the following parameters to get more info -V -d --timestamp.
2. How long does it take between the client "Connection to host ..." message and the the "iperf3 - error: unable ..."? Mainly, does the error occur immediately or it seems that it happens after a timeout?
3. After running the ptp4l , will a second try of running the client/server/both also fails?
4. Is it required that ptp4l will be run on both sides to cause the error, or is running it only on the client or server machines enough to cause the error?

.... finally appear the "Connecting to host..." msg. I always see this pattern when running ptp4l. ...

Although it is strange that it always happen when running ptp4l, it seems to happen only because of the output to screen buffering. Per tcpdump, in both cases it takes about 48ms between the ... ack 2, win 440 ... and the ... ack 42, win 437 ... messages.

[NicoEspla]

The tcpdumps info (which is very useful) shows that the client successfully sent on the control stream (TCP) the cookie (seq 1:38) and the parameters (seq 42-140) and that the server acked receiving them. The server also received the first 4 bytes UDP message - the Connect message, but for some reason did not reply to it.

The following information may help further evaluation:

1. Run the client and server with the following parameters to get more info -V -d --timestamp.
2. How long does it take between the client "Connection to host ..." message and the the "iperf3 - error: unable ..."? Mainly, does the error occur immediately or it seems that it happens after a timeout?
3. After running the ptp4l , will a second try of running the client/server/both also fails?
4. Is it required that ptp4l will be run on both sides to cause the error, or is running it only on the client or server machines enough to cause the error?

.... finally appear the "Connecting to host..." msg. I always see this pattern when running ptp4l. ...

Although it is strange that it always happen when running ptp4l, it seems to happen only because of the output to screen buffering. Per tcpdump, in both cases it takes about 48ms between the ... ack 2, win 440 ... and the ... ack 42, win 437 ... messages.

Hi David! I really appreciate your help on this....

Answering to your questions:

1. Run the client and server with the following parameters to get more info -V -d --timestamp. I got an error if I run the command with the "--timestamp" flag: iperf3: unrecognized option `--timestamp'

This is the result of running iperf with the other flags:

2. How long does it take between the client "Connection to host ..." message and the the "iperf3 - error: unable ..."? Mainly, does the error occur immediately or it seems that it happens after a timeout? It seems to be a time out, if I measure the time the error message is thrown after ~30 seconds of getting "Connection to host ..." message

3. After running the ptp4l , will a second try of running the client/server/both also fails? Yes, After running ptp4l I cannot get iperf working until I restart the machine. In the other case, if I do not run ptp4l I can get iperf working every time I run it.

4. Is it required that ptp4l will be run on both sides to cause the error, or is running it only on the client or server machines enough to cause the error? I got different results.

If I run ptp4l only on the server side I get the same result.

If I run ptp4l only on the client side I get the result shown below. What I notice here is that iperf is not ending after 3 seconds (as specified by -t 3) instead it just keep running and printing the same message in the server side.

Best regards

[davidBar-On]

Hi @NicoEspla, it seems that ptp4l cause some side effects on the server's machine, although it is terminated. I am not sure what they may be, but few suggestion to help with the analysis:

1. Try using other port. E.g. just the default port 5021. It may be that ptp4l somehow affects this port, although I didn't find any clue that that may be the case.
2. Try to run ptp4l on the server machine without terminating it (run it in a different shell window or as a service). This is to verify whether the problem happens when ptp4l runs or just when it is terminated.
3. You are using a very old version of iperf3. It may help if you will be using a newer version.
4. In case you are using an old ptp4l version it may help to upgrade it.

[davidBar-On]

@Genericreqd,

1. Does the problem happen also for TCP tests, i.e. when you run the client without the -u option?
2. Can you run PTP4l without canceling it before running iperf3? I.e. run PTP4l and iperf3 in separate windows, and don't cancel PTP4l before starting iperf3. This is to check whether PTP4l locks some time resource that is used by the UDP stack, and this resource is not being released when PTP4l is canceled.
3. You are using a very old version of iperf3. If possible, it may help to use a newer version (I expect this may help with debug/error messages - not with the problem). Also, running the client and server with the -d -V options may give more info.