[error] broken pipe when connecting iperf3 server

[ClSlaid]

I've been trying to test QUIC-IETF performance with your project and iperf3. I built your tunnel from source code(current master branch), and then ran the gotun client and server on different hosts, to proxy TCP connections between 2 iperf3 programs. However, the server-side then encounters a broken pipe error when writing to TCP.

Network:

# client iperf3
iperf3 -c 127.0.0.1 --port 7788

--->

# client go-tunnel
sudo ./bin/gotun -d socks5_client_adapted.yaml
# config file: connect to go-tunnel server directly

-- local ISP network -->

# server go-tunnel
sudo ./bin/gotun -d socks5_server_adapted.yaml
# config file: connect to 127.0.0.1:5201 (iperf port)

--->

# server iperf3
iperf3 -s

server iperf3 logs: server go-tunnel logs:

./gotun: Can't enable log rotation: logger is not file backed
config: 1 listeners
listen on :8877 quic with tls using cert certs/srv.crt, key certs/srv.key
        connect to 127.0.0.1:5201
<2>:2022/05/16 08:43:51.696942 [gotun] gotun - v0.7.3 [git:v0.7.3-0-gffa945b7f6ec-dirty - built on 2022-05-16T07:48.09Z] starting up (logging at DEBUG)...
<2>:2022/05/16 08:43:51.698078 [gotun-:8877] Starting Quic server ..
<2>:2022/05/16 08:43:51.698134 [gotun-:8877] Ratelimit: Global 20000000 req/s, Per-host: 10 req/s
<1>:2022/05/16 08:44:18.703182 [gotun-:8877] (tcpdial.go:48) 127.0.0.1:34838 connected to 127.0.0.1:5201
<1>:2022/05/16 08:44:18.703249 [gotun-:8877] (server.go:473) LHS 10.128.233.163:54811.0x0-10.128.227.23:8877.0x0, RHS 127.0.0.1:34838-127.0.0.1:5201
<1>:2022/05/16 08:44:18.712512 [gotun-:8877] (tcpdial.go:48) 127.0.0.1:34840 connected to 127.0.0.1:5201
<1>:2022/05/16 08:44:18.712565 [gotun-:8877] (server.go:473) LHS 10.128.233.163:54811.0x4-10.128.227.23:8877.0x4, RHS 127.0.0.1:34840-127.0.0.1:5201
<1>:2022/05/16 08:44:20.708738 [gotun-:8877-10.128.227.23:8877.0x0-127.0.0.1:34838-127.0.0.1:5201] (server.go:692) 10.128.227.23:8877.0x0: nr 0, read err deadline exceeded
<1>:2022/05/16 08:44:20.713126 [gotun-:8877-10.128.227.23:8877.0x4-127.0.0.1:34840-127.0.0.1:5201] (server.go:692) 127.0.0.1:34840: nr 0, read err read tcp 127.0.0.1:34840->127.0.0.1:5201: i/o timeout
<1>:2022/05/16 08:44:20.714573 [gotun-:8877-10.128.227.23:8877.0x0-127.0.0.1:34838-127.0.0.1:5201] (server.go:692) 127.0.0.1:34838: nr 0, read err read tcp 127.0.0.1:34838->127.0.0.1:5201: i/o timeout
<2>:2022/05/16 08:44:20.714661 [gotun-:8877-10.128.227.23:8877.0x0-127.0.0.1:34838-127.0.0.1:5201] 10.128.233.163:54811.0x0-10.128.227.23:8877.0x0: rd 142, wr 4; 127.0.0.1:34838-127.0.0.1:5201: rd 4, wr 142
<1>:2022/05/16 08:44:20.715817 [gotun-:8877-10.128.227.23:8877.0x4-127.0.0.1:34840-127.0.0.1:5201] (server.go:713) 127.0.0.1:5201: Write Err write tcp 127.0.0.1:34840->127.0.0.1:5201: write: broken pipe
<2>:2022/05/16 08:44:20.715898 [gotun-:8877-10.128.227.23:8877.0x4-127.0.0.1:34840-127.0.0.1:5201] 10.128.233.163:54811.0x4-10.128.227.23:8877.0x4: rd 16167005, wr 0; 127.0.0.1:34840-127.0.0.1:5201: rd 0, wr 16161365

[ClSlaid]

I got it, just set the read timeout will solve the problem, but why...?

[opencoff]

The logs show broken pipe when the server is connected to downstream iperf instance.

<1>:2022/05/16 08:44:20.715817 [gotun-:8877-10.128.227.23:8877.0x4-127.0.0.1:34840-127.0.0.1:5201] (server.go:713) 127.0.0.1:5201: Write Err write tcp 127.0.0.1:34840->127.0.0.1:5201: write: broken pipe

EPIPE (broken pipe) is seen when a TCP endpoint tries to write data and the connection is closed for writing; it's possible that iperf closed the connection?

[mmoya]

FWIW. I doubt it's iperf server closing the connectiong.

I have a trivial setup for testing ([hostA: iperf client] <-- gotun --> [hostB: iperf server]), gotun with minimal configuration (ie: no timeouts, just endpoints). The iperf server is not restarted between tests.

I replace gotun with stunnel and iperf client is able to finish the benchmark. I go back to gotun and the problem manifest again. It's always reproducible.

Using gotun - v0.7.3 [git:v0.7.3-0-gffa945b7f6ec-dirty; 2022-09-22T23:54.02Z]