UDPspeeder work with openvpn has mtu issue when I use mode 0

[lyt0112]

Hi I have two linux box , i can only ping each other with small ping packets...could u give me some suggestions?

One is client and the other one is server:

[root@]# speeder --help UDPspeeder V2 git version: 7d349251d0 build date: Jul 14 2020 01:34:20 repository: https://github.com/wangyu-/UDPspeeder

Shell script on the Client: [root@]# cat speeder.sh

set -x

killall -9 speeder kill -9 cat /var/run/openvpn/tun1.pid rm -f /var/run/openvpn/tun1.pid

/sbin/openvpn --dev-type tun --dev tun1 --local 100.100.100.1 --remote 127.0.0.1 --ifconfig 1.2.0.2 1.1.0.2 --port 4003 --rcvbuf 2000000 --sndbuf 2000000 --fragment 1200 --mssfix 1200 --txqueuelen 4000 --keepalive 10 60 --persist-tun --daemon --writepid /var/run/openvpn/tun1.pid &

/sbin/iptables -D INPUT -p udp --sport 14003 -j ACCEPT /sbin/iptables -I INPUT -p udp --sport 14003 -j ACCEPT

speeder -c -l 127.0.0.1:4003 -r 200.200.200.1:14003 -f20:10 --mode 0 --mtu 1200 log-level 5 --log-position --report 5 &

Shell script on the Server: [root@]# cat speeder.sh

set -x

killall -9 speeder kill -9 cat /var/run/openvpn/tun1.pid rm -f /var/run/openvpn/tun1.pid

/sbin/openvpn --dev-type tun --dev tun1 --local 0.0.0.0 --ifconfig 1.1.0.2 1.2.0.2 --port 4003 --rcvbuf 2000000 --sndbuf 2000000 --txqueuelen 4000 --keepalive 10 60 --fragment 1200 --mssfix 1200 --persist-tun --daemon --writepid /var/run/openvpn/tun1.pid &

/sbin/iptables -D INPUT -p udp --dport 14003 -j ACCEPT /sbin/iptables -I INPUT -p udp --dport 14003 -j ACCEPT

speeder -s -l 0.0.0.0:14003 -r 127.0.0.1:4003 -f20:10 --mode 0 --mtu 1200 log-level 5 --log-position --report 5 &

After execute the shell scripts, ping can work between 1.2.0.2 and 1.1.0.2 with small packet size ,but ssh to 1.1.0.2 can not work.. But : [root@]# ping 1.1.0.2 PING 1.1.0.2 (1.1.0.2) 56(84) bytes of data. 64 bytes from 1.1.0.2: icmp_seq=1 ttl=64 time=49.0 ms 64 bytes from 1.1.0.2: icmp_seq=2 ttl=64 time=48.0 ms 64 bytes from 1.1.0.2: icmp_seq=3 ttl=64 time=49.6 ms 64 bytes from 1.1.0.2: icmp_seq=4 ttl=64 time=48.8 ms 64 bytes from 1.1.0.2: icmp_seq=5 ttl=64 time=48.8 ms ^C --- 1.1.0.2 ping statistics --- 5 packets transmitted, 5 received, 0% packet loss, time 4004ms rtt min/avg/max/mdev = 48.058/48.863/49.607/0.552 ms [root@]# ping -s 400 1.1.0.2 PING 1.1.0.2 (1.1.0.2) 400(428) bytes of data. ^C --- 1.1.0.2 ping statistics --- 7 packets transmitted, 0 received, 100% packet loss, time 6001ms

[root@]# ping -s 300 1.1.0.2 PING 1.1.0.2 (1.1.0.2) 300(328) bytes of data. 308 bytes from 1.1.0.2: icmp_seq=1 ttl=64 time=49.3 ms 308 bytes from 1.1.0.2: icmp_seq=2 ttl=64 time=50.7 ms 308 bytes from 1.1.0.2: icmp_seq=3 ttl=64 time=50.6 ms 308 bytes from 1.1.0.2: icmp_seq=4 ttl=64 time=49.2 ms ^C --- 1.1.0.2 ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3001ms rtt min/avg/max/mdev = 49.217/50.000/50.748/0.753 ms

Log on the Client: [2020-08-26 17:29:30][INFO]argc=16 speeder -c -l 127.0.0.1:4003 -r 100.100.100.1:14003 -f20:10 --mode 0 --mtu 1200 log-level 5 --log-position --report 5 [2020-08-26 17:29:30][INFO]parsing address: 127.0.0.1:4003 [2020-08-26 17:29:30][INFO]its an ipv4 adress [2020-08-26 17:29:30][INFO]ip_address is {127.0.0.1}, port is {4003} [2020-08-26 17:29:30][INFO]parsing address: 100.100.100.1:14003 [2020-08-26 17:29:30][INFO]its an ipv4 adress [2020-08-26 17:29:30][INFO]ip_address is {100.100.100.1}, port is {14003} [2020-08-26 17:29:30][INFO][misc.cpp,func:print_parameter,line:265]jitter_min=0 jitter_max=0 output_interval_min=0 output_interval_max=0 fec_timeout=8 fec_mtu=1200 fec_queue_len=200 fec_mode=0 [2020-08-26 17:29:30][INFO][misc.cpp,func:print_parameter,line:266]fec_str=20:10 [2020-08-26 17:29:30][INFO][misc.cpp,func:print_parameter,line:267]fec_inner_parameter=1:10,2:10,3:10,4:10,5:10,6:10,7:10,8:10,9:10,10:10,11:10,12:10,13:10,14:10,15:10,16:10,17:10,18:10,19:10,20:10 [2020-08-26 17:29:30][INFO][tunnel_client.cpp,func:tunnel_client_event_loop,line:384]now listening at 127.0.0.1:4003 [2020-08-26 17:29:30][INFO][connection.h,func:report_as_client,line:235][report]client-->server:(original:0 pkt;0 byte) (fec:0 pkt,0 byte) server-->client:(original:0 pkt;0 byte) (fec:0 pkt;0 byte) [2020-08-26 17:29:35][INFO][connection.h,func:report_as_client,line:235][report]client-->server:(original:0 pkt;0 byte) (fec:0 pkt,0 byte) server-->client:(original:0 pkt;0 byte) (fec:0 pkt;0 byte) [2020-08-26 17:29:40][INFO][tunnel_client.cpp,func:data_from_local_or_fec_timeout,line:74]new packet from 200.200.200.1:4003,conv_id=c8fb84d9 [2020-08-26 17:29:41][INFO][connection.h,func:report_as_client,line:235][report]client-->server:(original:2 pkt;49 byte) (fec:14 pkt,322 byte) server-->client:(original:1 pkt;117 byte) (fec:7 pkt;231 byte) [2020-08-26 17:29:46][INFO][connection.h,func:report_as_client,line:235][report]client-->server:(original:2 pkt;49 byte) (fec:14 pkt,322 byte) server-->client:(original:2 pkt;142 byte) (fec:14 pkt;364 byte) [2020-08-26 17:29:51][INFO][connection.h,func:report_as_client,line:235][report]client-->server:(original:3 pkt;73 byte) (fec:27 pkt,556 byte) server-->client:(original:2 pkt;142 byte) (fec:14 pkt;364 byte) [2020-08-26 17:29:56][INFO][connection.h,func:report_as_client,line:235][report]client-->server:(original:3 pkt;73 byte) (fec:27 pkt,556 byte) server-->client:(original:4 pkt;191 byte) (fec:21 pkt;525 byte) [2020-08-26 17:30:01][INFO][connection.h,func:report_as_client,line:235][report]client-->server:(original:5 pkt;189 byte) (fec:55 pkt,1210 byte) server-->client:(original:9 pkt;651 byte) (fec:59 pkt;1589 byte) [2020-08-26 17:30:07][INFO][connection.h,func:report_as_client,line:235][report]client-->server:(original:6 pkt;281 byte) (fec:70 pkt,1630 byte) server-->client:(original:10 pkt;743 byte) (fec:67 pkt;1813 byte)

Log on the Server: [2020-08-26 17:29:25][INFO]argc=16 speeder -s -l 0.0.0.0:14003 -r 127.0.0.1:4003 -f20:10 --mode 0 --mtu 1200 log-level 5 --log-position --report 5 [2020-08-26 17:29:25][INFO]parsing address: 0.0.0.0:14003 [2020-08-26 17:29:25][INFO]its an ipv4 adress [2020-08-26 17:29:25][INFO]ip_address is {0.0.0.0}, port is {14003} [2020-08-26 17:29:25][INFO]parsing address: 127.0.0.1:4003 [2020-08-26 17:29:25][INFO]its an ipv4 adress [2020-08-26 17:29:25][INFO]ip_address is {127.0.0.1}, port is {4003} [2020-08-26 17:29:25][INFO][misc.cpp,func:print_parameter,line:265]jitter_min=0 jitter_max=0 output_interval_min=0 output_interval_max=0 fec_timeout=8 fec_mtu=1200 fec_queue_len=200 fec_mode=0 [2020-08-26 17:29:25][INFO][misc.cpp,func:print_parameter,line:266]fec_str=20:10 [2020-08-26 17:29:25][INFO][misc.cpp,func:print_parameter,line:267]fec_inner_parameter=1:10,2:10,3:10,4:10,5:10,6:10,7:10,8:10,9:10,10:10,11:10,12:10,13:10,14:10,15:10,16:10,17:10,18:10,19:10,20:10 [2020-08-26 17:29:25][INFO][tunnel_server.cpp,func:tunnel_server_event_loop,line:391]now listening at 0.0.0.0:14003 [2020-08-26 17:29:41][INFO][tunnel_server.cpp,func:local_listen_cb,line:204]new connection from 200.200.200.1:36277 [2020-08-26 17:29:41][INFO][connection.h,func:report_as_server,line:249][report][200.200.200.1:36277]client-->server:(original:0 pkt;0 byte) (fec:1 pkt;23 byte) server-->client:(original:0 pkt;0 byte) (fec:0 pkt;0 byte) [2020-08-26 17:29:41][INFO][tunnel_server.cpp,func:local_listen_cb,line:259][200.200.200.1:36277]new conv c8fb84d9,fd 6 created,fd64=4294967395 [2020-08-26 17:29:46][INFO][connection.h,func:report_as_server,line:249][report][200.200.200.1:36277]client-->server:(original:2 pkt;49 byte) (fec:14 pkt;322 byte) server-->client:(original:1 pkt;117 byte) (fec:15 pkt;495 byte) [2020-08-26 17:29:52][INFO][connection.h,func:report_as_server,line:249][report][200.200.200.1:36277]client-->server:(original:3 pkt;73 byte) (fec:27 pkt;556 byte) server-->client:(original:2 pkt;142 byte) (fec:28 pkt;742 byte) [2020-08-26 17:29:57][INFO][connection.h,func:report_as_server,line:249][report][200.200.200.1:36277]client-->server:(original:3 pkt;73 byte) (fec:27 pkt;556 byte) server-->client:(original:2 pkt;142 byte) (fec:28 pkt;742 byte) [2020-08-26 17:30:02][INFO][connection.h,func:report_as_server,line:249][report][200.200.200.1:36277]client-->server:(original:5 pkt;189 byte) (fec:55 pkt;1210 byte) server-->client:(original:9 pkt;651 byte) (fec:117 pkt;3164 byte) [2020-08-26 17:30:07][INFO][connection.h,func:report_as_server,line:249][report][200.200.200.1:36277]client-->server:(original:6 pkt;281 byte) (fec:70 pkt;1630 byte) server-->client:(original:10 pkt;743 byte) (fec:132 pkt;3584 byte) [2020-08-26 17:30:12][INFO][connection.h,func:report_as_server,line:249][report][200.200.200.1:36277]client-->server:(original:7 pkt;398 byte) (fec:85 pkt;2125 byte) server-->client:(original:11 pkt;768 byte) (fec:145 pkt;3831 byte) [2020-08-26 17:30:18][INFO][connection.h,func:report_as_server,line:249][report][200.200.200.1:36277]client-->server:(original:7 pkt;398 byte) (fec:85 pkt;2125 byte) server-->client:(original:11 pkt;768 byte) (fec:145 pkt;3831 byte) [2020-08-26 17:30:23][INFO][connection.h,func:report_as_server,line:249][report][200.200.200.1:36277]client-->server:(original:8 pkt;422 byte) (fec:98 pkt;2359 byte) server-->client:(original:12 pkt;792 byte) (fec:158 pkt;4065 byte) [2020-08-26 17:30:28][INFO][connection.h,func:report_as_server,line:249][report][200.200.200.1:36277]client-->server:(original:8 pkt;422 byte) (fec:98 pkt;2359 byte) server-->client:(original:12 pkt;792 byte) (fec:158 pkt;4065 byte) [2020-08-26 17:30:33][INFO][connection.h,func:report_as_server,line:249][report][200.200.200.1:36277]client-->server:(original:9 pkt;446 byte) (fec:111 pkt;2593 byte) server-->client:(original:13 pkt;816 byte) (fec:171 pkt;4299 byte)

[wangyu-]

I have tested your udpspeeder parameters with UDPing, no problem:

wangyu@wangyu-VirtualBox:~/UDPping$ ./udpping.py 127.0.0.1 4003
UDPping 127.0.0.1 via port 4003 with 64 bytes of payload
Reply from 127.0.0.1 seq=0 time=207.11 ms
Reply from 127.0.0.1 seq=1 time=202.13 ms
^C
--- ping statistics ---
2 packets transmitted, 2 received, 0.00% packet loss
rtt min/avg/max = 202.13/204.62/207.11 ms
wangyu@wangyu-VirtualBox:~/UDPping$ ./udpping.py 127.0.0.1 4003 "LEN=300"
UDPping 127.0.0.1 via port 4003 with 300 bytes of payload
Reply from 127.0.0.1 seq=0 time=204.90 ms
Reply from 127.0.0.1 seq=1 time=203.49 ms
Reply from 127.0.0.1 seq=2 time=205.85 ms
^C
--- ping statistics ---
3 packets transmitted, 3 received, 0.00% packet loss
rtt min/avg/max = 203.49/204.75/205.85 ms
wangyu@wangyu-VirtualBox:~/UDPping$ ./udpping.py 127.0.0.1 4003 "LEN=1500"
UDPping 127.0.0.1 via port 4003 with 1500 bytes of payload
Reply from 127.0.0.1 seq=0 time=211.60 ms
Reply from 127.0.0.1 seq=1 time=216.91 ms
Reply from 127.0.0.1 seq=2 time=212.90 ms

Why it doesn't work on your environment with openvpn? I don't really have an idea. Maybe your network doesn't allow large udp packages to pass?

To debug the problem, maybe you can try

1. does openvpn work without udpspeeder?
2. does udpspeeder work without openvpn? (with UDPping)

[wangyu-]

Maybe your network doesn't allow large udp packages to pass?

Or maybe your server is configured to not respond large packets?

[lyt0112]

Thanks for your quick reply~ 1.Openvpn can work fine without udpspeeder 2.Where can I find the document to test udpspeeder work without openvpn? I can only find udp2raw_amd64+udpspeeder+openvpn , I tested it but same problem. 3.My linux box's OS is Fedora 21 which is used to route my LAN traffic to internet. I use it for many years,all kind of packets can be passthrough. 4.I'll try test if any kernel parameters may effect it..If I find something,I will post here.

[wangyu-]

2.Where can I find the document to test udpspeeder work without openvpn?

there is not a step-to-step document, but you can test it easily with netcat or the UDPping I mentioned.

[wangyu-]

looks like your parameter of openvpn on client side is wrong. It doesn't even run on my side.

/sbin/openvpn --dev-type tun --dev tun1 --local 100.100.100.1 --remote 127.0.0.1 --ifconfig 1.2.0.2 1.1.0.2 --port 4003 --rcvbuf 2000000 --sndbuf 2000000 --fragment 1200 --mssfix 1200 --txqueuelen 4000 --keepalive 10 60 --persist-tun

should be

/sbin/openvpn --dev-type tun --dev tun1 --local 100.100.100.1 --remote 127.0.0.1 4003 --ifconfig 1.2.0.2 1.1.0.2 --rcvbuf 2000000 --sndbuf 2000000 --fragment 1200 --mssfix 1200 --txqueuelen 4000 --keepalive 10 60 --persist-tun

---

my full configuration: client:

./speederv2_amd64 -c -l 127.0.0.1:4003 -r 45.66.77.88:14003 -f20:20 --mode 0 --mtu 1200 log-level 5 --log-position
openvpn --dev-type tun --dev tun1 --local 0.0.0.0 --remote 127.0.0.1 4003 --ifconfig 1.2.0.2 1.1.0.2 --rcvbuf 2000000 --sndbuf 2000000 --fragment 1200 --mssfix 1200 --txqueuelen 4000 --keepalive 10 60 --persist-tun

server:

openvpn --dev-type tun --dev tun1 --local 0.0.0.0 --ifconfig 1.1.0.2 1.2.0.2 --port 4003 --rcvbuf 2000000 --sndbuf 2000000 --txqueuelen 4000 --keepalive 10 60 --fragment 1200 --mssfix 1200 --persist-tun
./speederv2 -s -l 0.0.0.0:14003 -r 127.0.0.1:4003 -f20:20 --mode 0 --mtu 1200 log-level 5 --log-position --report 5

with this configuration, no problem at all on my side:

wangyu@wangyu-VirtualBox:~$ ping 1.1.0.2 -s 1400
PING 1.1.0.2 (1.1.0.2) 1400(1428) bytes of data.
1408 bytes from 1.1.0.2: icmp_seq=1 ttl=64 time=220 ms
1408 bytes from 1.1.0.2: icmp_seq=2 ttl=64 time=285 ms
1408 bytes from 1.1.0.2: icmp_seq=3 ttl=64 time=219 ms
^C
--- 1.1.0.2 ping statistics ---
4 packets transmitted, 3 received, 25% packet loss, time 3005ms
rtt min/avg/max/mdev = 219.714/241.879/285.425/30.793 ms
wangyu@wangyu-VirtualBox:~$ ping 1.1.0.2 -s 9999
PING 1.1.0.2 (1.1.0.2) 9999(10027) bytes of data.
10007 bytes from 1.1.0.2: icmp_seq=1 ttl=64 time=245 ms
10007 bytes from 1.1.0.2: icmp_seq=2 ttl=64 time=269 ms
10007 bytes from 1.1.0.2: icmp_seq=3 ttl=64 time=290 ms
10007 bytes from 1.1.0.2: icmp_seq=4 ttl=64 time=416 ms

wangyu@wangyu-VirtualBox:~$ ssh root@1.1.0.2
root@1.1.0.2's password: 

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Thu Aug 27 02:31:48 2020 from 1.2.0.2
[root@45.66.77.88:~]
$ 

---

also, as a friendly reminder --mssfix and --fragment (and even the --mtu option for udpspeeder) is not necessary for --mode 0. for more than 95% scene, when you are using udpspeeder mode 0 you don't need to worry about mtu at all (both udpspeeder itself, or up level application such as openvpn)

[lyt0112]

Thanks for your lab test... I've tested your openvpn commands .. I found the problem caused by -f20:10 if I use -f20:10 ,it has mtu issue.. if I change to -f20:20 ,it can work .. I don't understand the reason...

[wangyu-]

I found the problem caused by -f20:10 if I use -f20:10 ,it has mtu issue.. if I change to -f20:20 ,it can work ..

It doesn't make a difference on my side, I have tested both -f20:10 and -f20:20.

[lyt0112]

I found a rule,FEC extra packets need to equal or more than the original packets ...

-f10:8 --mode 0 ping -s 1500 -c 100 1.1.0.2 --- 1.1.0.2 ping statistics --- 100 packets transmitted, 0 received, 100% packet loss, time 99000ms

ping -s 1500 -c 100 1.2.0.2 --- 1.2.0.2 ping statistics --- 100 packets transmitted, 0 received, 100% packet loss, time 99054ms

-f10:9 --mode 0 --- 1.1.0.2 ping statistics --- 100 packets transmitted, 24 received, 76% packet loss, time 99034ms rtt min/avg/max/mdev = 32.784/39.483/42.117/2.094 ms

--- 1.2.0.2 ping statistics --- 100 packets transmitted, 75 received, 25% packet loss, time 99108ms rtt min/avg/max/mdev = 33.594/38.918/41.555/1.500 ms

-f10:10 --mode 0 --- 1.1.0.2 ping statistics --- 100 packets transmitted, 100 received, 0% packet loss, time 99134ms rtt min/avg/max/mdev = 52.140/53.670/55.555/0.695 ms

--- 1.2.0.2 ping statistics --- 100 packets transmitted, 100 received, 0% packet loss, time 99140ms rtt min/avg/max/mdev = 51.712/53.311/93.025/4.036 ms

-f10:11 --mode 0 --- 1.1.0.2 ping statistics --- 100 packets transmitted, 100 received, 0% packet loss, time 99140ms rtt min/avg/max/mdev = 53.526/55.048/65.208/1.261 ms

--- 1.2.0.2 ping statistics --- 100 packets transmitted, 100 received, 0% packet loss, time 99114ms rtt min/avg/max/mdev = 52.750/54.244/58.277/0.738 ms

-f10:20 --mode 0 --- 1.1.0.2 ping statistics --- 100 packets transmitted, 100 received, 0% packet loss, time 99140ms rtt min/avg/max/mdev = 39.129/40.295/44.068/0.872 ms

--- 1.2.0.2 ping statistics --- 100 packets transmitted, 100 received, 0% packet loss, time 99134ms rtt min/avg/max/mdev = 38.470/39.448/41.829/0.552 ms

[wangyu-]

Currently I have no idea.

Maybe you can try to test with the lastest version of UDPspeeder with two clean linux machines, with iptables cleared, without ip route rules, and without other network service. And see if it works without problem. If so, then make the difference of the two environment smaller and smaller, and try to figure out what cause the problem.

the below article might help you fully clear iptables:

https://github.com/wangyu-/tinyfecVPN/wiki/how-to-fully-clear-iptables

[lyt0112]

Never mind,it is enough for me to use... I appreciate your help very much.

[wangyu-]

I have come up with something.

How large is your packet loss rate of you link? Are you using UDPspeeder on a link with very high packet loss rate? Or are you simulating packet loss by some software?

Note that all packets in mode 0 are equal, the side effect is, when -fx:y is used, if only less than x packets of the x+y packets are not lost, then all you oringinal packets in this current FEC group will be lost. So, as a result, if not correctly set the -fx:y parameter, on a link with super high loss rate UDPspeeder can actually enlarge your packet loss. Also there is an implementation detail may make the packet loss of larger packet higher.

You can find something related in this article, although this article is not focus on explain the detail I mentioned:

https://github.com/wangyu-/UDPspeeder/wiki/Fine-grained-FEC-Parameters

About how to set -fx:y according to your network's packet loss correctly: https://github.com/wangyu-/UDPspeeder/wiki/FEC%E4%B8%A2%E5%8C%85%E7%8E%87%E8%AE%A1%E7%AE%97

[lyt0112]

My Link's download speed: 100 Mbps , Upload speed: 40Mbps Most of time,I have no packet loss... Sometimes have 2-3% packet loss...

I want to remove the 2-3% packet loss,once it happen.

[wangyu-]

okay, nevermind

[liuxyon]

我有个问题,电信晚高峰丢包严重, 晚上平均大约30%丢包率, 白天10%左右, 我需要不停更改参数么? 还是会自动调节?