Download speed issue

[SoGooDFR]

Expected Behavior

Get full of my speed connection in dowload or youtube, stream...

Actual Behavior

I can get my full speed connection in a speedtest (80-100mbps) I get 4-6 mbps on youtube. I get 4-6 mbps on twitch.

I get only 4-6mpbs when i downlad file from http://ovh.net/files or http://ping.online.net...

Steps to Reproduce the Problem

1. Fresh install on both
2. Enter the basic configuration
3. See the results.

Specifications

• OpenMPTCProuter version: openmptcprouter v0.27 r7308-ebe1216c7c
• OpenMPTCProuter VPS version: < OpenMPTCProuter VPS 0.34 >
• OpenMPTCProuter platform: (RPI2/RPI3/x86/x86_64) Intel NUC x64 (Celeron J3455)

[SoGooDFR]

[fnoir]

Any news ?

[linuzel]

Hi ! I am facing the same kind of issues with two VDSL (40-45 mbps each) a single download max is capped at 2-3MB/sec.

I tried to adjust a lot of settings but nothing improved. I tried two different VPS and the exact same thing happens.

No load on both sides, I am using the x86_64 generic image on a pcengine apu2. Speedtest.net/nperf results are Ok.

Here is an example directly from the router : root@OpenMPTCProuter:~# curl http://ovh.net/files/1Gio.dat -o /dev/null % Total % Received % Xferd Average Dload Speed Upload Time Total Time Spent Time Current Speed 100 1024M 100 1024M 0 0 2998k 0 0:05:49 0:05:49 --:--:-- 3303k

[Ysurac]

What speed do you get if Shadowsocks and glorytun are disabled ?

[linuzel]

Not much better (but with higher peak speed) : root@OpenMPTCProuter:~# curl http://ovh.net/files/1Gio.dat -o /dev/null % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 1024M 100 1024M 0 0 3239k 0 0:05:23 0:05:23 --:--:-- 3728k

[boundouny]

I update with the 0.29 version, but nothing is better

[linuzel]

The same for me, after the update I still face this issue.

[Ysurac]

It's why the bug is not marked as fixed yet... My 4G connection is not really stable for now, difficult to make real tests.

To test each connection directly on OMR, you can use omr-test-speed wan1 (or wan2, wan3,...), this will bypass ShadowSocks and the VPN. So at least this permit to test if each connection is working properly and is usable.

[linuzel]

So i did a few tests with yout tool (omr-test-speed) but I had to use physical interface instead of wan name but it does not mater because I use 3 distincts physical interfaces I also used the 100Mio file.

eth1/wan1 (vdsl) : % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 100M 100 100M 0 0 4797k 0 0:00:21 0:00:21 --:--:-- 4933k eth2/wan2 (another vdsl) : % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 100M 100 100M 0 0 4637k 0 0:00:22 0:00:22 --:--:-- 4953k

omr (no bypass): % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 100M 100 100M 0 0 1671k 0 0:01:01 0:01:01 --:--:-- 2037k

[Ysurac]

Sorry but each test need to be run for at least 2 minutes to be usable.

[linuzel]

Oh sorry, here it is : root@OpenMPTCProuter:~# omr-test-speed eth1 % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 1024M 100 1024M 0 0 4145k 0 0:04:12 0:04:12 --:--:-- 5252k root@OpenMPTCProuter:~# omr-test-speed eth2 % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 1024M 100 1024M 0 0 4992k 0 0:03:30 0:03:30 --:--:-- 4986k root@OpenMPTCProuter:~# omr-test-speed % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 34 1024M 34 354M 0 0 1484k 0 0:11:46 0:04:04 0:07:42 1453k

[Ysurac]

ok and no error, warning or anything strange in OpenMPTCProuter status page ?

[linuzel]

No, the only warning is "No IPv6 access". I tried to do the speed test again with a "tail -f /var/log/*" on the vps and a logread -f on the router but nothing.

[Ysurac]

Can you give me the latency (available in status page) for each link ? What version of OpenMPTCProuter and VPS are you using ? What speed do you get if you disable ShadowSocks in OpenMPTCProuter (but with glorytun TCP enable) ? What speed do you get with ShadowSocks disabled but with Glorytun UDP ?

Sorry for so many tests, but my connection is very bad for now, and difficult to make tests :(

[linuzel]

Latency is stable at 33ms on wan1 and 39 on wan2. I tried to do a ping test while doing a speed test and there is no large variation (33-40 ms)

I am using openmptcprouter v0.29.1-1 and OpenMPCTProuter VPS 0.40 but it was the same on every version I tried.

It seems to be a little bit faster with ShadowSocks off (and Glorytun TCP as fallback) : % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 41 1024M 41 423M 0 0 2658k 0 0:06:34 0:02:43 0:03:51 2756k

With Glorytun UDP as fallback it is again a little bit faster : % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 56 1024M 56 576M 0 0 3882k 0 0:04:30 0:02:32 0:01:58 5146k

This is a really strange behavior, I feared my router was the issue so I tried in an intel nuc and the result was similar.

[Ysurac]

Interesting results. Can you change method in /etc/shadowsocks-libev/config.json on the VPS to chacha20 and set same method on OpenMPTCProuter Shadowsocks interface ? Then test again (with shadowsocks enabled). If not better, can you disable fast_open on both VPS and OMR, and test ? It's all ideas I have for now ;) Thanks

[linuzel]

It does not seems to change anything, I also tried salsa20 without any luck.

[Ysurac]

You can try to change, in OMR MPTCP interface, the TCP congestion. By default it's olia, you can try cubic.

After that, the only think would be to run sysctl -a net.mptcp.mptcp_debug=1 on OMR, run omr-test-speed and check log about MPTCP.

[linuzel]

I tried a few times changing the congestion control without too much success but this time I did it on the VPS (I don't know why I haven't thought about this before) via : sysctl -w net.ipv4.tcp_congestion_control=cubic

The result is a lot better : % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 1024M 100 1024M 0 0 5891k 0 0:02:57 0:02:57 --:--:-- 5380k

[Ysurac]

OK. A bit better. Can you test BBR TCP Congestion also ? On the VPS modprobe tcp_bbr && sysctl -w net.ipv4.tcp_congestion_control=bbr and BBR on OpenMPTCProuter too.

Seems to be better with my dirty 4G connection, but maybe not for you ?

[linuzel]

Ok, now we're talking ! % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 1024M 100 1024M 0 0 9653k 0 0:01:48 0:01:48 --:--:-- 9.8M

BBR does wonders here (with salsa20 method for shadowsocks).

[Ysurac]

Very good (78Mb/s). Not sure if we can do better with your connection. I will make some tests, put BBR as default (and maybe salsa20), and release a new version. This will permit more people to test :)

Thanks for all your tests and thanks to all others people that help here. Need more tests, but this seems to be the good way to solve this issue.

[SoGooDFR]

I can confirm that. The BBR tcp congestion solve the problem. On shadowsocks i use aes-128-gcm.

[fnoir]

The problem seems solved (BBR + aes-256-cfb). Good job !

Just a quick question, when the update will be released, how do we update the VPS? Use apt-get ?

[Ysurac]

You will only have to run the OpenMPTCProuter VPS script again. This will update all and keep key. New version should be out in the next 24 hours.

[Ysurac]

OpenMPTCProuter v0.30 and OpenMPTCProuter VPS v0.43 are released with BBR set by default.

[Maudissure]

It's worse for me with this version.

On Ookla speedtest and an ubuntu image download, only non master Wan connection is used with low rate.

With torrent download, bandwith used by all Wan ports is very low.

[Ysurac]

Can you give me the result of omr-test-speed and omr-test-speed wan1 etc... for each interface ? Connections types (4G, xDSL, FTTH, IPoAC,...) ? and latency for each WAN (visible on OpenMPTCProuter status page) ?

[Maudissure]

All Wan :

root@OpenMPTCProuter:~# omr-test-speed % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 1 10.0G 1 129M 0 0 2806k 0 1:02:16 0:00:47 1:01:29 3590k

Seems good with big variations (perharps the two LTE connections at this time).

Wan1 (ADSL) :

root@OpenMPTCProuter:~# omr-test-speed wan1 % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 10.0G 0 14.4M 0 0 664k 0 4:22:54 0:00:22 4:22:32 665k

Ok at start (full bandwith) then :

% Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 10.0G 0 60.1M 0 0 572k 0 5:05:20 0:01:47 5:03:33 0

Completely crash

Wan 2 (LTE) :

**root@OpenMPTCProuter:~# omr-test-speed wan2 % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 1 10.0G 1 189M 0 0 882k 0 3:18:04 0:03:39 3:14:25 1722k

2 10.0G 2 239M 0 0 846k 0 3:26:30 0:04:49 3:21:41 672k**

Big variation in download rates

Wan 3 (LTE):

root@OpenMPTCProuter:~# omr-test-speed wan3 % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 10.0G 0 39.0M 0 0 925k 0 3:08:53 0:00:43 3:08:10 1741k

Seems better, more stable connection

Didn't find latency for each Wan on status page ...

Usualy i get approximatively 20Mbps for each LTE connection, max for my ADSL is 5.5Mbps.

While an Ubuntu image download, it's not quite good :

wan2 | Actuellement : | 6.17 Mbit/s(789.63 kB/s) | Moyenne : | 30.27 kbit/s(3.78 kB/s) | Pointe : | 8.58 Mbit/s(1.07 MB/s) wan1 | Actuellement : | 5.17 Mbit/s(662.02 kB/s) | Moyenne : | 25.36 kbit/s(3.17 kB/s) | Pointe : | 10.3 Mbit/s(1.29 MB/s) wan3 | Actuellement : | 8.53 Mbit/s(1.07 MB/s) | Moyenne : | 41.76 kbit/s(5.22 kB/s) | Pointe :

And eventually the router crash.

[Ysurac]

If I ignore the crash (and use only real speed), aggregation seems quite good. Can you make a screen capture of OpenMPTCProuter status page ? (remove VPS ip before posting)

What are the result if you set back olia in MPTCP interface and sysctl -w net.ipv4.tcp_congestion_control=olia on the VPS ?

[Maudissure]

Hum, after crashing and RPI3B+ hard reboot, no more access to the router interface and no internet connection. The RPI3B+ booting is ok but the system seems completely down.

[Ysurac]

Are you sure you don't have power problem ? If you use USB LTE on it they can use too much power for RPI.

[Maudissure]

Nope, i have official RPI kit with 2.5A power block and my LTE devices are both routers linked to the home switch. All network home install is wired.

Fresh install in progress ...

[Maudissure]

So, the status page cannot load.

[Ysurac]

Can you give me the result of http://192.168.100.1/cgi-bin/luci/admin/system/openmptcprouter/interfaces_status (and remove your wan/vps ip if present)

[Maudissure]

There :

Failed to execute call dispatcher target for entry '/admin/system/openmptcprouter/interfaces_status'. The called action terminated with an exception: /usr/lib/lua/luci/controller/openmptcprouter.lua:676: attempt to concatenate field 'RecursiveDnsServer' (a table value) stack traceback: /usr/lib/lua/luci/controller/openmptcprouter.lua:676: in function 'callback' /usr/lib/lua/luci/model/uci.lua:295: in function 'foreach' /usr/lib/lua/luci/controller/openmptcprouter.lua:571: in function </usr/lib/lua/luci/controller/openmptcprouter.lua:432> (tail call): ?

[Ysurac]

Seems you have other IPv6 announced, but this should not crash here :)

[Maudissure]

Ok i just disabled IPV6 in Advanced configuration of openMPTCProuter page, and it works ! All ok but the message "Shadowsocks not working !"

And the pings : ADSL : 14ms LTE1 : 39ms LTE2 : 38ms

[Ysurac]

Check keys, and that chacha20 is used at both end.

[Maudissure]

Keys and Chacha20 ok on router, how do i verify on VPS ?

the file openmptcprouter_config.txt on VPS contain : Shadowsocks port: 65101 Shadowsocks encryption: aes-256-cfb Your shadowsocks key:

Glorytun port: 65001 Glorytun encryption: chacha20

[Ysurac]

in /etc/shadowsocks/config.json

[Maudissure]

hum ! directory /etc/shadowsocks/config.json does not exist ...

It's seem that shadowsock was not installed by the script.

i retry an install of the vps with the script...

... well, still no /etc/shadowsocks/config.json BUT i have a /etc/shadowsocks-libev/config.json

{ "server":["[::0]", "0.0.0.0"], "server_port":65101, "local_port":1081, "mode":"tcp_and_udp", "key":" "timeout":400, "method":"chacha20", "verbose":0, "prefer_ipv6": false, "fast_open": true, "reuse_port": true, "mptcp": true }

Do i need the same method on the VPS and on the router ? i suppose so ...

OK after a VPS reboot, all good now .. :D

But download of the ubuntu image still very slow :( . Anyway, thanks for all.

[boundouny]

With the new version, i have downloading full power . Thanks for all.

Best regards