aggregation starlink/2x5G Tmobile doesnt work v6.0

[jle1511]

Expect 3 WANS aggregate together.

Current Behavior:

Instead, only 1 WAN is working not the other 2. We have tested: Single Wan Starlink, it has no problem going through VPS with full speed (OMR-test-speed and omr-iperf vps -R) As soon we introduced Starlink WAN, we experienced 1 WAN full-speed behavior only. and other 2 WANS at no speed

Possible Solution

Steps to Reproduce the Problem

We have tested: 1 WAN only, speed is full speed, no delay in latency, regardless of its starlink/5G and either port. 2 WANs, (2x 5G TMOBILE) full speed at aggregation, either eth port (eth1,2,3, or USB3.0 port), both WAN run at max speed 3 WANs (Starlink, 2x 5G Tmobile), only 1 will run at full speed which is the master interface, other 2 WANS hop around 0-5kbps 2 WANS (Starlink and 1x 5G Tmobile), 1 will run at full speed, the master interface, other WAN hops around 0-5kbps

Context (Environment)

Nano Pi R5S Running latest version v6.0 VPS: BuyVM Latency on 3 WANS 40-50MS

Specifications

• OpenMPTCProuter version: [0.60-6.1 r0+24843-acf40c022e]
• OpenMPTCProuter VPS version: 0.1030 6.1.0-9-amd64
• OpenMPTCProuter VPS provider: buyvm
• OpenMPTCProuter platform: Nano Pi R5S with 2 WANS, and 1 WAN USB 3.0
• Country:

[Ysurac]

I would need omr-test-speed results and a screenshot of status page.

[jle1511]

omr-test-speed eth1
root@OpenMPTCProuter:~# omr-test-speed
Best server is http://mia.download.datapacket.com/10000mb.bin, running test:
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  6 9536M    6  649M    0     0  21.4M      0  0:07:24  0:00:30  0:06:54 22.5M

omr-test-speed eth2
  Best server is http://mia.download.datapacket.com/10000mb.bin, running test:
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  5 9536M    5  519M    0     0  17.0M      0  0:09:17  0:00:30  0:08:47 17.1M

omr-test-speed eth3
  Best server is http://atl.download.datapacket.com/10000mb.bin, running test:
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  6 9536M    6  584M    0     0  19.0M      0  0:08:20  0:00:30  0:07:50 26.7M

omr-test-speed
Best server is http://mia.download.datapacket.com/10000mb.bin, running test:
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  6 9536M    6  645M    0     0  21.9M      0  0:07:14  0:00:29  0:06:45 22.0M

As this running, i screenshot the realtime traffic, and only Wan1 (master) is running at full speed while omr-test-speed running:

the Wan2 and 3 is at 0.xx kbits/s where as Wan1 is full speed.

root@OpenMPTCProuter:~# omr-iperf vps -R
Connecting to host 45.61.188.181, port 65400
Reverse mode, remote host 45.61.188.181 is sending
[  5] local 192.168.5.42 port 36195 connected to 45.61.188.181 port 65400
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  18.7 MBytes   156 Mbits/sec                  
[  5]   1.00-2.00   sec  23.8 MBytes   200 Mbits/sec                  
[  5]   2.00-3.00   sec  24.3 MBytes   204 Mbits/sec                  
[  5]   3.00-4.00   sec  24.2 MBytes   203 Mbits/sec                  
[  5]   4.00-5.00   sec  24.0 MBytes   201 Mbits/sec                  
[  5]   5.00-6.00   sec  21.2 MBytes   178 Mbits/sec                  
[  5]   6.00-7.00   sec  22.8 MBytes   191 Mbits/sec                  
[  5]   7.00-8.00   sec  25.4 MBytes   213 Mbits/sec                  
[  5]   8.00-9.00   sec  24.1 MBytes   203 Mbits/sec                  
[  5]   9.00-10.00  sec  21.2 MBytes   178 Mbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.03  sec   240 MBytes   201 Mbits/sec    0             sender
[  5]   0.00-10.00  sec   230 MBytes   193 Mbits/sec                  receiver

Same thing with testing to VPS, only Master is at full speed:

[jle1511]

Ok, i found a problem. so you must reboot after making any changes, for example: changing anything in wizard, or interface etc..it doesnt work and only 1 WAN will run. I tested: Added 2nd interface turn on (only wan 1 run max speed) After reboot, Wan 1 + 2 are working aggregated Add Wan 3, will not aggregate with WAN1 + WAN2 (hence seem like adding anything will fall back to Wan1 max speed) After reboot, all 3 WANS are aggregated

Best server is http://mia.download.datapacket.com/10000mb.bin, running test:
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
 21 9536M   21 2033M    0     0  39.4M      0  0:04:01  0:00:51  0:03:10 40.8M

[Ysurac]

Thanks for the feedback. I will check why a reboot is needed.

[jle1511]

Hi,I just find out, out of 3 wans, occasionally the starlink is lagging the speedn vs other 2, for example, when i do omr test speed, 2x 4G up to 200Mbps, but starlink only around 1Kbps, and when i do omr test speed start link interface right after, it goes up to 300s Mbps. The nanopi R5s surely cable of over 500Mbps aggregated. Each lan is 2.5G and WAN port is 1G i think. I have bcp3 as option and shadowsocks rust 2022.I also notice omr by pass only work under shadowsocks and not any other proxy (even after the reboot)On Apr 24, 2024, at 3:26 PM, Ycarus (Yannick Chabanois) @.***> wrote: Thanks for the feedback. I will check why a reboot is needed.

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: @.***>

[kevinh-csalabs]

Are you using SQM and autorate for starlink?What scheduler and congestion control are you using for MPTCP? I've had good luck with ECF and bbr2.

Make sure multipath is enabled for all your wans!