Open SriramScorp opened 1 year ago
Hi @SriramScorp
Thank you for the bug report and sorry for the delay.
However, traffic seems to flow mostly through one interface. And, even that becomes stale within seconds. (...) When trying to download a file via the socks proxy, transfer rate drops to 0 within 5-8 secs and never recovers.
It seems to be an issue with the packet scheduler (if the different networks being used are OK)
5.15.0-1029-raspi
Do you mind trying to reproduce the issue with a more recent kernel version for both the client and the server side (at least for the sender side)? By order of preference: the version from our export
branch, the last version from Linus (v6.4-rc7), the last stable version (v6.3) or the last LTS (v6.1).
Some improvements have been done on the packet scheduler side but still, it should not stale.
pi@ubuntu:~$ ip mptcp endpoint show 192.168.0.124 id 1 signal subflow dev eth0 192.168.20.2 id 2 signal subflow dev wlan0
(...) I am trying to aggregate multiple WANs using shadowsocks. Could you provide more info on how to configure the path manager on the server and client side, viz. how to set add_addr_accepted, when to use the signal and/or subflow flags etc. ?
It is a different issue but it is strange to mark the endpoints with both signal
(server side, to announce IP addresses) and subflow
flags (client side, to initiate new subflows): you might have unexpected behaviours regarding the paths creation but you didn't add other endpoints and you don't accept ADD_ADDR on the server side so it should be fine I think. The rest looks good I think. Always good to start with higher limits and restrict later.
Also, can the endpoint id be any random, unique positive integer? Or should it be the same as the table ID from Configure Routing ?
The endpoint ID is optional and it is not linked to the table ID. Except if you have a specific use-case, it is often better not to set it.
Hello
Did you manage to reproduce the issue with a more recent kernel and with the proper endpoints flags?
No, I haven't gotten back to it yet. Got caught up with other stuff. As such, still using version 0.96.
Hi @matttbe!
Tried with a more recent kernel
as suggested. Issue seems to persist. Now, the connection doesn't become stale, yet data traffic seems to flow only through one of the interfaces.
Client-side info:
pi@raspberrypi:~$
pi@raspberrypi:~$ sed 's/$/\n/' /proc/device-tree/model
Raspberry Pi 5 Model B Rev 1.0
pi@raspberrypi:~$
pi@raspberrypi:~$
pi@raspberrypi:~$ cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux 12 (bookworm)"
NAME="Debian GNU/Linux"
VERSION_ID="12"
VERSION="12 (bookworm)"
VERSION_CODENAME=bookworm
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"
pi@raspberrypi:~$
pi@raspberrypi:~$
pi@raspberrypi:~$ uname -m
aarch64
pi@raspberrypi:~$
pi@raspberrypi:~$ uname -r
6.6.20+rpt-rpi-2712
pi@raspberrypi:~$
pi@raspberrypi:~$ grep MPTCP /boot/config-$(uname -r)
CONFIG_MPTCP=y
CONFIG_INET_MPTCP_DIAG=m
pi@raspberrypi:~$
pi@raspberrypi:~$ sysctl net.mptcp.enabled
net.mptcp.enabled = 1
pi@raspberrypi:~$
pi@raspberrypi:~$
pi@raspberrypi:~$ ip -br a
lo UNKNOWN 127.0.0.1/8 ::1/128
eth0 UP 192.168.0.102/24 fd01::85c3:db:1cc:819c/64 fe80::b403:e83b:30c7:6234/64
wlan0 UP 192.168.40.2/24 fe80::8d9e:bc3d:c9f7:d5e/64
pi@raspberrypi:~$
pi@raspberrypi:~$
pi@raspberrypi:~$ ip r
default via 192.168.0.1 dev eth0 proto dhcp src 192.168.0.102 metric 100
default via 192.168.40.1 dev wlan0 proto dhcp src 192.168.40.2 metric 600
192.168.0.0/24 dev eth0 proto kernel scope link src 192.168.0.102 metric 100
192.168.40.0/24 dev wlan0 proto kernel scope link src 192.168.40.2 metric 600
pi@raspberrypi:~$
pi@raspberrypi:~$
pi@raspberrypi:~$ ip rule list table 11
32765: from 192.168.0.102 lookup 11
pi@raspberrypi:~$
pi@raspberrypi:~$ ip rule list table 12
32764: from 192.168.40.2 lookup 12
pi@raspberrypi:~$
pi@raspberrypi:~$ ip route list table 11
default via 192.168.0.1 dev eth0
pi@raspberrypi:~$
pi@raspberrypi:~$ ip route list table 12
default via 192.168.40.1 dev wlan0
pi@raspberrypi:~$
pi@raspberrypi:~$
pi@raspberrypi:~$ ip mptcp limits show
add_addr_accepted 0 subflows 4
pi@raspberrypi:~$
pi@raspberrypi:~$ ip mptcp endpoint show
192.168.0.102 id 1 subflow dev eth0
192.168.40.2 id 3 subflow dev wlan0
pi@raspberrypi:~$
pi@raspberrypi:~$
pi@raspberrypi:~$ curl -x socks5h://127.0.0.1:1080 --connect-timeout 5 icanhazip.com
4.3.2.1 (obfuscated server IP)
pi@raspberrypi:~$
pi@raspberrypi:~$ ip mptcp monitor
[ CREATED] token=de9af55a remid=0 locid=0 saddr4=192.168.0.102 daddr4=4.3.2.1 sport=53656 dport=8388
[ ESTABLISHED] token=de9af55a remid=0 locid=0 saddr4=192.168.0.102 daddr4=4.3.2.1 sport=53656 dport=8388
[ CLOSED] token=de9af55a
^C
pi@raspberrypi:~$
When trying to download a file via the socks proxy,
pi@raspberrypi:~$
pi@raspberrypi:~$ curl -x socks5h://127.0.0.1:1080 --connect-timeout 5 -o /dev/null 'http://commondatastorage.googleapis.com/gtv-videos-bucket/sample/WhatCarCanYouGetForAGrand.mp4'
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 43.4M 100 43.4M 0 0 7260k 0 0:00:06 0:00:06 --:--:-- 8934k
pi@raspberrypi:~$
pi@raspberrypi:~$ ip mptcp monitor
[ CREATED] token=204bca95 remid=0 locid=0 saddr4=192.168.0.102 daddr4=4.3.2.1 sport=51616 dport=8388
[ ESTABLISHED] token=204bca95 remid=0 locid=0 saddr4=192.168.0.102 daddr4=4.3.2.1 sport=51616 dport=8388
[SF_ESTABLISHED] token=204bca95 remid=0 locid=3 saddr4=192.168.40.2 daddr4=4.3.2.1 sport=59163 dport=8388 backup=0 ifindex=3
[ CLOSED] token=204bca95
^C
pi@raspberrypi:~$
pi@raspberrypi:~$
pi@raspberrypi:~$ vnstat -i eth0 --fiveminutes | tail -n5
16:55 45.99 MiB | 745.37 KiB | 46.72 MiB | 1.31 Mbit/s
17:00 29.47 KiB | 25.30 KiB | 54.78 KiB | 1.50 kbit/s
17:05 48.12 MiB | 812.89 KiB | 48.91 MiB | 1.37 Mbit/s
17:10 23.09 KiB | 6.42 KiB | 29.50 KiB | 805 bit/s
------------------------+-------------+-------------+---------------
pi@raspberrypi:~$
pi@raspberrypi:~$ vnstat -i wlan0 --fiveminutes | tail -n5
16:55 87.99 KiB | 6.42 KiB | 94.40 KiB | 2.58 kbit/s
17:00 76 B | 384 B | 460 B | 12 bit/s
17:05 50.77 KiB | 4.72 KiB | 55.49 KiB | 1.51 kbit/s
17:10 0 B | 0 B | 0 B | 0 bit/s
------------------------+-------------+-------------+---------------
pi@raspberrypi:~$
After changing the interface metric,
pi@raspberrypi:~$
pi@raspberrypi:~$ ip r
default via 192.168.40.1 dev wlan0 proto dhcp src 192.168.40.2 metric 50
default via 192.168.0.1 dev eth0 proto dhcp src 192.168.0.102 metric 100
192.168.0.0/24 dev eth0 proto kernel scope link src 192.168.0.102 metric 100
192.168.40.0/24 dev wlan0 proto kernel scope link src 192.168.40.2 metric 600
pi@raspberrypi:~$
pi@raspberrypi:~$
pi@raspberrypi:~$ curl -x socks5h://127.0.0.1:1080 --connect-timeout 5 -o /dev/null 'http://commondatastorage.googleapis.com/gtv-videos-bucket/sample/WhatCarCanYouGetForAGrand.mp4'
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 43.4M 100 43.4M 0 0 6881k 0 0:00:06 0:00:06 --:--:-- 9275k
pi@raspberrypi:~$
pi@raspberrypi:~$ ip mptcp monitor
[ CREATED] token=bf8135b2 remid=0 locid=0 saddr4=192.168.40.2 daddr4=4.3.2.1 sport=42842 dport=8388
[ ESTABLISHED] token=bf8135b2 remid=0 locid=0 saddr4=192.168.40.2 daddr4=4.3.2.1 sport=42842 dport=8388
[SF_ESTABLISHED] token=bf8135b2 remid=0 locid=1 saddr4=192.168.0.102 daddr4=4.3.2.1 sport=56301 dport=8388 backup=0 ifindex=2
[ CLOSED] token=bf8135b2
^C
pi@raspberrypi:~$
pi@raspberrypi:~$
pi@raspberrypi:~$ vnstat -i eth0 --fiveminutes | tail -n5
18:10 41.18 KiB | 29.49 KiB | 70.68 KiB | 1.93 kbit/s
18:15 49.02 MiB | 873.39 KiB | 49.88 MiB | 1.39 Mbit/s
18:20 23.67 KiB | 85.88 KiB | 109.55 KiB | 2.99 kbit/s
18:25 46.06 MiB | 815.25 KiB | 46.86 MiB | 1.31 Mbit/s
------------------------+-------------+-------------+---------------
pi@raspberrypi:~$
pi@raspberrypi:~$ vnstat -i wlan0 --fiveminutes | tail -n5
18:10 0 B | 0 B | 0 B | 0 bit/s
18:15 94.11 KiB | 7.71 KiB | 101.82 KiB | 2.78 kbit/s
18:20 1.65 KiB | 2.65 KiB | 4.30 KiB | 117 bit/s
18:25 45.43 KiB | 6.48 KiB | 51.91 KiB | 1.42 kbit/s
------------------------+-------------+-------------+---------------
pi@raspberrypi:~$
Server-side info:
root@ubu23-mptcpv1:~#
root@ubu23-mptcpv1:~# cat /etc/os-release
PRETTY_NAME="Ubuntu 23.10"
NAME="Ubuntu"
VERSION_ID="23.10"
VERSION="23.10 (Mantic Minotaur)"
VERSION_CODENAME=mantic
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=mantic
LOGO=ubuntu-logo
root@ubu23-mptcpv1:~#
root@ubu23-mptcpv1:~#
root@ubu23-mptcpv1:~# uname -m
x86_64
root@ubu23-mptcpv1:~#
root@ubu23-mptcpv1:~# uname -r
6.5.0-9-generic
root@ubu23-mptcpv1:~#
root@ubu23-mptcpv1:~# grep MPTCP /boot/config-$(uname -r)
CONFIG_MPTCP=y
CONFIG_INET_MPTCP_DIAG=m
CONFIG_MPTCP_IPV6=y
root@ubu23-mptcpv1:~#
root@ubu23-mptcpv1:~# sysctl net.mptcp.enabled
net.mptcp.enabled = 1
root@ubu23-mptcpv1:~#
root@ubu23-mptcpv1:~#
root@ubu23-mptcpv1:~# ip mptcp limits show
add_addr_accepted 0 subflows 2
root@ubu23-mptcpv1:~#
root@ubu23-mptcpv1:~# ip mptcp endpoint show
4.3.2.1 id 1 signal dev eth0
root@ubu23-mptcpv1:~#
icanhazip.com
request
root@ubu23-mptcpv1:~#
root@ubu23-mptcpv1:~# ip mptcp monitor
[ CREATED] token=0ec5cf79 remid=0 locid=0 saddr4=4.3.2.1 daddr4=182.83.179.97 sport=8388 dport=30579
[ ESTABLISHED] token=0ec5cf79 remid=0 locid=0 saddr4=4.3.2.1 daddr4=182.83.179.97 sport=8388 dport=30579
[ CLOSED] token=a36d677e
[ CLOSED] token=0ec5cf79
^C
root@ubu23-mptcpv1:~#
googleapis.com
request
root@ubu23-mptcpv1:~#
root@ubu23-mptcpv1:~# ip mptcp monitor
[ CREATED] token=55ca4382 remid=0 locid=0 saddr4=4.3.2.1 daddr4=182.83.179.97 sport=8388 dport=30582
[ ESTABLISHED] token=55ca4382 remid=0 locid=0 saddr4=4.3.2.1 daddr4=182.83.179.97 sport=8388 dport=30582
[SF_ESTABLISHED] token=55ca4382 remid=3 locid=0 saddr4=4.3.2.1 daddr4=22.67.40.231 sport=8388 dport=7744 backup=0
[ CLOSED] token=b44e2385
[ CLOSED] token=55ca4382
^C
root@ubu23-mptcpv1:~#
After metric change,
root@ubu23-mptcpv1:~#
root@ubu23-mptcpv1:~# ip mptcp monitor
[ CREATED] token=ceb2158c remid=0 locid=0 saddr4=4.3.2.1 daddr4=22.67.40.231 sport=8388 dport=11778
[ ESTABLISHED] token=ceb2158c remid=0 locid=0 saddr4=4.3.2.1 daddr4=22.67.40.231 sport=8388 dport=11778
[SF_ESTABLISHED] token=ceb2158c remid=1 locid=0 saddr4=4.3.2.1 daddr4=182.83.179.97 sport=8388 dport=30919 backup=0
[ CLOSED] token=0fffe81c
[ CLOSED] token=ceb2158c
^C
root@ubu23-mptcpv1:~#
Hi, New to MPTCP v1. Just tried this tutorial.
ip mptcp monitor
suggests subflows getting created. However, traffic seems to flow mostly through one interface. And, even that becomes stale within seconds.Client-side info:
When trying to download a file via the socks proxy, transfer rate drops to 0 within 5-8 secs and never recovers.
Server-side info:
icanhazip.com
requestgoogleapis.com
requestI am trying to aggregate multiple WANs using shadowsocks. Could you provide more info on how to configure the path manager on the server and client side, viz. how to set
add_addr_accepted
, when to use thesignal
and/orsubflow
flags etc. ?Also, can the endpoint
id
be any random, unique positive integer? Or should it be the same as the table ID from Configure Routing ?