Always shows MPTCP not working on curl http://www.multipath-tcp.org

[govwin]

Hi,

I installed *.deb from https://github.com/multipath-tcp/mptcp/releases, booted to new kernel, and configured routing based on http://multipath-tcp.org/pmwiki.php/Users/ConfigureRouting. Below are the details of the system. It is a virtual machine on VirtualBox. curl http://www.multipath-tcp.org is always showing no MPTCP. I cannot find any problem. How can I get MPTCP running?

$ curl http://www.multipath-tcp.org
Nay, Nay, Nay, your have an old computer that does not speak MPTCP. Shame on you!
$ uname -a
Linux a 4.19.126.mptcp #20200611235134 SMP Thu Jun 11 23:55:29 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
$ sudo dmesg | grep MPTCP
[    0.967051] MPTCP: Stable release v0.95.1
$ sysctl net.mptcp
net.mptcp.mptcp_checksum = 1
net.mptcp.mptcp_debug = 0
net.mptcp.mptcp_enabled = 1
net.mptcp.mptcp_path_manager = fullmesh
net.mptcp.mptcp_scheduler = default
net.mptcp.mptcp_syn_retries = 3
net.mptcp.mptcp_version = 0
$ ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: enp0s3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 08:00:27:d9:9f:6f brd ff:ff:ff:ff:ff:ff
    inet 10.0.2.15/24 brd 10.0.2.255 scope global dynamic noprefixroute enp0s3
       valid_lft 82052sec preferred_lft 82052sec
    inet6 fe80::1414:466b:b318:b742/64 scope link noprefixroute
       valid_lft forever preferred_lft forever
3: enp0s8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 08:00:27:a0:34:fa brd ff:ff:ff:ff:ff:ff
    inet 10.0.3.15/24 brd 10.0.3.255 scope global dynamic noprefixroute enp0s8
       valid_lft 82052sec preferred_lft 82052sec
    inet6 fe80::10f4:3077:79ba:8f98/64 scope link noprefixroute
       valid_lft forever preferred_lft forever
$ ip rule show
0:      from all lookup local
32764:  from 10.0.3.15 lookup 2
32765:  from 10.0.2.15 lookup 1
32766:  from all lookup main
32767:  from all lookup default
$ ip route
default via 10.0.2.2 dev enp0s3
default via 10.0.2.2 dev enp0s3 proto dhcp metric 100
default via 10.0.3.2 dev enp0s8 proto dhcp metric 101
10.0.2.0/24 dev enp0s3 proto kernel scope link src 10.0.2.15 metric 100
10.0.3.0/24 dev enp0s8 proto kernel scope link src 10.0.3.15 metric 101
169.254.0.0/16 dev enp0s3 scope link metric 1000
$ ip route show table 1
default via 10.0.2.2 dev enp0s3
10.0.2.0/24 dev enp0s3 scope link
$ ip route show table 2
default via 10.0.3.2 dev enp0s8
10.0.3.0/24 dev enp0s8 scope link

[matttbe]

Hi @govwin ,

The best would be to look at packet traces (tcpdump/wireshark) to check what is sent (TCP SYN with MPTCP options?) and what is received (with MPTCP).

If you see sent MPTCP option but none are received or if you are not allow to take packet traces, the best is to use a tool like tracebox to check if MPTCP options are not dropped by a host between you and the end server. Check http://www.tracebox.org/ or #230 or many other tickets mentioning tracebox.

[govwin]

Hi @matttbe ,

thank you for the reply. I now have checked packets with wireshark on Windows host. There is no option kind 30 in the first SYN packet sent to destination 130.104.228.140. The last 12 bytes are options and in hex 02 04 05 b4 01 03 03 08 01 01 04 02 and there is no MPTCP options in these.

[matttbe]

Can you see the MPTCP options in packet taken from the VM?

sudo tcpdump -i enp0s3 -n tcp and host 130.104.228.140

[govwin]

I think there is (1e0c) MPTCP option in the VM tcpdump (attached below). That means either VirtualBox or Windows 10 host removes that option right?

a@a:~$ sudo tcpdump -e -nnXSs 0 -vv host 130.104.228.140
tcpdump: listening on enp0s3, link-type EN10MB (Ethernet), capture size 262144 by
17:15:17.379353 08:00:27:d9:9f:6f > 52:54:00:12:35:02, ethertype IPv4 (0x0800), lngth 72)
    10.0.2.15.51858 > 130.104.228.140.80: Flags [S], cksum 0x733e (incorrect -> 0 ecr 0,nop,wscale 7,mptcp capable csum {0x1430516b5d9209ed}], length 0
        0x0000:  4500 0048 2a8f 4000 4006 9d1d 0a00 020f  E..H*.@.@.......
        0x0010:  8268 e48c ca92 0050 7664 ec49 0000 0000  .h.....Pvd.I....
        0x0020:  d002 faf0 733e 0000 0204 05b4 0402 080a  ....s>..........
        0x0030:  7cb4 4d7d 0000 0000 0103 0307 1e0c 0081  |.M}............
        0x0040:  1430 516b 5d92 09ed                      .0Qk]...

[matttbe]

If you see MPTCP in the option (mptcp capable) in the VM but not on the host, yes, that means either VirtualBox or Windows 10 host removes the option.

I guess it could be linked to the network settings on VirtualBox. I didn't use it for a while but maybe I remember it was good to use it in a "bridge" mode but I'm not sure I would be able to help more than that. Feel free to share what you find.

[govwin]

Thanks a lot, the Bridge Adapter mode does not remove mptcp capable option and the problem is solved. NAT was creating the problem.

a@a:~$ curl http://www.multipath-tcp.org
Yay, you are MPTCP-capable! You can now rest in peace.

[yogiex]

hello, im using ubuntu 18 LTS and setting bridge but curl not working

[matttbe]

@yogiex : as mentioned above, to debug this issue, you need to look if MPTCP options are sent in the connection request and if you receive MPTCP ones as well. In the original issue, VirtualBox was stripping MPTCP option but middleboxes on the Internet could do the same.

Best is then to take a packet capture (tcpdump or wireshark) and/or use a tool like tracebox, see above.