use linux-3.11.10-patch for linux ,and test the mptcp,fullmesh can't work. mptcp_capable and join on the same ipaddress

[leidemon]

hi,mptcp team: I use the v2ray for http proxy;and use curl for test,I want to test fullmesh，but it does not work(use the tcpdump -i vlan2 -n -v -e to check mptcp,but it does not capture any packages).I try ip link to set multipath backup; close mptcp_checksum;，it also no working. Hope for your replay. 1、mptcp-v3.11.10-18ffa79.patch 2、use v2ray for http_proxy; 3、env: 5 packets transmitted, 5 packets received, 0% packet loss round-trip min/avg/max = 26.994/48.041/94.723 ms /tmp# ifconfig usb1 usb1 Link encap:Ethernet HWaddr 02:1E:10:1F:00:00
inet addr:10.23.237.147 Bcast:10.23.237.151 Mask:255.255.255.248 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:7970 errors:0 dropped:0 overruns:0 frame:0 TX packets:6094 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:7836946 (7.4 MiB) TX bytes:465616 (454.7 KiB)

tmp# ifconfig vlan2 vlan2 Link encap:Ethernet HWaddr 00:90:BB:18:00:CC
inet addr:192.168.4.137 Bcast:192.168.4.255 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:19 errors:0 dropped:0 overruns:0 frame:0 TX packets:46 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:1932 (1.8 KiB) TX bytes:8312 (8.1 KiB)

tmp# ping 120.42.46.98 -I vlan2 PING 120.42.46.98 (120.42.46.98): 56 data bytes 64 bytes from 120.42.46.98: seq=0 ttl=243 time=51.483 ms 64 bytes from 120.42.46.98: seq=1 ttl=243 time=38.065 ms 64 bytes from 120.42.46.98: seq=2 ttl=243 time=34.667 ms 64 bytes from 120.42.46.98: seq=3 ttl=243 time=57.866 ms 64 bytes from 120.42.46.98: seq=4 ttl=243 time=47.338 ms 64 bytes from 120.42.46.98: seq=5 ttl=243 time=48.705 ms ^C --- 120.42.46.98 ping statistics --- 6 packets transmitted, 6 packets received, 0% packet loss round-trip min/avg/max = 34.667/46.354/57.866 ms admin@Four-Faith:/tmp# ping 120.42.46.98 -I usb1 PING 120.42.46.98 (120.42.46.98): 56 data bytes 64 bytes from 120.42.46.98: seq=0 ttl=245 time=94.723 ms 64 bytes from 120.42.46.98: seq=1 ttl=245 time=33.064 ms 64 bytes from 120.42.46.98: seq=2 ttl=245 time=26.994 ms 64 bytes from 120.42.46.98: seq=3 ttl=245 time=46.233 ms 64 bytes from 120.42.46.98: seq=4 ttl=245 time=39.193 ms ^C --- 120.42.46.98 ping statistics --- 5 packets transmitted, 5 packets received, 0% packet loss round-trip min/avg/max = 26.994/48.041/94.723 ms

4、mptcp debug :/tmp# curl http://speedtest.newark.linode.com/100MB-newark.bin -o /dev/null net/mptcp/mptcp_ctrl.c: mptcp_alloc_mpcb: created mpcb with token 0x3321c50b net/mptcp/mptcp_ctrl.c: mptcp_add_sock: token 0x3321c50b pi 1, src_addr:172.16.32.1:40355 dst_addr:172.16.32.1:8889, cnt_subflows now 1 net/mptcp/mptcp_ctrl.c: mptcp_select_ack_sock subsk is null, copied 0, cseq 236397728 net/mptcp/mptcp_ctrl.c: mptcp_select_ack_sock pi 1 state 2 last_dseq 0 net/mptcp/mptcp_ctrl.c: mptcp_alloc_mpcb: created mpcb with token 0x8a2afa9f net/mptcp/mptcp_ctrl.c: mptcp_add_sock: token 0x8a2afa9f pi 1, src_addr:172.16.32.1:8889 dst_addr:172.16.32.1:40355, cnt_subflows now 1 net/mptcp/mptcp_ctrl.c: mptcp_select_ack_sock subsk is null, copied 0, cseq 2777443490 net/mptcp/mptcp_ctrl.c: mptcp_select_ack_sock pi 1 state 3 last_dseq 0 % Total % Received % Xferd Average Speenet/mptcp/mptcp_ctrl.c: mptcp_add_sock: token 0x3321c50b pi 2, src_addr:0.0.0.0:0 dst_addr:0.0.0.0:0, cnt_subflows now 2 d Time Timenet/mptcp/mptcp_ipv4.c: mptcp_init4_subsockets: token 0x3321c50b pi 2 src_addr:10.23.237.147:0 dst_addr:172.16.32.1:8889 Time Currenet/mptcp/mptcp_ctrl.c: mptcp_add_sock: token 0x8a2afa9f pi 2, src_addr:172.16.32.1:8889 dst_addr:10.23.237.147:58306, cnt_subflows now 2 nt net/mptcp/mptcp_ctrl.c: mptcp_add_sock: token 0x3321c50b pi 3, src_addr:0.0.0.0:0 dst_addr:0.0.0.0:0, cnt_subflows now 3 net/mptcp/mptcp_ctrl.c: mptcp_alloc_mpcb: created mpcb with token 0x47d64cc5 Dload Uplonet/mptcp/mptcp_ctrl.c: mptcp_add_sock: token 0x47d64cc5 pi 1, src_addr:10.23.237.147:57580 dst_addr:120.42.46.98:23122, cnt_subflows now 1 ad Total Spenet/mptcp/mptcp_ctrl.c: mptcp_select_ack_sock subsk is null, copied 0, cseq 1623164748 nt Left Speenet/mptcp/mptcp_ctrl.c: mptcp_select_ack_sock pi 1 state 2 last_dseq 0 d 0 0 net/mptcp/mptcp_ipv4.c: mptcp_init4_subsockets: token 0x3321c50b pi 3 src_addr:192.168.4.137:0 dst_addr:172.16.32.1:8889 0 0 0 net/mptcp/mptcp_ctrl.c: mptcp_add_sock: token 0x47d64cc5 pi 2, src_addr:0.0.0.0:0 dst_addr:0.0.0.0:0, cnt_subflows now 2 0 0 net/mptcp/mptcp_ipv4.c: mptcp_init4_subsockets: token 0x47d64cc5 pi 2 src_addr:172.16.32.1:0 dst_addr:120.42.46.98:23122 0 --:--:-- --:--net/mptcp/mptcp_ctrl.c: mptcp_add_sock: token 0x47d64cc5 pi 3, src_addr:0.0.0.0:0 dst_addr:0.0.0.0:0, cnt_subflows now 3 :-- --:--:-- net/mptcp/mptcp_ipv4.c: mptcp_init4_subsockets: token 0x47d64cc5 pi 3 src_addr:192.168.4.137:0 dst_addr:120.42.46.98:23122 02022/01/26 14:net/mptcp/mptcp_ctrl.c: mptcp_add_sock: token 0x8a2afa9f pi 3, src_addr:172.16.32.1:8889 dst_addr:192.168.4.137:54023, cnt_subflows now 3 33:12 172.16.32.net/mptcp/mptcp_ctrl.c: mptcp_add_sock: token 0x3321c50b pi 4, src_addr:0.0.0.0:0 dst_addr:0.0.0.0:0, cnt_subflows now 4 1:40355 acceptednet/mptcp/mptcp_ipv4.c: mptcp_init4_subsockets: token 0x3321c50b pi 4 src_addr:172.16.32.1:0 dst_addr:10.23.237.147:8889 http://speedtesnet/mptcp/mptcp_ctrl.c: mptcp_add_sock: token 0x8a2afa9f pi 4, src_addr:10.23.237.147:8889 dst_addr:172.16.32.1:58725, cnt_subflows now 4 t.newark.linode.net/mptcp/mptcp_ctrl.c: mptcp_add_sock: token 0x3321c50b pi 5, src_addr:0.0.0.0:0 dst_addr:0.0.0.0:0, cnt_subflows now 5 com/100MB-newarknet/mptcp/mptcp_ipv4.c: mptcp_init4_subsockets: token 0x3321c50b pi 5 src_addr:10.23.237.147:0 dst_addr:10.23.237.147:8889 .bin [proxy] net/mptcp/mptcp_ctrl.c: mptcp_add_sock: token 0x8a2afa9f pi 5, src_addr:10.23.237.147:8889 dst_addr:10.23.237.147:51252, cnt_subflows now 5 net/mptcp/mptcp_ctrl.c: mptcp_add_sock: token 0x3321c50b pi 6, src_addr:0.0.0.0:0 dst_addr:0.0.0.0:0, cnt_subflows now 6 net/mptcp/mptcp_ipv4.c: mptcp_init4_subsockets: token 0x3321c50b pi 6 src_addr:192.168.4.137:0 dst_addr:10.23.237.147:8889 net/mptcp/mptcp_ctrl.c: mptcp_add_sock: token 0x8a2afa9f pi 6, src_addr:10.23.237.147:8889 dst_addr:192.168.4.137:50395, cnt_subflows now 6 net/mptcp/mptcp_ctrl.c: mptcp_add_sock: token 0x3321c50b pi 7, src_addr:0.0.0.0:0 dst_addr:0.0.0.0:0, cnt_subflows now 7 net/mptcp/mptcp_ipv4.c: mptcp_init4_subsockets: token 0x3321c50b pi 7 src_addr:172.16.32.1:0 dst_addr:192.168.4.137:8889 net/mptcp/mptcp_ctrl.c: mptcp_add_sock: token 0x8a2afa9f pi 7, src_addr:192.168.4.137:8889 dst_addr:172.16.32.1:37106, cnt_subflows now 7 net/mptcp/mptcp_ctrl.c: mptcp_add_sock: token 0x3321c50b pi 8, src_addr:0.0.0.0:0 dst_addr:0.0.0.0:0, cnt_subflows now 8 net/mptcp/mptcp_ipv4.c: mptcp_init4_subsockets: token 0x3321c50b pi 8 src_addr:10.23.237.147:0 dst_addr:192.168.4.137:8889 net/mptcp/mptcp_ctrl.c: mptcp_add_sock: token 0x8a2afa9f pi 8, src_addr:192.168.4.137:8889 dst_addr:10.23.237.147:40343, cnt_subflows now 8 net/mptcp/mptcp_ctrl.c: mptcp_add_sock: token 0x3321c50b pi 9, src_addr:0.0.0.0:0 dst_addr:0.0.0.0:0, cnt_subflows now 9 net/mptcp/mptcp_ipv4.c: mptcp_init4_subsockets: token 0x3321c50b pi 9 src_addr:192.168.4.137:0 dst_addr:192.168.4.137:8889 net/mptcp/mptcp_ctrl.c: mptcp_add_sock: token 0x8a2afa9f pi 9, src_addr:192.168.4.137:8889 dst_addr:192.168.4.137:55409, cnt_subflows now 9 3 100M 3 3807k 0 0 779k 0 0:02:11 0:00:04 0:02:07 790k^Cnet/mptcp/mptcp_ctrl.c: mptcp_close: Close of meta_sk with tok 0x3321c50b

/tmp# net/mptcp/mptcp_ctrl.c: mptcp_del_sock: Removing subsock tok 0x3321c50b pi:9 state 7 is_meta? 0 net/mptcp/mptcp_ctrl.c: mptcp_close: Close of meta_sk with tok 0x8a2afa9f net/mptcp/mptcp_ctrl.c: mptcp_del_sock: Removing subsock tok 0x3321c50b pi:8 state 7 is_meta? 0 net/mptcp/mptcp_ctrl.c: mptcp_del_sock: Removing subsock tok 0x3321c50b pi:7 state 7 is_meta? 0 net/mptcp/mptcp_ctrl.c: mptcp_del_sock: Removing subsock tok 0x3321c50b pi:6 state 7 is_meta? 0 net/mptcp/mptcp_ctrl.c: mptcp_del_sock: Removing subsock tok 0x3321c50b pi:5 state 7 is_meta? 0 net/mptcp/mptcp_ctrl.c: mptcp_del_sock: Removing subsock tok 0x3321c50b pi:4 state 7 is_meta? 0 net/mptcp/mptcp_ctrl.c: mptcp_del_sock: Removing subsock tok 0x3321c50b pi:3 state 7 is_meta? 0 net/mptcp/mptcp_ctrl.c: mptcp_del_sock: Removing subsock tok 0x3321c50b pi:2 state 7 is_meta? 0 2022/01/26 14:33net/mptcp/mptcp_ctrl.c: mptcp_del_sock: Removing subsock tok 0x3321c50b pi:1 state 7 is_meta? 0 :17 [Warning] [2net/mptcp/mptcp_ctrl.c: mptcp_sock_destruct destroying meta-sk 535608694] githunet/mptcp/mptcp_ctrl.c: mptcp_del_sock: Removing subsock tok 0x8a2afa9f pi:9 state 7 is_meta? 0 b.com/v2fly/v2ranet/mptcp/mptcp_ctrl.c: mptcp_del_sock: Removing subsock tok 0x8a2afa9f pi:7 state 7 is_meta? 0 y-core/v4/app/prnet/mptcp/mptcp_ctrl.c: mptcp_del_sock: Removing subsock tok 0x8a2afa9f pi:6 state 7 is_meta? 0 oxyman/inbound: net/mptcp/mptcp_ctrl.c: mptcp_del_sock: Removing subsock tok 0x8a2afa9f pi:5 state 7 is_meta? 0 connection ends net/mptcp/mptcp_ctrl.c: mptcp_del_sock: Removing subsock tok 0x8a2afa9f pi:4 state 7 is_meta? 0

github.com/v2fnet/mptcp/mptcp_ctrl.c: mptcp_del_sock: Removing subsock tok 0x8a2afa9f pi:3 state 7 is_meta? 0 ly/v2ray-core/v4net/mptcp/mptcp_ctrl.c: mptcp_del_sock: Removing subsock tok 0x8a2afa9f pi:2 state 7 is_meta? 0 /proxy/http: connet/mptcp/mptcp_ctrl.c: mptcp_del_sock: Removing subsock tok 0x8a2afa9f pi:1 state 7 is_meta? 0 nection ends > gnet/mptcp/mptcp_ctrl.c: mptcp_del_sock: Removing subsock tok 0x8a2afa9f pi:8 state 7 is_meta? 0 ithub.com/v2fly/net/mptcp/mptcp_ctrl.c: mptcp_close: Close of meta_sk with tok 0x47d64cc5 v2ray-core/v4/prnet/mptcp/mptcp_ctrl.c: mptcp_del_sock: Removing subsock tok 0x47d64cc5 pi:3 state 7 is_meta? 0 oxy/http: failednet/mptcp/mptcp_ctrl.c: mptcp_del_sock: Removing subsock tok 0x47d64cc5 pi:1 state 7 is_meta? 0 to write responnet/mptcp/mptcp_ctrl.c: mptcp_del_sock: Removing subsock tok 0x47d64cc5 pi:2 state 7 is_meta? 0 se > readfrom tcnet/mptcp/mptcp_ctrl.c: mptcp_sock_destruct destroying meta-sk p 172.16.32.1:88net/mptcp/mptcp_ctrl.c: mptcp_sock_destruct destroying meta-sk 89->172.16.32.1:40355: write tcp 172.16.32.1:8889->172.16.32.1:40355: write: broken pipe

/tmp# cat /proc/sys/net/mptcp/mptcp_debug 1 /tmp# cat /proc/sys/net/mptcp/mptcp_enabled 1 /tmp# cat /proc/sys/net/mptcp/mptcp_path_manager fullmesh /tmp# cat /proc/sys/net/mptcp/mptcp_checksum 1 /tmp#

/tmp# /usr/sbin/ip link set vlan2 multipath backup /tmp# cat /sys/devices/virtual/net/vlan2/flags 0x101003

[matttbe]

Hi @leidemon

Similar to your previous issue #371, this version 3.11 is very old, from 8 years ago! Obviously not maintained any more.

I have no idea if this is due to a configuration issue or a bug in the kernel. All I can say is that you should try to reproduce the issue with MPTCP v0.95 (v4.19) or v0.96 (v5.4).

I guess I should better close this ticket as it is unlikely someone will send a patch for this old kernel. If you can reproduce it with a supported kernel, please open a new issue.

[leidemon]

Hi @matttbe Thank you for your reply.I know it is too old.any suggestion for it to debug(setup? param? kernel debug?)?

[matttbe]

I don't know sorry.

Do you have access to /proc/net/mptcp_fullmesh in this old kernel? Did you set the routing rules? https://multipath-tcp.org/pmwiki.php/Users/ConfigureRouting Did you check packet traces to see if SYN are emitted?

[leidemon]

I don't know sorry.

Do you have access to /proc/net/mptcp_fullmesh in this old kernel? Did you set the routing rules? https://multipath-tcp.org/pmwiki.php/Users/ConfigureRouting Did you check packet traces to see if SYN are emitted?

Hi,@matttbe: Thank you for your reply.I'm sorry I didn't reply until today ,we just have finished the Spring Festival holiday. 1、I can not access the /proc/net/mptcp_fullmesh,only /proc/net/mptcp 2、yes,I set the routing rules.in the above ,you can see ,the two links is works well. 3、yes,I capture the packets,it only works on the main links usb1-mptcp.zip .

[matttbe]

2、yes,I set the routing rules.in the above ,you can see ,the two links is works well.

I don't think we are talking about the same thing. ip link set vlan2 multipath backup will tell the scheduler to send data over this subflow only if no other non-backup subflows are available. In other words, it will use subflows linked to vlan2 only when your vlan1 interface or its attached attached network is down. (I don't know if your old kernel supports this feature.)

What you need is to tell the kernel that connections with usb1's source address have to go thought usb1 interface: https://multipath-tcp.org/pmwiki.php/Users/ConfigureRouting

If your default route is via vlan2, it is enough to add rules for usb1 interface:

ip rule add from 10.23.237.147 table 2
ip route add default via <nexthop> dev usb1 table 2

(check your existing route if you don't know what's your nexthop on this interface → something from 10.23.237.145 to 10.23.237.150 but not .147)

[leidemon]

2、是的，我设置了路由规则。在上面，你可以看到，两个链接都很好。

我不认为我们在谈论同一件事。 ip link set vlan2 multipath backup只有在没有其他非备份子流可用时，才会告诉调度程序通过该子流发送数据。换句话说，vlan2只有当您的vlan1接口或其连接的网络关闭时，它才会使用链接的子流。（不知道你的旧内核是否支持这个功能。）

您需要告诉内核与usb1源地址的连接必须经过思考usb1接口：https ://multipath-tcp.org/pmwiki.php/Users/ConfigureRouting

如果您的默认路由是 via vlan2，则为usb1接口添加规则就足够了：

ip rule add from 10.23.237.147 table 2
ip route add default via <nexthop> dev usb1 table 2

（如果您不知道此接口上的下一跳是什么，请检查您现有的路线→从 10.23.237.145 到 10.23.237.150 但不是 0.147 的东西） Hi,@matttbe I know，I try the both，vlan2 multipath(on/backup),they both can not work，I also set the policy route(both can ping v2ray server through network)。it seems that something wrong with kernel config?Is MPTCP incompatible with other kernel options？

[matttbe]

If the routing rules are correct, I don't know why it doesn't work. Probably a bug in the kernel but with a so old kernel, I don't know what else you can do apart from debugging and patching the kernel yourself :-/

[leidemon]

如果路由规则是正确的，我不知道为什么它不起作用。可能是内核中的一个错误，但是内核太旧了，除了自己调试和修补内核之外，我不知道您还能做什么：-/

ok,thank you.