multipath-tcp / mptcp

⚠️⚠️⚠️ Deprecated 🚫 Out-of-tree Linux Kernel implementation of MultiPath TCP. 👉 Use https://github.com/multipath-tcp/mptcp_net-next repo instead ⚠️⚠️⚠️
https://github.com/multipath-tcp/mptcp_net-next
Other
889 stars 335 forks source link

kernel: __mptcp_init4_subsockets: MPTCP subsocket connect() failed, error -101 #472

Closed VenkateswaranJ closed 1 year ago

VenkateswaranJ commented 2 years ago

sometimes I used to get this message when I tried to create subflows via Netlink.

mptcp

This is my current set up and I just up and down switch ports 1 & 2. I have some script in my server machine which tells ip address add/remove to the mptcp userspace path manager and the PM will create subflows based on the detected IP.

So as per the error code -101 ENETUNREACH the network is unreachable but I wonder that I only send create subflow command to mptcp kernel when I detect new ip then how it's possible (I mean the kernel routing table should have already been updated).

matttbe commented 2 years ago

Hello

Do you have such error with a custom userspace PM? If yes, when do have such issues? Only why you get a signal a new IP is available? Or also when you establish a new subflow after a connection has been established?

If you have issues when trying to use a new IP, it is indeed maybe because the routing is not ready. I don't know how the routing is managed on your side but maybe that's an external module which is in charge of adding ip rule (ip rule add from <IP> table <table>). In this case, it is better to wait for the routing rules to be ready before continuing.

Similar to https://github.com/intel/mptcpd/commit/fda8f3ec3a8990920e3a2e27704a3b23a670adc9

VenkateswaranJ commented 2 years ago

Do you have such an error with a custom userspace PM?

Yes

also when do you establish a new subflow after a connection has been established?

Not always but sometimes.

In this case, it is better to wait for the routing rules to be ready before continuing.

I'll try this idea.

I have one more question. In the above diagram switch, ports are configured with a spanning tree which means when I pull out the cable and connect back it will take around 30 to 40 seconds to complete the port check but in my host, I already get the IP address added event and the userspace PM starts to send create_subflow() request to the kernel but the packet can't reach another side because of the switch port is still in spanning tree check and not allowing any packet to another side.

So how can I tackle this problem? should I need to ping the destination IP before sending the create_subflow() request? Or is there any rtnetlink event that tells the reachability of destination IP?

matttbe commented 2 years ago

also when do you establish a new subflow after a connection has been established?

Not always but sometimes.

Do you wait for the ESTABLISHED event? Because MPTCP_EVENT_CREATED is too early to create subflows but I guess you don't use that.

In this case, it is better to wait for the routing rules to be ready before continuing.

I'll try this idea.

:+1:

I have one more question. In the above diagram switch, ports are configured with a spanning tree which means when I pull out the cable and connect back it will take around 30 to 40 seconds to complete the port check but in my host, I already get the IP address added event and the userspace PM starts to send create_subflow() request to the kernel but the packet can't reach another side because of the switch port is still in spanning tree check and not allowing any packet to another side.

So how can I tackle this problem? should I need to ping the destination IP before sending the create_subflow() request? Or is there any rtnetlink event that tells the reachability of destination IP?

I know there are netlink commands to validate that a route is OK but if you need to verify an IP is OK outside a container/host, you will need to ping.

VenkateswaranJ commented 2 years ago

Do you wait for the ESTABLISHED event?

Yes, I wait for this.

I know there are NetLink commands to validate that a route is OK but if you need to verify an IP is OK outside a container/host, you will need to ping.

the route is OK and the destination IP is reachable, aren't they both are same? or does the route is okonly means that the host can reach the specific route gateway something like that?

matttbe commented 2 years ago

You can maybe have a route but the next hop is not reachable.

VenkateswaranJ commented 2 years ago

@matttbe Just one more question before closing this issue. I want to edit the mptcp option in the packet using iptable, is it possible ?

I want to capture announce packet and edit the Add Adress(3) subtype information (basically I want to change the announce ip in the packet to some external ip).

Is this possible to achive via iptable rules ?

I have done this similar thing with scappy but I need to do some checksum calculation and stuff.

matttbe commented 2 years ago

I'm pretty sure you cannot do the modification directly. To edit the packet, you might need to use an NFQueue and edit the packet like you did before with Scappy. You can edit some fields with IPTables and TC but only specific fields, probably not the address in MPTCP options.

VenkateswaranJ commented 1 year ago

@matttbe We can close this one.

matttbe commented 1 year ago

Thanks!