Detecting which is Client side in Netlink path manager

[VenkateswaranJ]

I'm not sure whether I can ask this question on the GitHub issue page, but I couldn't find any other active forum to get a response.

In the above image FullMesh path manager creates 9 subflows between 3 interfaces. But one thing I would like to know is how it finds which side is the server and which side is the client. Because I see TCP syn request sent only from Client side for all 9 subflows (that means create subflow only initiated from client-side). Server-side only announcing its IP but never tries to initiate subflow connection (correct me if I'm wrong). Looks like it follows the client-server pattern of the initial subflow, and that's the way it avoids creating two subflows between the same set of IP addresses.

In case if Netlink path manager wants to do the same, how can it find which side is the server and which is the client, because mptcp events don't provide this info. Of course, it can take that info from the userspace client-server application, but I would like to know if there is any other better way of detecting it?

My main goal is to avoid creating two subflows between the same set of IP addresses. If I follow the fullmesh PM way then it can be avoided because the CREATE_SUBFLOW command is only sent from the client-side.

Also, in fullMesh PM why client announces its IP to the server because the server never gonna initiate subflow. Is there any logic behind it?

[matttbe]

But one thing I would like to know is how it finds which side is the server and which side is the client.

From the kernel side, it saves the info on who established the initial connection. Then by design, it doesn't create new subflows if it is the server side (because of NAT, etc.):

https://github.com/multipath-tcp/mptcp/blob/d6c8c0c0ca6cfd3992a925acee79df9cdc2f0a7f/net/mptcp/mptcp_fullmesh.c#L1361-L1363

In case if Netlink path manager wants to do the same, how can it find which side is the server and which is the client, because mptcp events don't provide this info. Of course, it can take that info from the userspace client-server application, but I would like to know if there is any other better way of detecting it?

Good point. Typically you know in advance if your userspace PM will be executed on a server or on a client. Maybe it is possible to query the kernel via Netlink about connections. Like what ss does. (Or by looking at /proc/net/mptcp maybe?) When designing the Netlink PM, it was not needed to expose this in the CREATE event (is it an in/outgoing connection?). This could be modified if needed but I guess in most cases, you don't really need this, no?

My main goal is to avoid creating two subflows between the same set of IP addresses. If I follow the fullmesh PM way then it can be avoided because the CREATE_SUBFLOW command is only sent from the client-side.

Indeed, safer to create subflow from the client side. In MPTCP Upstream project, by design, we don't allow subflows created by the server to the client.

Also, in fullMesh PM why client announces its IP to the server because the server never gonna initiate subflow. Is there any logic behind it?

Just in case another PM would want to do that. I agree in most cases, it is not needed.

[VenkateswaranJ]

Good point. Typically you know in advance if your userspace PM will be executed on a server or a client. Maybe it is possible to query the kernel via Netlink about connections. Like what ss does. (Or by looking at /proc/net/mptcp maybe?)

Yes, this works if the machine only acts as a server or client. In some situations, there are multiple Mptcp connections exist between two machines and each connection will be created by a different userspace application. So for example in Host A Application One creates a Tcp server and Application Two creates a Tcp client and the userspace path manager in Host A can only be either Client or Server(I mean either it can announce it's IP address or creating subflows). Userspace path manager can't behave differently for each connection unless we know which one is a server and which is a client on a per-connection basis (But fullmesh can do since it has per connection info).

Indeed, safer to create subflow from the client-side. In the MPTCP Upstream project, by design, we don't allow subflows created by the server to the client.

Hmm, So somehow userspace path manager needs to know whether it's server-side or client-side for each connection. Otherwise, it tries to execute create subflow command on the server-side but it's not allowed in the upstream kernel.

I see that fm path manager only has a mechanism to retry failed subflow creation once, but if the server announces the address and for some reason, it gets failed to reach the client, then there is no retry for announcing the address again isn't it? I can simulate it with scapy by dropping the announce message packet on the path, but please let me know if you know it already. https://github.com/multipath-tcp/mptcp/blob/d6c8c0c0ca6cfd3992a925acee79df9cdc2f0a7f/net/mptcp/mptcp_fullmesh.c#L1871

This won't be triggered if the client-side didn't receive the server announced address. https://github.com/multipath-tcp/mptcp/blob/d6c8c0c0ca6cfd3992a925acee79df9cdc2f0a7f/net/mptcp/mptcp_fullmesh.c#L1869

Does the server-side try announcing this address again until a new subflow has been created for it?

[matttbe]

Yes, this works if the machine only acts as a server or client.

I understand there are very specific use cases for that but then I would suggest to use multiple net namespace to work around this. For MPTCP Upstream project, we should probably consider about adding this info in the netlink notification from the kernel.

[matttbe]

Hmm, So somehow userspace path manager needs to know whether it's server-side or client-side for each connection. Otherwise, it tries to execute create subflow command on the server-side but it's not allowed in the upstream kernel.

Yes but likely, the client will not accept incoming connections: NAT, firewall, etc.

but if the server announces the address and for some reason, it gets failed to reach the client, then there is no retry for announcing the address again isn't it?

That's correct: with MPTCPv0 (RFC 6824), the ADD_ADDR are not reliable. It is reliable with MPTCPv1 (RFC8684). As a workaround, the server can retransmit the ADD_ADDR if no connection is seen.

Does the server-side try announcing this address again until a new subflow has been created for it?

Not with the FM PM. But a userspace PM could do that indeed.

[VenkateswaranJ]

Yes but likely, the client will not accept incoming connections: NAT, firewall, etc.

But somehow userspace PM needs to know should I create a subflow or just announce my address for this Mptcp connection. If you think this can be added to Netlink established event in the upstream kernel then please mention it in this issue https://github.com/multipath-tcp/mptcp_net-next/projects/12#card-72242834

And close this issue if not needed anymore.

[matttbe]

But somehow userspace PM needs to know should I create a subflow or just announce my address for this Mptcp connection.

I agree with that. It is just that it is quite unusual these days to have in the same NS apps that act as a client and as a server. But this is a good point and I think for the upstream kernel, we should add this info, easier than having to retrieve this info via another netlink request.

please mention it in this issue https://github.com/multipath-tcp/mptcp_net-next/projects/12#card-72242834

If you refer to https://github.com/multipath-tcp/mptcp_net-next/issues/186 , this task is to add "commands" support in the upstream kernel. But the "events" are already there. I added a point in the next weekly meeting we have to discuss with others if it makes sense to add a new field in the netlink events.