matttbe
commented
5 months ago Hello,
I have a server-client connected with two interfaces.
Just to know what you expect, can each address reaches any address of the other peer? e.g. 10.10.10.2 <-> 20.20.20.3?
Or here you can only have max 2 subflows in this setup?
client side mptcp config
$ ip mptcp endpoint show 20.20.20.2 id 3 fullmesh dev enp0s9 $ ip mptcp limits show add_addr_accepted 2 subflows 2
Mmh, fullmesh alone, without subflow in this case (on the endpoints used for the additional subflows) doesn't make sense, it means:
- when looking at creating new subflows, the endpoint will not be picked (because
subflowis not set) - when an
ADD_ADDRis received, only the endpoints with thefullmeshwill be taken (if there is at least one, otherwise the source address is picked using the routing table).
So here, I would expect the kernel not to create any additional subflows, except if the server sends an ADD_ADDR (which is not the case, when looking below).
FYI, the fullmesh flag can make sense to be used alone (without subflow) if you want the address used by the initial subflow to create a subflow to each address received via ADD_ADDR from the other peer.
Maybe this update of the website could help better understanding how to use the PM works? https://github.com/multipath-tcp/mptcp.dev/pull/12 Do not hesitate to suggest modifications!
server side mptcp config
$ ip mptcp endpoint show 20.20.20.3 id 3 fullmesh dev enp0s9 $ ip mptcp limits show add_addr_accepted 0 subflows 2
fullmesh doesn't make sense here when used alone: nothing to create, and typically it is the server that is sending ADD_ADDR not the one receiving them. If you want the server to announce the endpoint, you need to use the signal flag.
The fullmesh flag is there to create more subflows, not to announce more addresses.
When I connect the client to the server it creates 2 subflows (one is the initial subflow)
To be honest, I'm not sure how you ended up with 2 subflows with what you described: even in the 'monitor' output, I don't see any received ANNOUNCED (or is it taken from the server side), so how can it know it can establish a subflow from 20.20.20.3 to 20.20.20.2?
Question: I delete the endpoint from the client side.
sudo ip mptcp endpoint delete id 3This removes the subflow from client and server (remove address command)
Now I added the endpoint again to the client.
sudo ip mptcp endpoint add 20.20.20.2 dev enp0s9 fullmeshBut the server won't signal its second interface address
20.20.20.3to create a new subflow. Is it the correct behaviour?
Was it even announcing the endpoint before? Can you try with the signal flag instead?
If I understand correctly, the server announces its address only once when the MPTCP connection is established and will not announce it again. Is that correct?
When a RM_ADDR, it should make the ID as "available" again, then when re-added, it should send the ADD_ADDR again I think. Can you try with the signal flag please?
Here is the output from ip mptcp monitor
$ sudo ip mptcp monitor [ CREATED] token=917c83e8 remid=0 locid=0 saddr4=10.10.10.3 daddr4=10.10.10.2 sport=5000 dport=47930 [ ESTABLISHED] token=917c83e8 remid=0 locid=0 saddr4=10.10.10.3 daddr4=10.10.10.2 sport=5000 dport=47930 [ SF_ESTABLISHED] token=917c83e8 remid=1 locid=1 saddr4=20.20.20.3 daddr4=20.20.20.2 sport=5000 dport=57783 backup=0 [ REMOVED] token=917c83e8 remid=1 [ SF_CLOSED] token=917c83e8 remid=1 locid=1 saddr4=20.20.20.3 daddr4=20.20.20.2 sport=5000 dport=57783 backup=0As you can see, after the
SF_CLOSEDthere is noSF_ESTABLISHED
Again, best to retry with the proper flags.
Also, I wonder why the
sscommand only shows the initial subflow. It never shows the second subflow.$ ss -M State Recv-Q Send-Q Local Address:Port Peer Address:Port ESTAB 0 0 10.10.10.3:5000 10.10.10.2:47930
-M will only list the MPTCP connections, so only the addresses used by the application, the ones of the initial subflow. (Note that -i will show more info, like the number of subflows, etc.).
If you want to see the subflows, you can use ss -ti as root.
For more details: https://www.mptcp.dev/debugging.html#ss
I have a server-client connected with two interfaces.
-----client---------------------server--- 10.10.10.2 <----------> 10.10.10.3 20.20.20.2 <----------> 20.20.20.3
client side mptcp config
server side mptcp config
When I connect the client to the server it creates 2 subflows (one is the initial subflow)
Question: I delete the endpoint from the client side.
This removes the subflow from client and server (remove address command)
Now I added the endpoint again to the client.
But the server won't signal its second interface address
20.20.20.3to create a new subflow. Is it the correct behaviour?If I understand correctly, the server announces its address only once when the MPTCP connection is established and will not announce it again. Is that correct?
Here is the output from ip mptcp monitor
As you can see, after the
SF_CLOSEDthere is noSF_ESTABLISHEDAlso, I wonder why the
sscommand only shows the initial subflow. It never shows the second subflow.