MPTCP Netlink API Question.

[pak-ji]

Test Environments

• client

  • has two NIC (192.168.1.9, 192.168.1.10)

• Server

  • has one NIC (192.168.1.11)
  • port 9999

Question

The client and server exchanged address id via CMD_ANNOUNCE in advance.
After that, The client proceed a CMD_SUB_CREATE.

The attributes of CMD_SUB_CREATE are set as follows:

• token : local token
• family : AF_INET
• loc_id : local id (unique id in mptcp session)
• rem_id : remote id (unique id in mptcp session)
• daddr4 : remote ipv4 address (192.168.1.11)
• dport : remote server port (9999)

Did I make a mistake in setting the attributes?

[matttbe]

Hi @pak-ji

Sorry, I'm not sure to get the issue.

When do you send these different commands? After having received some events?

What do you try to do exactly? Establishing a new subflow between 192.168.1.10 and 192.168.1.11? What do you see? Netlink error? Not the correct behaviour? Kernel error?

(BTW, probably not a good idea to have everything in the same subnet)

[pak-ji]

Hi @matttbe

I'm sorry. I didn't write down what I needed.

I didn't receive an error.
I couldn't see the SYN packet of MP_JOIN when running tcpdump.

Q) I want to know if the attributes set are correct. (MPTCP_CMD_SUB_CREATE)

token : local token family : AF_INET loc_id : local id (unique id in mptcp session) rem_id : remote id (unique id in mptcp session) daddr4 : remote ipv4 address (192.168.1.11) dport : remote server port (9999)
(loc_id of 192.168.1.10 is 1, rem_id of 192.168.1.11 is 10)

[VenkateswaranJ]

@pak-ji

You are missing source address and source port which are not optional. Please look at this issue https://github.com/intel/mptcpd/issues/134

[VenkateswaranJ]

I didn't receive an error. Are you sure ? How you are checking netlink error

[pak-ji]

@VenkateswaranJ

https://github.com/multipath-tcp/mptcp/blob/mptcp_trunk/include/uapi/linux/mptcp.h#L90-L92

 *   - MPTCP_CMD_SUB_CREATE: token, family, loc_id, rem_id, daddr4 | daddr6,
 *                           dport [, saddr4 | saddr6, sport, backup, if_idx]
 *       Create a new subflow.

Source address (ip) and source port are optional on this netlink api doc.

https://github.com/intel/mptcpd/blob/master/src/netlink_pm_mptcp_org.c#L316-#L330

static int mptcp_org_add_subflow()
{     
        /*
          Payload:
              Token
              Address family
              Local address ID
              Remote address ID
              Remote address
              Remote port

              Optional attributes:
                  Local address
                  Local port
                  Backup priority flag
                  Network interface index
         */
...
}

Also, source(local) address and source(local) port are also optional in "mptcpd".
Am I not understanding it well?

[VenkateswaranJ]

@pak-ji please check this issue https://github.com/intel/mptcpd/issues/134 It's a mistake I think.

[pak-ji]

I'll test more.

Thank you. :) @VenkateswaranJ
@matttbe

[pak-ji]

@VenkateswaranJ

I tried CMD_SUB_CREATE. But it failed. Error packets received after CMD_SUB_CREATE are :

struct nlmsgerr

60 00 00 00 
02 00 00 00 
3A 9F 03 61 
54 0E C0 92 

F5 FF FF FF  ERRNO : -11
4C 00 00 00 
16 00 05 00
3A 9F 03 61 
54 0E 00 00 
08 01 00 00 
08 00 01 00 TOKEN
18 89 2D 09 
06 00 02 00 FAMILY
02 00 00 00 
05 00 03 00 LOC_ID
04 00 00 00 
05 00 04 00 REM_ID
08 00 00 00 
08 00 05 00 SADDR4
C0 A8 01 0A 
08 00 07 00 DADDR4
79 B8 C2 10 
06 00 0A 00 DPORT
27 0F 00 00

The error number is "-11" in the packet.

Generic netlink, this is EAGAIN. (https://github.com/torvalds/linux/blob/master/include/uapi/asm-generic/errno-base.h) And Netlink, this is NOSUPPORT. (https://android.googlesource.com/platform/external/libnl-headers/+/refs/heads/ics-mr0/netlink/errno.h)

What should I understand as?

[pak-ji]

I found -EAGAIN(-11) on this link "https://github.com/multipath-tcp/mptcp/blob/mptcp_v0.95/net/mptcp/mptcp_netlink.c#L830-L838"

Does this error mean that EVENT_ESTABLISHED must be received to proceed with CMD_SUB_CREATE?

[matttbe]

@pak-ji please check this issue intel/mptcpd#134 It's a mistake I think.

The source address and port are optional. I fixed the doc a few weeks ago: https://github.com/multipath-tcp/mptcp/commit/1260f8d1a751bd04d657f132469d9448fba22eb1

Does this error mean that EVENT_ESTABLISHED must be received to proceed with CMD_SUB_CREATE?

Yes, that's correct, you need to wait for the connection to be in "fully established" before initiating new subflows. So yes, waiting for MPTCP_EVENT_ESTABLISHED is the proper way to wait before creating a new subflow as indicating in the "doc" :)

https://github.com/multipath-tcp/mptcp/blob/1c4915d06c6f5371a14d58f3d5e3661b31a36447/include/uapi/linux/mptcp.h#L56-L58

Please close the issue if your question has been replied.

[pak-ji]

@matttbe @VenkateswaranJ

Thanks to you, I solved it. Thank you. :)