XfrmStateAdd() returns "invalid argument"

[matanbroner]

Hello,

Within the context of setting up a 5G based IPSec tunnel, I am running XfrmStateAdd as follows:

xfrmState := new(netlink.XfrmState)

// Set all of the encr. and integ. algos up here, code removed for brevity

xfrmState.Src = childSecurityAssociation.PeerPublicIPAddr
xfrmState.Dst = childSecurityAssociation.LocalPublicIPAddr
xfrmState.Proto = netlink.XFRM_PROTO_ESP
xfrmState.Mode = netlink.XFRM_MODE_TUNNEL
xfrmState.Spi = int(childSecurityAssociation.SPI)
xfrmState.Mark = mark
xfrmState.Auth = xfrmIntegrityAlgorithm
xfrmState.Crypt = xfrmEncryptionAlgorithm
xfrmState.ESN = childSecurityAssociation.ESN

var err error
if err = netlink.XfrmStateAdd(xfrmState); err != nil {
return fmt.Errorf("set XFRM state rule failed 1: %+v", err)
}

I have checked that all attributes that are needed are indeed set correctly (I think?). This is the output of calling String() on my state object:

Dst: 192.168.56.102, Src: 192.168.56.101, Proto: esp, Mode: tunnel, SPI: 0x1, ReqID: 0x0, ReplayWindow: 0, Mark: (0x5,0x0), OutputMark: 0, Ifid: 0, Auth: {Name: hmac(sha1), Key: 0xa69d020aaf2a98a06c48323e9198749b9d20926a}, Crypt: {Name: cbc(aes), Key: 0x2ac985a2ddf0fba51dfd592f34e48b2fb76921ea4b70b391648e6d069b861865}, Aead: <nil>, Encap: <nil>, ESN: false

I can provide specifics as to the values of each of my state's attributes as needed. When I use strace to see what the actual system call looks like, I see this:

sendto(10, {{len=460, type=XFRM_MSG_NEWSA, flags=NLM_F_REQUEST|NLM_F_ACK|NLM_F_EXCL|NLM_F_CREATE, seq=1, pid=0}, "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"...}, 460, 0, {sa_family=AF_NETLINK, nl_pid=0, nl_groups=00000000}, 12) = 460

... removed for brevity

recvfrom(10,  <unfinished ...>
[pid  1491] futex(0xc000041548, FUTEX_WAKE_PRIVATE, 1 <unfinished ...>
[pid  1495] <... futex resumed>)        = 0
[pid  1492] <... recvfrom resumed>{{len=480, type=NLMSG_ERROR, flags=0, seq=1, pid=1490}, {error=-EINVAL, msg={{len=460, type=XFRM_MSG_NEWSA, flags=NLM_F_REQUEST|NLM_F_ACK|NLM_F_EXCL|NLM_F_CREATE, seq=1, pid=0}, "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"...}}}, 65536, 0, {sa_family=AF_NETLINK, nl_pid=0, nl_groups=00000000}, [112->12]) = 480

Am I mistaken that the send payload is all zeros? Am I doing something wrong or is this a kernel issue perhaps? Any feedback is greatly appreciated, thanks.

[vishvananda]

The zeros aren't concerning because the first chunk of values of the XfrmSa message is for the selector which is used when setting policy, not setting state. It has been a while since i used the xfrm stuff but i expect there is an invalid setting somewhere. Perhaps you have to set up the policy first before setting the state? I remember the xfrm stuff being tricky to set up properly. Here is how I did it: https://github.com/vishvananda/wormhole/blob/48b11afa3a84d31575f03ae755dc78724396df8e/server/tunnel.go#L520

[vishvananda]

For tunnels, IIRC you also have to create a userspace UDP listener to make it work, which is not documented well.

[matanbroner]

@vishvananda Thanks for the replies! I tried setting the policy first and it also returns an invalid argument error. I do set up a GRE tunnel for the user plane data coming into the CHILD_SA, does that need to happen before the XFRM rule is set up?

[rajprithiv88]

With the new kernel change, netlink was expecting if IF_ID is passed, it should not be passed as 0. With netlink go package, application always sends if_id as 0 if not specified and kernel netlink rejects it.

I can see the same commit is reverted later, and we should not see the issue in the latest version of kernel if reverted.

commit 68ac0f3810e76a853b5f7b90601a05c3048b8b54 Author: Antony Antony antony.antony@secunet.com Date: Sun Dec 12 11:35:00 2021 +0100

xfrm: state and policy should fail if XFRMA_IF_ID 0

xfrm ineterface does not allow xfrm if_id = 0 
fail to create or update xfrm state and policy.

With this commit:
 ip xfrm policy add src 192.0.2.1 dst 192.0.2.2 dir out if_id 0
 RTNETLINK answers: Invalid argument

 ip xfrm state add src 192.0.2.1 dst 192.0.2.2 proto esp spi 1 \ 
            reqid 1 mode tunnel aead 'rfc4106(gcm(aes))' \
            0x1111111111111111111111111111111111111111 96 if_id 0
 RTNETLINK answers: Invalid argument

commit a3d9001b4e287fc043e5539d03d71a32ab114bcb Author: Kai Lueke kailueke@linux.microsoft.com AuthorDate: Thu Mar 3 15:55:10 2022 +0100 Commit: Steffen Klassert steffen.klassert@secunet.com CommitDate: Sun Mar 6 08:38:28 2022 +0100

Revert "xfrm: state and policy should fail if XFRMA_IF_ID 0"

This reverts commit 68ac0f3810e76a853b5f7b90601a05c3048b8b54 because ID
0 was meant to be used for configuring the policy/state without
matching for a specific interface (e.g., Cilium is affected, see
https://github.com/cilium/cilium/pull/18789 and
https://github.com/cilium/cilium/pull/19019).