enable Gratuitous ARP for SR-IOV device

[zshi-redhat]

The issue is to request adding GARP support in sriov-cni

What would you like to be added?

GARP packet is sent by kernel when /proc/sys/net/ipv4/conf//arp_notify is set to 1.

03:33:54.410993 00:11:22:33:44:66 > Broadcast, ethertype ARP (0x0806), length 60: Request who-has 10.56.217.175 tell 10.56.217.175, length 46

What is the use case for this feature / enhancement?

IP address allocated to VF may be re-used while SR-IOV pods are restarted/recreated in a cluster. It is desired to broadcast GARP packet while IP assignment is changed for a particular VF MAC.

[martinkennelly]

@zshi-redhat So this wouldn't be possible with tuning CNI chained because we don't know the interface name in advance?

{
  "name": "mytuning",
  "type": "tuning",
  "sysctl": {
          "net.ipv4.conf.${INTERFACE_NAME}.arp_notify": "1"
  }
}

So, a potential SRIOV CNI NAD spec would look like below?:

{
  "cniVersion": "0.3.1",
  "name": "sriov-advanced",
  "type": "sriov",
  "trust": "on",
  "garp": "on",
  ...  
}

[zshi-redhat]

@martinkennelly I just got to know that it's not necessary to specify the interface name in CNI net-attach-def config, chained CNI like tuning CNI inherits the CNI result passed down from previous CNI which contains the args.IfName.

I need to check if it's possible to use tuning CNI for arp notificuation. What I noticed is that tuning CNI implements its own logic for sending arp notification, which relies on specifying a MAC change in the same tuning plugin. but in sriov case, we use sriov-cni to change the MAC address. Another thing is kernel/driver has support for generating garp packet for VF devices, while current implementation in tuning CNI doesn't make use of it. Also kernel/driver requires the device link be restarted in order to send the garp packet, not sure if tunning CNI would restart the link (down & up).

[adrianchiris]

Question: shouldn't we send GARP by default on CmdAdd ?

[zshi-redhat]

Question: shouldn't we send GARP by default on CmdAdd ?

Nope with current sriov-cni implementation.

Two changes are required in sriov-cni to be able to send GARP: 1) Enable arp_notify for the VF device 2) Restart VF link (down & up) while IP has been assigned to VF Both the two are not met in sriov-cni.

[adrianchiris]

@zshi-redhat im aware we are not doing that in SR-IOV CNI today, My question is, should we do that by default ? (say if IPAM is specified)

[zshi-redhat]

@zshi-redhat im aware we are not doing that in SR-IOV CNI today, My question is, should we do that by default ? (say if IPAM is specified)

I think so.

[adrianchiris]

I saw mentions of using tuning plugin for that hence my previous question.

My 2 cent is that we should not rely on chained plugins for correct operation of SRIOV cni. It seems that sending GARP is a missing step in CmdAdd call

[zshi-redhat]

I'm working on a PR to add garp support in sriov-cni.

[TothFerenc]

For IPv6: net.ipv6.conf.<interface>.ndisc_notify=1 Most probably it is enabled by default, but better would be to set it for sure.