Default $arp_restricted=true breaks Calico overlay network

[eumel8]

Describe the bug Calico used proxy arp technique for routing in overlay network. A default setting of $arp_restricted=true, like introduced some days ago here and handled here will break communication in Kubernetes cluster with Calico CNI.

Expected behavior default variable in init.pp should be set to false

Actual behavior default value of $arp_restricted is true so net.ipv4.conf.all.arp_ignore is set to 2. Arp requests are ignored. Workaround

class { 'os_hardening':
   arp_restricted  => false,
}

OS / Environment Ubuntu 18.04

Puppet Version

# puppet --version
5.5.22

Additional context also handled here: https://github.com/kubernetes/kubernetes/issues/71555

[mcgege]

Hi @eumel8 , thanks for this information ... setting net.ipv4.conf.all.arp_ignore to 1 has been the default now for a long time, this change is setting it to 2 instead of 1. But the default for puppet-os-hardening was always >0 ...

I fear that if we change this default now to 0 we'll break much more - and setting this value is a security recommendation.

[schurzi]

I think having arp_ignore on 1 or 2 is one of the targets of this role, because we deal with os hardening. We should never set it to 0 by default, but provide the user a good solution to adjust this setting. That is currently possible.

From an internal viewpoint, our baseline (https://github.com/dev-sec/linux-baseline/blob/8ee448e3e220ad2d2e19db3e508ffb14c9550daf/controls/sysctl_spec.rb#L104) checks if this sysctl is set to 1 and the Ansible implementation also uses 1. I think we should strive to be somewhat feature equivalent in our implementations. So we should decide if we set the default to 2 in the baseline and subsequently in all implementations from us.

[mcgege]

I share your view, we should open a discussion on this value ... maybe I was a bit too fast implementing the defaults of my company here, sorry.

Independent of this the baseline should honor that both values are secure and used in the wild, therefore I pushed this PR :-)

[eumel8]

@mcgege indeed, arp_ignore = 1 was the default and it seems it worked so far on older installations. With a value of 2 it's definitly broken. I understand the security recommendation, from user perspective a global option like kubernetes_cluster = true would be nice. At the moment we have set in our cluster:

 class { 'os_hardening':
   umask  => '027',
   enable_ipv4_forwarding  => true,
   arp_restricted  => false,
 }

[mcgege]

@eumel8 I'd now revert to the old default of arp_ignore = 1 and introduce a new switch arp_ignore_samenet to enforce the more strict value 2 with #256

[mcgege]

New release 2.2.10 is out, please try

[eumel8]

@mcgege many thanks, works now as expected.

only arp_ignore_samenet => true breaks now calico CNI because net.ipv4.conf.all.arp_ignore = 2 instead 1