projectcalico / calico

Cloud native networking and network security
https://docs.tigera.io/calico/latest/about/
Apache License 2.0
6.02k stars 1.34k forks source link

Configuring KubeControllersConfiguration in a declarative way #5592

Open invidian opened 2 years ago

invidian commented 2 years ago

Expected Behavior

KubeControllersConfiguration/default can be configured declaratively with only non-default values.

Current Behavior

Right now, KubeControllersConfiguration/default resource is being created automatically once Calico is deployed, which makes user rely on either calico-kube-controllers deployment configuration (e.g. environment variables) or on kubectl patch or calicoctl patch after installing Calico.

Possible Solution

Include KubeControllersConfiguration/default in the release manifests and Helm chart with empty spec so user can customize it.

Others

As a workaround, is it OK to create the following KubeControllersConfiguration/default before calico-kube-controllers is running?

apiVersion: crd.projectcalico.org/v1
kind: KubeControllersConfiguration
metadata:
  name: default
spec:
  controllers:
    node:
      hostEndpoint:
        autoCreate: Enabled

The status values field remains the same after reconciliation, however, the spec created by the controller looks like following:

spec:
  controllers:
    namespace:
      reconcilerPeriod: 5m0s
    node:
      hostEndpoint:
        autoCreate: Disabled
      reconcilerPeriod: 5m0s
      syncLabels: Enabled
    policy:
      reconcilerPeriod: 5m0s
    serviceAccount:
      reconcilerPeriod: 5m0s
    workloadEndpoint:
      reconcilerPeriod: 5m0s
  etcdV3CompactionPeriod: 10m0s
  healthChecks: Enabled
  logSeverityScreen: Info
caseydavenport commented 2 years ago

Yep, agree this is a gap right now.

You can do this after the API server is running (e.g., kubectl replace -f config.yaml but we have a gap right now where it's hard to do this prior to the Calico API server running.

sebhoss commented 10 months ago

@caseydavenport would the following workflow be ok for new installations?

  1. apply calico-cni manifests
  2. deploy calico-apiserver
  3. apply (replace?) KubeControllersConfiguration
  4. apply network policies
caseydavenport commented 10 months ago

@sebhoss yes that should work

nullxx commented 1 month ago
kubectl apply --server-side --validate=false -f - <<EOF
apiVersion: projectcalico.org/v3
kind: KubeControllersConfiguration
metadata:
  name: default
spec:
  controllers:
    node:
      hostEndpoint:
        autoCreate: Enabled
EOF

Or just an ArgoCD template:

apiVersion: projectcalico.org/v3
kind: KubeControllersConfiguration
metadata:
  annotations:
    argocd.argoproj.io/sync-options: "ServerSideApply=true, Validate=false"
  name: default
spec:
  controllers:
    node:
      hostEndpoint:
        autoCreate: Enabled
caseydavenport commented 1 month ago

I'm obligated to discourage the use of crd.projectcalico.org/1 in favor of projectcalico.org/v3

More information here: https://github.com/projectcalico/calico/issues/6412

nullxx commented 1 month ago

Thank you @caseydavenport I have updated the previous comment