search

kubernetes / kops

Kubernetes Operations (kOps) - Production Grade k8s Installation, Upgrades and Management
https://kops.sigs.k8s.io/
Apache License 2.0
15.97k stars 4.65k forks source link

Give each kops-controller controller unique names #16812

Closed rifelpet closed 2 months ago

rifelpet commented 2 months ago

This makes progress on fixing ipv6 support.

For IPV6 clusters we enable an ipam controller in kops-controller:

https://github.com/kubernetes/kops/blob/423b640de6cd18c99fea5c43f938e029a8d8aad6/pkg/apis/kops/cluster.go#L949-L951

https://github.com/kubernetes/kops/blob/423b640de6cd18c99fea5c43f938e029a8d8aad6/upup/pkg/fi/cloudup/template_functions.go#L774-L776

https://github.com/kubernetes/kops/blob/423b640de6cd18c99fea5c43f938e029a8d8aad6/cmd/kops-controller/main.go#L221-L227

controller-runtime's manager picks "names" for each controller it manages, and defaults to the kind it watches:

https://github.com/kubernetes-sigs/controller-runtime/blob/38546806f2faf5973e3321a7bd5bb3afdbb5767d/pkg/builder/controller.go#L247-L257

This means that alongside the node bootstrap controller, there are multiple controllers watching nodes. This results in a conflict which causes kops-controller to crash loop:

I0906 00:17:26.074975       1 main.go:373] "msg"="enabling IPAM controller" "logger"="setup"
I0906 00:17:26.075026       1 awsipam.go:47] Starting aws ipam controller
E0906 00:17:26.077961       1 main.go:230] "msg"="unable to create controller" "error"="controller with name node already exists. Controller names must be unique to avoid multiple controllers reporting to the same metric" "controller"="NodeController" "logger"="setup"

This PR gives each controller a unique name to avoid the conflict.

justinsb commented 2 months ago

/approve /lgtm

(We discussed in office hours, this uniqueness requirement was introduced in https://github.com/kubernetes-sigs/controller-runtime/commit/2b941650bce159006c88bd3ca0d132c7bc40e947 )

k8s-ci-robot commented 2 months ago

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: justinsb

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files: - ~~[OWNERS](https://github.com/kubernetes/kops/blob/master/OWNERS)~~ [justinsb] Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment