Service account cannot list resource "mutatingwebhookconfigurations" after installing in `karpenter` namespace

[denniszag]

Description

Observed Behavior:

{"level":"INFO","time":"2024-08-21T11:51:07.719Z","logger":"controller","message":"k8s.io/client-go@v0.30.1/tools/cache/reflector.go:232: failed to list *v1.MutatingWebhookConfiguration: mutatingwebhookconfigurations.admissionregistration.k8s.io is forbidden: User \"system:serviceaccount:karpenter:karpenter\" cannot list resource \"mutatingwebhookconfigurations\" in API group \"admissionregistration.k8s.io\" at the cluster scope","commit":"490ef94"}

{"level":"ERROR","time":"2024-08-21T11:51:07.719Z","logger":"controller","message":"k8s.io/client-go@v0.30.1/tools/cache/reflector.go:232: Failed to watch *v1.MutatingWebhookConfiguration: failed to list *v1.MutatingWebhookConfiguration: mutatingwebhookconfigurations.admissionregistration.k8s.io is forbidden: User \"system:serviceaccount:karpenter:karpenter\" cannot list resource \"mutatingwebhookconfigurations\" in API group \"admissionregistration.k8s.io\" at the cluster scope","commit":"490ef94"}
{"level":"ERROR","time":"2024-08-21T11:51:07.926Z","logger":"webhook","message":"http: TLS handshake error from 172.33.20.216:47284: tls: no certificates configured\n","commit":"490ef94"}

Karpenter is installed in karpenter namespace, not kube-system. Expected Behavior: The role should have those actions.

Reproduction Steps (Please include YAML): Default values with the following overrides:

settings:
  clusterName: XXXX
  featureGates:
    drift: true
    spotToSpotConsolidation: true

Versions:

• Chart Version: v1.0.0

• Kubernetes Version (kubectl version): v1.30.2-eks-db838b0

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request

• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request

• If you are interested in working on this issue or have submitted a pull request, please leave a comment

[jmdeal]

Based on the commit in the log messages, you're running Karpenter v0.37.0, not v1.0.0. How did you go about installing the chart? If you referenced the chart in the repo at the tag, see #5415. The OCI repo should be used as a source for the chart.