digitalocean / DOKS

Managed Kubernetes designed for simple and cost effective container orchestration.
https://www.digitalocean.com/products/kubernetes/
Apache License 2.0
80 stars 4 forks source link

kube-state-metrics serviceaccount installed to kube-system namespace by default #25

Closed devurandom closed 2 years ago

devurandom commented 3 years ago

It appears that in a fresh DigitalOcean Kubernetes cluster the kube-system namespace already contains a kube-state-metrics serviceaccount:

apiVersion: v1
kind: ServiceAccount
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"v1","imagePullSecrets":[],"kind":"ServiceAccount","metadata":{"annotations":{},"name":"kube-state-metrics","namespace":"kube-system"}}
  creationTimestamp: "2021-04-24T16:33:13Z"
  managedFields:
  - apiVersion: v1
    fieldsType: FieldsV1
    fieldsV1:
      f:secrets:
        .: {}
        k:{"name":"kube-state-metrics-token-766fc"}:
          .: {}
          f:name: {}
    manager: kube-controller-manager
    operation: Update
    time: "2021-04-24T16:33:13Z"
  - apiVersion: v1
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          .: {}
          f:kubectl.kubernetes.io/last-applied-configuration: {}
    manager: kubectl-client-side-apply
    operation: Update
    time: "2021-04-24T16:33:13Z"
  name: kube-state-metrics
  namespace: kube-system
  resourceVersion: "288"
  uid: 1e228525-a73b-4a93-83d2-09c06ca719b8
secrets:
- name: kube-state-metrics-token-766fc

This is even though kube-state-metrics was not installed (and is not running in any namespace).

The cluster is at Kubernetes version 1.20.2-do.0 and was created using the Terraform DigitalOcean provider version 2.7.0.

This trips the kube-state-metrics Helm chart version 2.13.2, which also tries to create this serviceaccount:

Error: rendered manifests contain a resource that already exists. Unable to continue with install: ServiceAccount "kube-state-metrics" in namespace "kube-system" exists and cannot be imported into the current release: invalid ownership metadata; label validation error: missing key "app.kubernetes.io/managed-by": must be set to "Helm"; annotation validation error: missing key "meta.helm.sh/release-name": must be set to "kube-state-metrics"; annotation validation error: missing key "meta.helm.sh/release-namespace": must be set to "kube-system"

In a cluster created a long time ago with Kubernetes 1.18 and using a different version of the kube-state-metrics Helm chart this was not the case.

Is it possible to not create this serviceaccount automatically?

devurandom commented 3 years ago

Not only a service account exists already, but also the clusterrole and clusterrolebinding, which both trip up the Helm chart.

ianmuscat commented 2 years ago

Using the following Helm chart values worked around the issue for me.

serviceAccount:
  create: false
  name: kube-state-metrics
rbac:
  create: false
timoreimann commented 2 years ago

Apologies for the late note, we fixed this in 1.21.2-do.0 and 1.20.8-do.0, respectively. We do not delete the prior policy files automatically, however, so affected customers will need to delete/overwrite them on their own.