agapoff / check_kubernetes

Nagios/Icinga/Zabbix style plugin for checking Kubernetes
64 stars 36 forks source link

Problem with creating service accounts #31

Closed lisek84 closed 1 year ago

lisek84 commented 1 year ago

Since new versions of k8s are not generating secret tokens by default. Applying YML file provided here does nothing. I belive that documentation should be updated.

Why it's not working? I belive it's described here: https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.24.md#no-really-you-must-read-this-before-you-upgrade

https://kubernetes.io/docs/concepts/configuration/secret/#service-account-token-secrets

So currently applying account.yml file creates all ok, but there is no secret inside and there is no way to get token from that like described in documentation.

agapoff commented 1 year ago

Thank you for pointing that out. I will eventually upgrade K8s to the new version, face the issue and come up with the solution.

lisek84 commented 1 year ago

I did it that way:

kubectl create token monitoring -n monitoring --duration=87600h

And after that, prepared YAML with;

apiVersion: v1 kind: Secret metadata: name: monitoring namespace: monitoring annotations: kubernetes.io/service-account.name: "monitoring" type: kubernetes.io/service-account-token

and applied that file. It's working, but tls it's not working since this user have to access to kube-system. Rest works.

hydrapolic commented 1 year ago

In case you're using terraform:

resource "kubernetes_namespace_v1" "monitoring" {
  metadata {
    annotations = {
      name = "monitoring"
    }

    name = "monitoring"
  }
}

resource "kubernetes_service_account_v1" "monitoring" {
  metadata {
    name      = "monitoring"
    namespace = kubernetes_namespace_v1.monitoring.metadata.0.name
  }

  secret {
    name = "monitoring"
  }
}

resource "kubernetes_secret_v1" "monitoring" {
  depends_on = [
    kubernetes_service_account_v1.monitoring
  ]

  metadata {
    name      = "monitoring"
    namespace = kubernetes_namespace_v1.monitoring.metadata.0.name

    annotations = {
      "kubernetes.io/service-account.name"      = "monitoring"
      "kubernetes.io/service-account.namespace" = "monitoring"
    }
  }

  type = "kubernetes.io/service-account-token"
}

resource "kubernetes_cluster_role_v1" "monitoring" {
  metadata {
    name = "monitoring"
  }

  rule {
    api_groups = [""]
    resources  = ["pods", "nodes", "secrets", "persistentvolumes"]
    verbs      = ["get", "list"]
  }

  rule {
    api_groups = ["extensions", "apps"]
    resources  = ["deployments", "replicasets", "daemonsets", "statefulsets"]
    verbs      = ["get", "list"]
  }

  rule {
    api_groups = ["batch"]
    resources  = ["jobs"]
    verbs      = ["get", "list"]
  }
}

resource "kubernetes_cluster_role_binding_v1" "monitoring" {
  metadata {
    name = "monitoring"
  }
  role_ref {
    api_group = "rbac.authorization.k8s.io"
    kind      = "ClusterRole"
    name      = "monitoring"
  }
  subject {
    kind      = "ServiceAccount"
    name      = "monitoring"
    namespace = kubernetes_namespace_v1.monitoring.metadata.0.name
  }
}
agapoff commented 1 year ago

Thank you for your help. I've updated the account.yaml file and README. I've also added terraform config to the repo (might be useful).