clastix / kamaji

Kamaji is the Hosted Control Plane Manager for Kubernetes.
https://kamaji.clastix.io
Apache License 2.0
1.14k stars 104 forks source link

document issue to get kubeconfig #590

Closed sonyafenge closed 1 month ago

sonyafenge commented 1 month ago

This documents mentioned how to get kubeconfig

kubectl get secrets -n ${TENANT_NAMESPACE} ${TENANT_NAME}-admin-kubeconfig -o json \
  | jq -r '.data["admin.conf"]' \
  | base64 --decode \
  > ${TENANT_NAMESPACE}-${TENANT_NAME}.kubeconfig

Followed instructions above to get kubeconfig and get error for any resource:

Error from server (Forbidden): nodes is forbidden: User "kubernetes-admin" cannot list resource "nodes" in API group "" at the cluster scope

Repro Steps:

  1. using EKS to create management cluster
  2. install capa, cluster-api-control-plane-provider-kamaji
  3. generate kamaji cluster
  4. get kubeconfig by
    clusterctl get kubeconfig kamaji-tc5 > kamaji-tc5-kubeconfig.config
  5. or get kubeconfig by
    kubectl get secrets kamaji-tc5-admin-kubeconfig -o json \
    | jq -r '.data["admin.conf"]' \
    | base64 --decode \
    > kamaji-tc5.kubeconfig
  6. use kubeconfig, get error.

Fix suggetions: checked secret and found their is another super-admin.conf, suggest to change to:

kubectl get secrets -n ${TENANT_NAMESPACE} ${TENANT_NAME}-admin-kubeconfig -o json \
  | jq -r '.data["super-admin.conf"]' \
  | base64 --decode \
  > ${TENANT_NAMESPACE}-${TENANT_NAME}.kubeconfig
prometherion commented 1 month ago

Hey, thanks for the report, however, this is not a bug per se.

We have a set of internal controllers that install the required RBAC for the kubernetes-admin user, if these are not running it's a symptom something is broken with your Kamaji installation.

The super-admin user works since it belongs to the user group system:masters which bypass entirely the RBAC in Kubernetes, and should be only used by external operators and not users.

Please, check the logs of Kamaji since it seems the Operator is not able to connect to the Tenant Control Planes to interact with it.

prometherion commented 1 month ago

Closing due to lack of feedback.