search

kubernetes-sigs / kubespray

Deploy a Production Ready Kubernetes Cluster
Apache License 2.0
16.15k stars 6.47k forks source link

User "system:anonymous" cannot get resource "configmaps" in API group "" in the namespace "kube-public" #9890

Closed Huskydog9988 closed 9 months ago

Huskydog9988 commented 1 year ago

Environment:

Linux 5.15.0-67-generic x86_64
PRETTY_NAME="Ubuntu 22.04.2 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.2 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy

Kubespray version (commit) (git rev-parse --short HEAD): 2ae3ea9ee

Network plugin used: cilium

Full inventory with variables (ansible -i inventory/sample/inventory.ini all -m debug -a "var=hostvars[inventory_hostname]"):

https://gist.github.com/Huskydog9988/19cedb17c3c416db98cf908779c07da0 (Don't worry about secrets this is just a test cluster.)

Command used to invoke ansible: ansible-playbook -i /inventory/inventory.ini --become --become-user=root --ask-become-pass -e "@/hardening.yaml" cluster.yml

Output of ansible run:

TASK [kubernetes/control-plane : Joining control plane node to the cluster.] **************************************************************************************************
fatal: [node2]: FAILED! => {"attempts": 3, "changed": true, "cmd": ["/usr/local/bin/kubeadm", "join", "--config", "/etc/kubernetes/kubeadm-controlplane.yaml", "--ignore-preflight-errors=all", "--skip-phases="], "delta": "0:05:00.116433", "end": "2023-03-13 22:38:22.951670", "msg": "non-zero return code", "rc": 1, "start": "2023-03-13 22:33:22.835237", "stderr": "\t[WARNING FileExisting-ethtool]: ethtool not found in system path\nerror execution phase preflight: couldn't validate the identity of the API Server: configmaps \"cluster-info\" is forbidden: User \"system:anonymous\" cannot get resource \"configmaps\" in API group \"\" in the namespace \"kube-public\"\nTo see the stack trace of this error execute with --v=5 or higher", "stderr_lines": ["\t[WARNING FileExisting-ethtool]: ethtool not found in system path", "error execution phase preflight: couldn't validate the identity of the API Server: configmaps \"cluster-info\" is forbidden: User \"system:anonymous\" cannot get resource \"configmaps\" in API group \"\" in the namespace \"kube-public\"", "To see the stack trace of this error execute with --v=5 or higher"], "stdout": "[preflight] Running pre-flight checks", "stdout_lines": ["[preflight] Running pre-flight checks"]}

Anything else do we need to know:

I suspect this is a perms issue with anon auth, and the provided hardened config but I'm not sure what the exact cause is. (#9474 seems to be a similar issue.)

neolit123 commented 1 year ago

I suspect this is a perms issue with anon auth

joining nodes with bootstrap tokens (kubeadm default) requires anon auth. alternatively a kubeconfig with certs can be used, but not sure kubespray supports that.

https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm-join/#file-or-https-based-discovery

note: CA validation should be disabled too. check the above join doc for more info.

Huskydog9988 commented 1 year ago

I'm fairly new to k8s, so I think I need this spelled out for me. The solution your suggesting is that I used certs to connect the nodes instead of the tokens kubespray is using now? If so, could you point me to where I might find the necessary config values?

neolit123 commented 1 year ago

If so, could you point me to where I might find the necessary config values?

unclear to me if this is possible. i will leave it to the kubespray maintainers to respond.

MrFreezeex commented 1 year ago

AFAIK you should be getting this error when you disable kube_api_anonymous_auth but it doesn't appear to be the case here? Did you set this to false in a previous run maybe? If so I would recommend that you run a reset.yml if you can trash the cluster if not maybe you can try to run the upgrade cluster playbook.

joining nodes with bootstrap tokens (kubeadm default) requires anon auth. alternatively a kubeconfig with certs can be used, but not sure kubespray supports that.

https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm-join/#file-or-https-based-discovery

note: CA validation should be disabled too. check the above join doc for more info.

Yeah last time I checked anon auth disabled was broken in Kubespray

Huskydog9988 commented 1 year ago

I most certainty did try it with kube_api_anonymous_auth disabled, but I think I ran the reset playbook after. I'll try the upgrade playbook first when I get chance, I'll let you know if either doesn't resolve the issue.

k8s-triage-robot commented 1 year ago

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

k8s-triage-robot commented 1 year ago

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

dhruvik7 commented 1 year ago

Was anyone able to get this work? I only run into this on 22.04

volcano1111 commented 1 year ago

Not sure why it is closed, the issue is still here. I ran into this on Debian 12 with kube_api_anonymous_auth: true. I ran upgrade-cluster.yml playbook and it didn't help. reset.yml did help tho.

boberbeck commented 12 months ago

Hey, is there a solution here yet? I ran Reset.yml but unfortunately that didn't help.

k8s-triage-robot commented 9 months ago

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues according to the following rules:

You can:

Please send feedback to sig-contributor-experience at kubernetes/community.

/close not-planned

k8s-ci-robot commented 9 months ago

@k8s-triage-robot: Closing this issue, marking it as "Not Planned".

In response to [this](https://github.com/kubernetes-sigs/kubespray/issues/9890#issuecomment-1902022438): >The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs. > >This bot triages issues according to the following rules: >- After 90d of inactivity, `lifecycle/stale` is applied >- After 30d of inactivity since `lifecycle/stale` was applied, `lifecycle/rotten` is applied >- After 30d of inactivity since `lifecycle/rotten` was applied, the issue is closed > >You can: >- Reopen this issue with `/reopen` >- Mark this issue as fresh with `/remove-lifecycle rotten` >- Offer to help out with [Issue Triage][1] > >Please send feedback to sig-contributor-experience at [kubernetes/community](https://github.com/kubernetes/community). > >/close not-planned > >[1]: https://www.kubernetes.dev/docs/guide/issue-triage/ Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.