Open pcgeek86 opened 2 years ago
Hi @wansir , could you please help to take a look at this issue?
@pcgeek86 It seems this issue https://medium.com/@denisstortisilva/kubernetes-eks-calico-and-custom-admission-webhooks-a2956b49bd0d
Is the solution to delete Calico from cluster? @wansir
If not, what steps should I take to fix it?
So far I have:
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 7m44s default-scheduler Successfully assigned kubesphere-system/ks-installer-85dcfff87d-hwqd4 to ip-192-168-80-59.us-west-2.compute.internal
Warning FailedCreatePodSandBox 7m43s kubelet Failed to create pod sandbox: rpc error: code = Unknown desc = [failed to set up sandbox container "1b0845610c98e66285fdacf7293c4f9382b30d730d3f43a0fb96ecaa30370209" network for pod "ks-installer-85dcfff87d-hwqd4": networkPlugin cni failed to set up pod "ks-installer-85dcfff87d-hwqd4_kubesphere-system" network: error getting ClusterInformation: connection is unauthorized: Unauthorized, failed to clean up sandbox container "1b0845610c98e66285fdacf7293c4f9382b30d730d3f43a0fb96ecaa30370209" network for pod "ks-installer-85dcfff87d-hwqd4": networkPlugin cni failed to teardown pod "ks-installer-85dcfff87d-hwqd4_kubesphere-system" network: error getting ClusterInformation: connection is unauthorized: Unauthorized]
Normal SandboxChanged 2m33s (x25 over 7m43s) kubelet Pod sandbox changed, it will be killed and re-created.
So far I have:
- Removed Calico by manifests
- Removed KubeSphere by manifests
- Re-deployed KubeSphere by manifests
- Now the installer for KubeSphere will not start at all. The installer pod is hung.
Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 7m44s default-scheduler Successfully assigned kubesphere-system/ks-installer-85dcfff87d-hwqd4 to ip-192-168-80-59.us-west-2.compute.internal Warning FailedCreatePodSandBox 7m43s kubelet Failed to create pod sandbox: rpc error: code = Unknown desc = [failed to set up sandbox container "1b0845610c98e66285fdacf7293c4f9382b30d730d3f43a0fb96ecaa30370209" network for pod "ks-installer-85dcfff87d-hwqd4": networkPlugin cni failed to set up pod "ks-installer-85dcfff87d-hwqd4_kubesphere-system" network: error getting ClusterInformation: connection is unauthorized: Unauthorized, failed to clean up sandbox container "1b0845610c98e66285fdacf7293c4f9382b30d730d3f43a0fb96ecaa30370209" network for pod "ks-installer-85dcfff87d-hwqd4": networkPlugin cni failed to teardown pod "ks-installer-85dcfff87d-hwqd4_kubesphere-system" network: error getting ClusterInformation: connection is unauthorized: Unauthorized] Normal SandboxChanged 2m33s (x25 over 7m43s) kubelet Pod sandbox changed, it will be killed and re-created.
I will verify this issue later. BTW, I think it is limited by the cluster network. You can recreate an EKS cluster and use AWS VPC CNI instead
Same issue meat here, @wansir any thoughts?
is there any workaround support EKS with Calico network?
still not work change [ks-controller-manager] pod hostNetwork: true
Error log:
W0927 23:22:18.495218 1 client_config.go:615] Neither --kubeconfig nor --master was specified. Using the inClusterConfig. This might not work. I0927 23:22:18.496499 1 server.go:202] setting up manager I0927 23:22:18.520026 1 deleg.go:130] controller-runtime/metrics "msg"="metrics server is starting to listen" "addr"=":8080" E0927 23:22:18.520266 1 deleg.go:144] controller-runtime/metrics "msg"="metrics server failed to listen. You may want to disable the metrics server or use another port if it is due to conflicts" "error"="error listening on :8080: listen tcp :8080: bind: address already in use" F0927 23:22:18.520279 1 server.go:207] unable to set up overall controller manager: error listening on :8080: listen tcp :8080: bind: address already in use
Progress update:
simply set use hostNetwork: true won't work, because node-local-dns already occupied this 8080 port. Stuck here again...
[Pull request](url https://github.com/kubesphere/kubesphere/pull/5255) still need manually set hostNetwork: true manually enable control panel communicate with ks-controller-manager.
issue fixed with
General remarks
Describe the bug
eksctl
.ks-console
service type toLoadBalancer
.Versions used(KubeSphere/Kubernetes) KubeSphere: see below Kubernetes: 1.21.5 EKS
Environment How many nodes and their hardware configuration:
For example: EKS master 2 nodes: 8cpu/16g
(and other info are welcomed to help us debugging)
To Reproduce
See above.
Expected behavior
I should be able to change password or skip changing password, so I can login to KubeSphere Console.