Open yoplait opened 2 years ago
Basically this doesn't work anymore with the status on the hcloud-cloud-controller-manager:
| => KUBECONFIG=secrets/admin.conf kubectl describe nodes | egrep "Taints:|Name:"
Name: k8s-helsinki-master-1
Taints: node-role.kubernetes.io/master:NoSchedule
Warning FailedToCreateRoute 55m route_controller Could not create route 10893038-cf0b-4461-ac46-40d70e587670 10.244.0.0/24 for node k8s-helsinki-master-1 after 235.061594ms: hcloud/CreateRoute: hcops/AllServersCache.ByName: k8s-helsinki-master-1 hcops/AllServersCache.getCache: not found
All nodes are failing because of the network problem ....
KUBECONFIG=secrets/admin.conf kubectl get events -A -w --sort-by='{.lastTimestamp}'
default 3m55s Warning SyncLoadBalancerFailed service/tmp-web Error syncing load balancer: failed to ensure load balancer: hcloud/loadBalancers.EnsureLoadBalancer: hcops/LoadBalancerOps.ReconcileHCLBTargets: hcops/providerIDToServerID: missing prefix hcloud://:
default 66s Warning FailedToCreateRoute node/k8s-helsinki-node-1 (combined from similar events): Could not create route 2eea7cc6-31c2-4cc1-88f9-e893f841222f 10.244.3.0/24 for node k8s-helsinki-node-1 after 728.214426ms: hcloud/CreateRoute: hcops/AllServersCache.ByName: k8s-helsinki-node-1 hcops/AllServersCache.getCache: not found
default 66s Warning FailedToCreateRoute node/k8s-helsinki-master-3 (combined from similar events): Could not create route a4508fa8-3ad1-4a64-be11-c182135f663d 10.244.2.0/24 for node k8s-helsinki-master-3 after 470.881203ms: hcloud/CreateRoute: hcops/AllServersCache.ByName: k8s-helsinki-master-3 hcops/AllServersCache.getCache: not found
default 66s Warning FailedToCreateRoute node/k8s-helsinki-master-2 (combined from similar events): Could not create route 7c7e89ff-3721-470e-844f-551691779032 10.244.1.0/24 for node k8s-helsinki-master-2 after 249.027669ms: hcloud/CreateRoute: hcops/AllServersCache.ByName: k8s-helsinki-master-2 hcops/AllServersCache.getCache: not found
default 55s Warning FailedToCreateRoute node/k8s-helsinki-master-1 (combined from similar events): Could not create route 10893038-cf0b-4461-ac46-40d70e587670 10.244.0.0/24 for node k8s-helsinki-master-1 after 970.318239ms: hcloud/CreateRoute: hcops/AllServersCache.ByName: k8s-helsinki-master-1 hcops/AllServersCache.getCache: not found
default 55s Warning FailedToCreateRoute node/k8s-helsinki-node-2 (combined from similar events): Could not create route 43207003-9d37-4ba8-b7db-ccb3f6bfab61 10.244.4.0/24 for node k8s-helsinki-node-2 after 764.871302ms: hcloud/CreateRoute: hcops/AllServersCache.ByName: k8s-helsinki-node-2 hcops/AllServersCache.getCache: not found
default 42s Warning FailedScheduling pod/tmp-web-5cf74bc8c8-jjrpv 0/5 nodes are available: 2 node(s) had taint {node.kubernetes.io/network-unavailable: }, that the pod didn't tolerate, 3 node(s) had taint {node-role.kubernetes.io/master: }, that the pod didn't tolerate.
default 12s Warning FailedScheduling pod/tmp-web-5cf74bc8c8-k8j7n 0/5 nodes are available: 2 node(s) had taint {node.kubernetes.io/network-unavailable: }, that the pod didn't tolerate, 3 node(s) had taint {node-role.kubernetes.io/master: }, that the pod didn't tolerate.
Hi team, looks like there are some issues with the scripts:
I am trying to deploy 3 masters, 2 workers clusters and looks like some issue is happening with CNI and CoreDNS pods:
Any ideas?
Looks like following some links: coredns-pod-is-not-running-in-kubernetes?