search

vilisseranen / terraform-kubeadm

Deploy a Kubernetes cluster with Terraform and kubeadm on cloud.ca
4 stars 2 forks source link

Dashboard and kube-dns not being created #4

Open syed opened 6 years ago

syed commented 6 years ago

After creating the cluster, I see that kube-dns and kubernetes-dashboard are still in ContainerCreating state. 

[kubernetes@kata-gvisor-bastion01 ~]$ kubectl get pods --all-namespaces
NAMESPACE     NAME                                           READY     STATUS              RESTARTS   AGE
kube-system   etcd-kata-gvisor-master01                      1/1       Running             0          21h
kube-system   kube-apiserver-kata-gvisor-master01            1/1       Running             0          21h
kube-system   kube-controller-manager-kata-gvisor-master01   1/1       Running             0          21h
kube-system   kube-dns-86f4d74b45-6k5lw                      0/3       ContainerCreating   0          21h
kube-system   kube-proxy-8szr2                               1/1       Running             0          21h
kube-system   kube-proxy-gv46r                               1/1       Running             0          21h
kube-system   kube-scheduler-kata-gvisor-master01            1/1       Running             0          21h
kube-system   kubernetes-dashboard-7d5dcdb6d9-k5g8z          0/1       ContainerCreating   0          21h
kube-system   weave-net-bkzqm                                2/2       Running             0          21h
kube-system   weave-net-tq7n9                                2/2       Running             0          21h
[kubernetes@kata-gvisor-bastion01 ~]$

Haven't debugged this but have you seen this before @vilisseranen ?

vilisseranen commented 6 years ago

Yes, I have seen that before.

I believe that happens when the network overlay is created too early on the master. That's why there is this ugly sleep 60 in the script to bootstrap the master (https://github.com/vilisseranen/terraform-kubeadm/blob/master/templates/bootstrap_master.sh.tpl#L18).

If you know a legitimate condition on which we can wait on, I would gladly accept to change this script.

syed commented 6 years ago

Any workarounds to this? Can I fix my cluster after this has happened?

vilisseranen commented 6 years ago

Increasing the timeout should solve the issue. You can fix your cluster by killing the DNS pods, they will get recreated shortly after.

On Thu, Jun 7, 2018, 17:08 Syed Mushtaq Ahmed, notifications@github.com wrote:

Any workarounds to this? Can I fix my cluster after this has happened?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/vilisseranen/terraform-kubeadm/issues/4#issuecomment-395565617, or mute the thread https://github.com/notifications/unsubscribe-auth/AD5z-PeRvSnLIQ9FJEje8q93IfRsrYUAks5t6ZZDgaJpZM4Uba6i .