search

platform9 / cctl

Apache License 2.0
47 stars 8 forks source link

Check if interface set in `iface` option exists #137

Open vannrt opened 5 years ago

vannrt commented 5 years ago

If a network interface name is set and it doesn't exist, cctl will hang trying to create the Kubernetes cluster (actually ssh-provider will hang because nodeadm hangs waiting for kubelet to converge).

ojmhetar commented 5 years ago

Root cause is that keepalived is in an active but not operational state and kubeadm will wait for the control plane to be up using the API endpoint (vip).

Mon Jan 14 23:38:06 2019: Starting VRRP child process, pid=7 Mon Jan 14 23:38:06 2019: Registering Kernel netlink reflector Mon Jan 14 23:38:06 2019: Registering Kernel netlink command channel Mon Jan 14 23:38:06 2019: Opening file '/usr/local/etc/keepalived/keepalived.conf'. Mon Jan 14 23:38:06 2019: WARNING - interface ens3 for vrrp_instance K8S_APISERVER doesn't exist Mon Jan 14 23:38:06 2019: Non-existent interface specified in configuration Mon Jan 14 23:38:06 2019: Stopped Mon Jan 14 23:38:06 2019: VRRP child process(7) died: Respawning

Keepalived will not fail fast upon receiving an "invalid" configuration. (It is possible that the interface may come up later). A timeout for nodeadm init could prevent indefinite hangs in the future.

Add the interface check to nodeadm might be preferred, so as to not run into the same issue when running init manually on the node.