search

alta3 / kubernetes-the-alta3-way

The greatest k8s installer on the planet!
223 stars 34 forks source link

1.18 Install is failing #4

Closed sgriffith3 closed 4 years ago

sgriffith3 commented 4 years ago

Version 1.18 Install failed yesterday in a student environment. Here is the relevant ansible output:

TASK [etcd : Reload the daemon for the services] *****************************************************************************************************************************************************************
fatal: [master2]: FAILED! => {"changed": false, "msg": "Unable to restart service etcd.service: Job for etcd.service failed because the control process exited with error code.\nSee \"systemctl status etcd.servi
ce\" and \"journalctl -xe\" for details.\n"}                                                                                                                                                                      
fatal: [master1]: FAILED! => {"changed": false, "msg": "Unable to restart service etcd.service: Job for etcd.service failed because the control process exited with error code.\nSee \"systemctl status etcd.servi
ce\" and \"journalctl -xe\" for details.\n"}                                                                                                                                                                      
fatal: [master3]: FAILED! => {"changed": false, "msg": "Unable to restart service etcd.service: Job for etcd.service failed because the control process exited with error code.\nSee \"systemctl status etcd.servi
ce\" and \"journalctl -xe\" for details.\n"}           

I then ssh'd into master-01, and performed the `sudo journalctl -xe` command. Here is the relevant output.

Apr 02 11:55:21 k8s-2607-master-01 etcd[9268]: started streaming with peer cfa9e50b358bbc20 (writer) Apr 02 11:55:21 k8s-2607-master-01 etcd[9268]: started streaming with peer cfa9e50b358bbc20 (stream Message reader) Apr 02 11:55:21 k8s-2607-master-01 etcd[9268]: set the initial cluster version to 3.4 Apr 02 11:55:21 k8s-2607-master-01 etcd[9268]: cluster cannot be downgraded (current version: 3.2.17 is lower than determined cluster version: 3.4). Apr 02 11:55:21 k8s-2607-master-01 kube-apiserver[20879]: W0402 11:55:21.656914 20879 clientconn.go:1208] grpc: addrConn.createTransport failed to connect to {https://10.5.211.211:2379 0 }. Err :c Apr 02 11:55:21 k8s-2607-master-01 systemd[1]: etcd.service: Main process exited, code=exited, status=1/FAILURE Apr 02 11:55:21 k8s-2607-master-01 systemd[1]: etcd.service: Failed with result 'exit-code'. Apr 02 11:55:21 k8s-2607-master-01 systemd[1]: Failed to start etcd.



Something has set the initial cluster version to 3.4, however it fails when trying to set it to 3.2.17.

We have set the etcd version in group_vars/all.yaml to be 3.2.17. This is what is causing the problem.  I will attempt to update this to 3.4 and verify that the problem goes away. Then update the kubernetes-the-alta3-way playbook.
sgriffith3 commented 4 years ago

Upgraded etcd version to 3.4.0 and the install worked.

When going to fix up the playbook I realized that I am an idiot and was working on 1.17 not 1.18. Closing the issue.