Replaced weave by Flannel and added versioning on cluster components.

tedsluis commented 6 years ago

Replaced weave with Flannel and added versioning on docker, kubeadm, kubelet, kubectl, kubernetes images and flannel. Added cleanup.yml playbook.

Description

Kubernetes 1.10.x (latest) doesn't run (yet) on Raspberry (see https://github.com/kubernetes/kubernetes/issues?utf8=%E2%9C%93&q=is%3Aissue+raspberry).
until now rak8s always installs the latest version of it's components: docker-ce, kubeadm, kubelet, kubectl and kubernetes images.

After many tests and reading upstream kubernetes issues, I came to the conclusion that rak8s only runs when using docker-ce 18-04, kubeadm 1.9.7-00, kubelet 1.9.7-00, kubectl 1.9.7-00 and kubernetes images 1.9.7 in combination with flannel 0.10.0 (instead of weave).

This pull request provides two things:

versioning of the cluster components (specify the versions in the inventory file).
a cleanup.yml playbook that removes docker, docker images, kubernetes and its packages and it performs a reboot of the raspberries.

Added variables to the inventory file:

[all:vars]
kubernetes_package_version="1.9.7-00"
kubernetes_version="v1.9.7"
docker_ce_version="18.04.0~ce~3-0~raspbian"
flannel_version="v0.10.0"

Testing

Down here a test run of the cleanup.yml playbook:

pi@ansible-node ~/rak8s $ ansible-playbook cleanup.yml 

PLAY [all] *********************************************************************

TASK [setup] *******************************************************************
ok: [node2]
ok: [node1]
ok: [master]

TASK [cleanup : Reset Kubernetes] **********************************************
changed: [master]
changed: [node1]
changed: [node2]

TASK [cleanup : Remove docker images] ******************************************
changed: [master]
changed: [node1]
changed: [node2]

TASK [cleanup : apt-get update] ************************************************
changed: [node2]
changed: [node1]
changed: [master]

TASK [cleanup : apt-get purge packages] ****************************************
ok: [master] => (item=[u'kubelet', u'kubeadm', u'kubectl', u'doceker-ce'])
ok: [node2] => (item=[u'kubelet', u'kubeadm', u'kubectl', u'doceker-ce'])
ok: [node1] => (item=[u'kubelet', u'kubeadm', u'kubectl', u'doceker-ce'])

TASK [cleanup : Remove pod logging] ********************************************
changed: [node1]
 [WARNING]: Consider using file module with state=absent rather than running rm

changed: [master]
changed: [node2]

TASK [cleanup : Reboot] ********************************************************
ok: [master]
ok: [node1]
ok: [node2]

TASK [cleanup : Wait for Reboot] ***********************************************
ok: [node2 -> localhost]
ok: [master -> localhost]
ok: [node1 -> localhost]

PLAY RECAP *********************************************************************
master                     : ok=8    changed=4    unreachable=0    failed=0   
node1                      : ok=8    changed=4    unreachable=0    failed=0   
node2                      : ok=8    changed=4    unreachable=0    failed=0

After running the cleanup.yml playbook the raspberries are more or less fresh.

Down here a run of the cluster.yml playbook.

pi@ansible-node ~/rak8s $ ansible-playbook cluster.yml 

PLAY [all] *********************************************************************

TASK [setup] *******************************************************************
ok: [node2]
ok: [node1]
ok: [master]

TASK [common : Enabling cgroup options at boot] ********************************
ok: [node1]
ok: [node2]
ok: [master]

TASK [common : apt-get update] *************************************************
ok: [node2]
ok: [node1]
ok: [master]

TASK [common : apt-get upgrade] ************************************************
ok: [node2]
ok: [master]
ok: [node1]

TASK [common : Reboot] *********************************************************
skipping: [master]
skipping: [node1]
skipping: [node2]

TASK [common : Wait for Reboot] ************************************************
skipping: [master]
skipping: [node1]
skipping: [node2]

TASK [kubeadm : Disable Swap] **************************************************
changed: [node1]
changed: [master]
changed: [node2]

TASK [kubeadm : Determine if docker is installed] ******************************
ok: [master]
ok: [node1]
ok: [node2]

TASK [kubeadm : Run Docker 18.04.0~ce~3-0~raspbian Install Script] *************
changed: [node1]
changed: [master]
changed: [node2]

TASK [kubeadm : Pass bridged IPv4 traffic to iptables' chains] *****************
ok: [node2]
ok: [node1]
ok: [master]

TASK [kubeadm : Install apt-transport-https] ***********************************
ok: [master]
ok: [node1]
ok: [node2]

TASK [kubeadm : Add Google Cloud Repo Key] *************************************
changed: [master]
 [WARNING]: Consider using get_url or uri module rather than running curl

changed: [node1]
changed: [node2]

TASK [kubeadm : Add Kubernetes to Available apt Sources] ***********************
ok: [master]
ok: [node1]
ok: [node2]

TASK [kubeadm : apt-get update] ************************************************
changed: [node1]
changed: [master]
changed: [node2]

TASK [kubeadm : Install k8s 1.9.7-00 Y'all] ************************************
changed: [node1] => (item=[u'kubelet=1.9.7-00', u'kubeadm=1.9.7-00', u'kubectl=1.9.7-00'])
changed: [node2] => (item=[u'kubelet=1.9.7-00', u'kubeadm=1.9.7-00', u'kubectl=1.9.7-00'])
changed: [master] => (item=[u'kubelet=1.9.7-00', u'kubeadm=1.9.7-00', u'kubectl=1.9.7-00'])

PLAY [master] ******************************************************************

TASK [master : Reset Kubernetes Master] ****************************************
changed: [master]

TASK [master : Initialize Master  v1.9.7] **************************************
changed: [master]

TASK [master : Create Kubernetes config directory] *****************************
ok: [master]

TASK [master : Copy admin.conf to config directory] ****************************
changed: [master]

TASK [master : Join Kubernetes Cluster] ****************************************
changed: [master]

TASK [master : Install Flannel (Networking)] ***********************************
changed: [master]

TASK [master : Poke kubelet] ***************************************************
changed: [master]

TASK [dashboard : Install k8s Dashboard] ***************************************
changed: [master]

TASK [dashboard : Configure Dashboard Access] **********************************
changed: [master]

TASK [dashboard : Force Rebuild Dashboard Pods] ********************************
changed: [master]

TASK [dashboard : Fetch kubeconfig file] ***************************************
changed: [master]

PLAY [all:!master] *************************************************************

TASK [workers : Reset Kubernetes] **********************************************
changed: [node1]
changed: [node2]

TASK [workers : Join Kubernetes Cluster] ***************************************
changed: [node2]
changed: [node1]

TASK [workers : Poke kubelet] **************************************************
changed: [node2]
changed: [node1]

PLAY RECAP *********************************************************************
master                     : ok=24   changed=15   unreachable=0    failed=0   
node1                      : ok=16   changed=8    unreachable=0    failed=0   
node2                      : ok=16   changed=8    unreachable=0    failed=0   

pi@master:~ $ kubectl get nodes
NAME      STATUS    ROLES     AGE       VERSION
master    Ready     master    12m       v1.9.7
node1     Ready     <none>    9m        v1.9.7
node2     Ready     <none>    9m        v1.9.7

The cluster is up and running with flannel as network plugin!

Issue Number

My change fixes issue #33

chris-short commented 6 years ago

Huh... I have k8s 1.10.1 running with Docker version 18.03.0-ce, build 0520e24. How did you settle on these versions?

tedsluis commented 6 years ago

@chris-short: Thanks for your update. Just to be clear, when you say 1.10.1 version, to what component(s) are you referring? What kubernetes images version do you run? And what kubeadm version at the time of deployment did you run? And kubelet version do you currently run?

I tried to reproduce your setup with the following combination of versions:

docker	kubeadm	kubernetes images	kubelet	result
18.03-ce	1.10.1-00	1.10.1	1.10.1-00	failed on kubeadm init with error "The kubelet is not running"
18.03-ce	1.9.7-00	1.10.1	1.10.1-00	failed on kubeadm init
18.03-ce	1.9.7-00	1.10.1	1.9.7-00	playbook finished successful, flannel CrashLoopBackOff
18.03-ce	1.9.7-00	1.9.7-00	1.9.7-00	playbook finished successful, flannel works okay

Could it be that you installed your rak8s cluster with kubeadm version 1.9.x? And later upgraded some packages and/or images?

hkoessler commented 6 years ago

@tedsluis I tried your changes on my fresh raspberry pi cluster. It also works for me with theses version numbers

kubernetes_package_version="1.9.7-00" kubernetes_version="v1.9.7" docker_ce_version="18.04.0~ce~3-0~raspbian" flannel_version="v0.10.0"

So there is at least a second test :-)

I am using 2018-04-18-raspbian-stretch-lite as the rasbian image.

Greetings Heinz

chris-short commented 6 years ago

I hope this works. 🤞

FYI... I'm splitting my six node cluster into two three node ones so I can better troubleshoot and iterate.

tedsluis commented 6 years ago

quote chris-short: > FYI... I'm splitting my six node cluster into two three node ones so I can better troubleshoot and iterate.

Thanks, I am sure we all benefit when more people do regular testing!

chris-short / rak8s