Reinstalling kubernetes the pharos way

[mthird]

What's the best way to reset and reinstall pharos without having to pave the machines?

I've tried kubeadm reset, removing the /etc/pharos and /etc/kubernetes directories, uninstalling kubeadm/kubelet/docker.io, then running the up command, but it gets stuck at verifying port 6443 is listening.

The syslog shows

Jun 12 17:15:52 dev-k8s-master1 kubelet[8686]: Flag --read-only-port has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jun 12 17:15:52 dev-k8s-master1 kubelet[8686]: I0612 17:15:52.270879    8686 feature_gate.go:226] feature gates: &{{} map[]}
Jun 12 17:15:52 dev-k8s-master1 kubelet[8686]: F0612 17:15:52.270959    8686 server.go:218] unable to load client CA file /etc/kubernetes/pki/ca.crt: open /etc/kubernetes/pki/ca.crt: no such file or directory
J

I trolled through the source code but I don't see where the ca.crt file is explictly created.

Any help would be appreciated.

[mthird]

This is for version 1.1.1, btw.

[mthird]

Just discovered something interesting ... but removing the worker nodes, the pharos log now shows the certificate generation step.

My cluster.yaml file:

hosts:
  - address: "172.17.0.103"
    role: master
    user: vmadmin
    ssh_key_path: ~/.ssh/ae/id_rsa
    #- address: "172.17.0.104"
    #role: worker
    #user: vmadmin
    #ssh_key_path: ~/.ssh/ae/id_rsa
    #- address: "172.17.0.105"
    #role: worker
    #user: vmadmin
    #ssh_key_path: ~/.ssh/ae/id_rsa
network:
  provider: calico
  calico:
    ipip_mode: CrossSubnet
addons:
  kubernetes-dashboard:
    enabled: true
  host-upgrades:
    enabled: true
    interval: "7d"
  openebs:
    enabled: true

[mthird]

For anyone else that wants to reset pharos without reinstalling the OS, the following script takes care of it.

Before figuring out every directory that needed to be deleted, I saw the following errors:

Internal error occurred: invalid padding on input

Note that this will destroy any data stored in existing containers and volumes!

sudo kubeadm reset && sudo apt-get remove -y --allow-change-held-packages kubeadm kubelet kubectl docker.io && sudo rm -rf /etc/kubernetes /etc/pharos /var/etcd ~/.kube /var/lib/docker /var/lib/kubelet /var/lib/cni /var/lib/etcd /var/lib/weave /var/lib/calico

[jakolehm]

I think that reset example is missing few things... more complete reset would be something like:

#!/bin/sh

systemctl stop kubelet
systemctl disable kubelet
systemctl stop docker
systemctl disable docker
kubeadm reset
apt-get purge -y --allow-change-held-packages --purge kubeadm kubelet kubectl docker.io
apt-get autoremove -y
rm -rf /etc/kubernetes \
    /etc/pharos \
    /etc/systemd/system/kubelet.service \
    /etc/systemd/system/kubelet.service.d \
    ~/.kube \
    /var/lib/docker \
    /var/lib/containerd \
    /var/lib/kubelet \
    /opt/cni \
    /var/lib/etcd \
    /var/lib/weave \
    /var/lib/calico \
    /usr/local/bin/crictl

systemctl daemon-reload
systemctl reset-failed