techno-tim / k3s-ansible

The easiest way to bootstrap a self-hosted High Availability Kubernetes cluster. A fully automated HA k3s etcd install with kube-vip, MetalLB, and more. Build. Destroy. Repeat.
https://technotim.live/posts/k3s-etcd-ansible/
Apache License 2.0
2.41k stars 1.05k forks source link

Verify that all nodes actually joined - Debian #52

Closed fredrikscode closed 2 years ago

fredrikscode commented 2 years ago

Expected Behavior

I run ansible-playbook site.yml -i inventory/k3s-cluster/hosts.ini and get a k3s-cluster deployed.

Current Behavior

Runs through the playbook until the verification process which it goes through without success.

Steps to Reproduce

  1. Modify hosts.ini with ip's
  2. Add variable to all.yml: ansible_ssh_private_key_file: ~/.ssh/ansible
  3. Run site.yml using: ansible-playbook site.yml -i inventory/k3s-cluster/hosts.ini

Context (variables)

Operating system: Debian 11

Hardware: (CPU/RAM/Disk type)

Variables Used:

all.yml

k3s_version: v1.24.3+k3s1
ansible_user: ansible
ansible_ssh_private_key_file: ~/.ssh/ansible
systemd_dir: /etc/systemd/system
system_timezone: "Europe/Stockholm"
flannel_iface: "eth0"
apiserver_endpoint: "10.0.0.100"
k3s_token: "strawberrycake"
extra_server_args: "--disable servicelb --disable traefik"
extra_agent_args: ""
kube_vip_tag_version: "v0.5.0"
metal_lb_speaker_tag_version: "v0.13.4"
metal_lb_controller_tag_version: "v0.13.4"
metal_lb_ip_range: "10.0.0.80-10.0.0.90"

Hosts

host.ini

[master]
k3s-master-01
k3s-master-02
k3s-master-03

[node]
k3s-worker-01
k3s-worker-02
k3s-worker-03

[k3s_cluster:children]
master
node

Possible Solution

timothystewart6 commented 2 years ago

This seems to be a Debian specific issue. It works fine with Ubuntu

➜  kubectl get nodes
NAME     STATUS   ROLES                       AGE    VERSION
k3s-01   Ready    control-plane,etcd,master   113s   v1.24.3+k3s1
k3s-02   Ready    control-plane,etcd,master   95s    v1.24.3+k3s1
k3s-03   Ready    control-plane,etcd,master   73s    v1.24.3+k3s1
k3s-04   Ready    <none>                      28s    v1.24.3+k3s1
k3s-05   Ready    <none>                      29s    v1.24.3+k3s1
fredrikscode commented 2 years ago

Hm, weird. I'll try with Ubuntu in a minute or two

fredrikscode commented 2 years ago

Tested with no more modifications other than what has been documented above and I still experience the same thing using Ubuntu 22.10. Also tested with 20.04 so I can't seem to blame this on that unfortunately.

Ended up with this error, which I've seen on Debian as well:

fatal: [k3s-master-01]: FAILED! => {"attempts": 20, "changed": false, "cmd": ["k3s", "kubectl", "get", "nodes", "-l", "node-role.kubernetes.io/master=true", "-o=jsonpath={.items[*].metadata.name}"], "delta": "0:00:00.104545", "end": "2022-08-20 22:50:14.638833", "msg": "", "rc": 0, "start": "2022-08-20 22:50:14.534288", "stderr": "", "stderr_lines": [], "stdout": "ubuntu", "stdout_lines": ["ubuntu"]}

timothystewart6 commented 2 years ago

Not sure, I can run it multiple times in my lab (just did at least 20x to fix a bug). Are you sure you're using the correct ethernet adapter name?