oomichi / try-kubernetes

12 stars 5 forks source link

vagrant-kubespray: missing groups #117

Closed oomichi closed 2 years ago

oomichi commented 2 years ago

ansible-playbook のグループに適切なIPアドレスが渡っていないため、Kubespray での Kubernetes クラスタ構築が成功していないように見える。

ASK [kubernetes/preinstall : gather os specific variables] ********************
ok: [k8s-1] => (item=/home/oomichi/kubespray/roles/kubernetes/preinstall/vars/../vars/ubuntu.yml)
ok: [k8s-2] => (item=/home/oomichi/kubespray/roles/kubernetes/preinstall/vars/../vars/ubuntu.yml)
ok: [k8s-3] => (item=/home/oomichi/kubespray/roles/kubernetes/preinstall/vars/../vars/ubuntu.yml)
Friday 01 April 2022  19:59:27 +0000 (0:00:00.099)       0:00:43.712 ********** 
Friday 01 April 2022  19:59:27 +0000 (0:00:00.047)       0:00:43.759 ********** 

TASK [kubernetes/preinstall : check /usr readonly] *****************************
ok: [k8s-1]
ok: [k8s-2]
ok: [k8s-3]
Friday 01 April 2022  19:59:27 +0000 (0:00:00.502)       0:00:44.262 ********** 
Friday 01 April 2022  19:59:27 +0000 (0:00:00.047)       0:00:44.309 ********** 
Friday 01 April 2022  19:59:27 +0000 (0:00:00.042)       0:00:44.352 ********** 

PLAY RECAP *********************************************************************
k8s-1                      : ok=71   changed=0    unreachable=0    failed=0    skipped=248  rescued=0    ignored=0   
k8s-2                      : ok=69   changed=0    unreachable=0    failed=0    skipped=171  rescued=0    ignored=0   
k8s-3                      : ok=65   changed=0    unreachable=0    failed=0    skipped=137  rescued=0    ignored=0   
localhost                  : ok=4    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   

Friday 01 April 2022  19:59:27 +0000 (0:00:00.068)       0:00:44.421 ********** 
=============================================================================== 
Gather minimal facts ---------------------------------------------------- 1.66s
download : download | Download files / images --------------------------- 1.25s
Gather necessary facts (hardware) --------------------------------------- 1.25s
download : download | Download files / images --------------------------- 1.14s
download : download | Download files / images --------------------------- 1.13s
download : download | Download files / images --------------------------- 1.09s
download : download | Download files / images --------------------------- 1.08s
download : download | Download files / images --------------------------- 1.07s
Gather necessary facts (network) ---------------------------------------- 0.90s
download : download | Download files / images --------------------------- 0.87s
download : download | Download files / images --------------------------- 0.83s
download : download | Download files / images --------------------------- 0.73s
policy_controller/calico : Set cert dir --------------------------------- 0.68s
kubernetes/preinstall : get content of /etc/resolv.conf ----------------- 0.68s
kubernetes/preinstall : check resolvconf -------------------------------- 0.67s
kubernetes/preinstall : set is_fedora_coreos ---------------------------- 0.67s
kubernetes/preinstall : check if booted with ostree --------------------- 0.66s
kubernetes/preinstall : check systemd-resolved -------------------------- 0.52s
kubernetes/preinstall : check systemd-resolved -------------------------- 0.52s
kubernetes/preinstall : set is_fedora_coreos ---------------------------- 0.51s
oomichi commented 2 years ago

どのように groups が生成されるかをコードから調査

          ansible.groups = {
            "etcd" => ["#{$instance_name_prefix}-[1:#{$etcd_instances}]"],
            "kube_control_plane" => ["#{$instance_name_prefix}-[1:#{$kube_master_instances}]"],
            "kube_node" => ["#{$instance_name_prefix}-[1:#{$kube_node_instances}]"],
            "k8s_cluster:children" => ["kube_control_plane", "kube_node"],
          }
oomichi commented 2 years ago

コマンドライン

 INFO subprocess: Starting process: ["/home/oomichi/kubespray-venv/bin/ansible-playbook", "--connection=ssh", "--timeout=30", "--limit=all,localhost", "--inventory-file=/home/oomichi/kubespray/.vagrant/provisioners/ansible/inventory", "--become", "--tags=facts", "--forks=3", "--flush-cache", "-e ansible_become_pass=vagrant", "cluster.yml"]

/home/oomichi/kubespray/.vagrant/provisioners/ansible/inventory/ 配下には hosts.ini ファイルが無い。

ls .vagrant/provisioners/ansible/inventory
group_vars  inventory.ini  vagrant_ansible_inventory

以下のコメントがヒント?

# if $inventory has a hosts.ini file use it, otherwise copy over
# vars etc to where vagrant expects dynamic inventory to be

vagrant が .vagrant/provisioners/ansible/inventory/vagrant_ansible_inventory に自動的に ansible のインベントリファイルを作成するらしい。

$ cat .vagrant/provisioners/ansible/inventory/vagrant_ansible_inventory 
# Generated by Vagrant

k8s-2 ansible_host=192.168.121.228 ansible_port=22 ansible_user='vagrant' ansible_ssh_private_key_file='/home/oomichi/.vagrant.d/insecure_private_key' ip=172.18.8.102 flannel_interface=eth1 kube_network_plugin=flannel kube_network_plugin_multus=False download_run_once=True download_localhost=False download_cache_dir=/home/oomichi/kubespray_cache download_force_cache=False download_keep_remote_cache=False docker_rpm_keepcache=1 kubeconfig_localhost=True kubectl_localhost=True local_path_provisioner_enabled=False local_path_provisioner_claim_root=/opt/local-path-provisioner/ ansible_ssh_user=vagrant
k8s-1 ansible_host=192.168.121.76 ansible_port=22 ansible_user='vagrant' ansible_ssh_private_key_file='/home/oomichi/.vagrant.d/insecure_private_key' ip=172.18.8.101 flannel_interface=eth1 kube_network_plugin=flannel kube_network_plugin_multus=False download_run_once=True download_localhost=False download_cache_dir=/home/oomichi/kubespray_cache download_force_cache=False download_keep_remote_cache=False docker_rpm_keepcache=1 kubeconfig_localhost=True kubectl_localhost=True local_path_provisioner_enabled=False local_path_provisioner_claim_root=/opt/local-path-provisioner/ ansible_ssh_user=vagrant
k8s-3 ansible_host=192.168.121.210 ansible_port=22 ansible_user='vagrant' ansible_ssh_private_key_file='/home/oomichi/.vagrant.d/insecure_private_key' ip=172.18.8.103 flannel_interface=eth1 kube_network_plugin=flannel kube_network_plugin_multus=False download_run_once=True download_localhost=False download_cache_dir=/home/oomichi/kubespray_cache download_force_cache=False download_keep_remote_cache=False docker_rpm_keepcache=1 kubeconfig_localhost=True kubectl_localhost=True local_path_provisioner_enabled=False local_path_provisioner_claim_root=/opt/local-path-provisioner/ ansible_ssh_user=vagrant

[etcd]
k8s-[1:3]

[kube_control_plane]
k8s-[1:2]

[kube_node]
k8s-[1:3]

[k8s_cluster:children]
kube_control_plane
kube_node

インベントリファイルの内容は正しいように見える。

oomichi commented 2 years ago

https://github.com/kubernetes-sigs/kubespray/blob/3782573ede21e05b5b99c251f9b0825b9e804a9a/Vagrantfile#L259

ansible.tags = ['facts']

で facts を指定しているため、実際のデプロイが走らない模様。 これは CI でデプロイ失敗が多発したため、一時的にデプロイを走らせないようにしている措置。