search

xetys / hetzner-kube

A CLI tool for provisioning kubernetes clusters on Hetzner Cloud
Apache License 2.0
745 stars 116 forks source link

Hetzner Cloud Volumes support and problems with cluster extend after manual updates of nodes #256

Closed Edke closed 5 years ago

Edke commented 5 years ago

Hi everyone.

I'm a K8s newbie and evaluating different k8s solutions, came upon your awesome hetzner-kube as I'm a Hetzner Online really satisfied user.

Of course usage of Hetzner volumes would be welcome, I investigated and found https://github.com/hetznercloud/csi-driver and was surprised that no mention about project and it's integration.

After some hacking I got it working according to it's guidelines. Only info not clear was a feature-gates part. That I managed with these steps, found here https://github.com/hetznercloud/csi-driver/issues/17#issuecomment-462685563:

  1. ssh to each node
  2. kubeadm alpha kubelet config download
  3. sudo systemctl restart kubelet

After this I got Volumes working nicely.

BUt now and I'm not sure what is a cause, can't add more workers to cluster:

$ hetzner-kube cluster add-worker --datacenters nbg1-dc3  --name he1                                                           ────────────────────────
2019/03/09 11:24:18 creating server 'he1-worker-06'...
  --- [======================================] 100%
2019/03/09 11:24:22 Created node 'he1-worker-06' with IP 116.203.115.180                                                          
2019/03/09 11:24:22 creating server 'he1-worker-07'...
  --- [======================================] 100%
2019/03/09 11:24:25 Created node 'he1-worker-07' with IP 116.203.115.156                                                          
2019/03/09 11:24:25 sleep for 30s...
he1-master-01        : wireguard configured                7.7% [--------------]                                                  
he1-worker-01        : configure wireguard                 5.9% [--------------]                                                  
he1-worker-02        : wireguard configured               11.8% [--------------]                                                  
he1-worker-03        : configure wireguard                 5.9% [--------------]                                                  
he1-worker-04        : configure wireguard                 5.9% [--------------]                                                  
he1-worker-05        : configure wireguard                 5.9% [--------------]                                                  
he1-worker-06        : configure wireguard                64.7% [========>-----]                                                  
he1-worker-07        : wireguard configured               70.6% [=========>----]                                                  
2019/03/09 11:29:16 run failed
command:systemctl enable wg-quick@wg0 && systemctl restart wg-quick@wg0 && systemctl enable overlay-route.service && systemctl restart overlay-route.service
stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wg-quick@wg0.service → /lib/systemd/system/wg-quick@.service.  
Created symlink /etc/systemd/system/multi-user.target.wants/overlay-route.service → /etc/systemd/system/overlay-route.service.    
Job for overlay-route.service failed because the control process exited with error code.                                          
See "systemctl status overlay-route.service" and "journalctl -xe" for details.                                                    

err:Process exited with status 1

So I have questions:

  1. Is Cloud Volumes planned as feature so no manual tweaking of nodes is required ? As addon it would be great.
  2. Does my current problem with new worker nodes has anything to do with manual nodes modification ?

PS: Sorry, I know this isn't a proper issue more it's a question, I will create new propers ones from this one after discussion.

mavimo commented 5 years ago

@Edke should be fixed here https://github.com/xetys/hetzner-kube/pull/258 most relevant changes to make it available for new nodes & c https://github.com/xetys/hetzner-kube/pull/258/commits/c4d78d9a588bd1ab67f60b5bdb969c55cb0f4a13#diff-65af547d7ac35df5808831c9bf26e0f1R32

mavimo commented 5 years ago

@Edke fixed in 0.4.1

Edke commented 5 years ago

@mavimo I updated hetzner-kube and tried to recreate new cluster with it. Can't get hcloud-volumes to work though

hcloud-csi-nodes keep crashing

hcloud-csi-controller-0                      4/4     Running            0          12m
hcloud-csi-node-j62ml                        1/2     CrashLoopBackOff   7          12m
hcloud-csi-node-xmwq5                        1/2     CrashLoopBackOff   7          12m
hcloud-csi-node-z8rtk                        1/2     CrashLoopBackOff   7          12m