techno-tim / k3s-ansible

The easiest way to bootstrap a self-hosted High Availability Kubernetes cluster. A fully automated HA k3s etcd install with kube-vip, MetalLB, and more. Build. Destroy. Repeat.
https://technotim.live/posts/k3s-etcd-ansible/
Apache License 2.0
2.41k stars 1.05k forks source link

add kube-vip as a service load balancer #432

Closed gereonvey closed 9 months ago

gereonvey commented 9 months ago

Proposed Changes

Thanks a lot for your work @timothystewart6, love to follow your tutorials! In the video for this project you mentioned that you didn't get kube-vip to work for services...

This PR allows to use kube-vip as loadbalancer for services, replacing MetalLB.

You can choose if you want to use MetalLB or kube-vip as the loadbalancer for services, just by changing variable service_lb_type to either metallb or kube-vip. The IP range for the service loadbalancer can be specified via the service_lb_ip_range variable.

Breaking changes for existing inventories:

Checklist

timothystewart6 commented 9 months ago

Thank you for this! I love the idea of adding support for using kube-vips service load balancer. Is there any way we could make this a non breaking change?

Something like

metal_lb_ip_range

kube_vip_lb_ip_range

I know that service_lb_ip_range and service_lb_type more scalable however I'd like to not break the contract / api if we really don't have to because it also breaks the documentation :)

We just did something similar in https://github.com/techno-tim/k3s-ansible/pull/414 (making changes backwards compatible). Let me know what you think!

Thanks!

timothystewart6 commented 9 months ago

Also, if you are up to it, it would be great to have a molecule test for this. It will be a lot of copy and paste but would be helpful to know if it works and if future changes break it. You can also see #414 for an example test. Single nodes is fine!

gereonvey commented 9 months ago

I added a molecule scenario for kube-vip, based on single-node (similar to the calico scenario). Unfortunately I can't test this, it seems I can't get Vagrant to work on my laptop.

timothystewart6 commented 9 months ago

Thank you! I just approved it for CI, so it will now run remote. I should write up a guide at some point for configuring molecule tests locally. Thank you for the test too!

timothystewart6 commented 9 months ago

Looks like lint. You can fix it by installing pre-commit and it will fix it for you.

gereonvey commented 9 months ago

Looks like lint. You can fix it by installing pre-commit and it will fix it for you.

Sorry for that, forgot to run it before commit... installed it now. Also I noticed that the new molecule scenario needs to be listed in the test workflow.

timothystewart6 commented 9 months ago

The tests are passing but this is because metallb is still getting installed.

https://github.com/techno-tim/k3s-ansible/actions/runs/7688568449/job/20950037294#step:13:222

I think you might need to add an override for metallb to unset the metallb IP range since it's in the default all group vars

gereonvey commented 9 months ago

The tests are passing but this is because metallb is still getting installed.

What makes you believe that metallb is getting installed? The task is being skipped.

TASK [k3s_server : Deploy metallb manifest] ************************************
  skipping: [control1] => {"changed": false, "false_condition": "kube_vip_lb_ip_range is not defined", "skip_reason": "Conditional result was False"}

In fact, if you scroll further you see how kube-vip is being deployed.

https://github.com/techno-tim/k3s-ansible/actions/runs/7688568449/job/20950037294#step:13:222

I think you might need to add an override for metallb to unset the metallb IP range since it's in the default all group vars

No, the whole logic is relying on kube_vip_lb_ip_range being defined or not.

timothystewart6 commented 9 months ago

Thank you! You're right! I am not sure what I was looking at. Thank you for this!