search

hcloud-talos / terraform-hcloud-talos

This repository contains a Terraform module for creating a Kubernetes cluster with Talos in the Hetzner Cloud.
https://registry.terraform.io/modules/hcloud-talos/talos
MIT License
28 stars 7 forks source link

Support for nodes in different regions #18

Open Silvest89 opened 1 month ago

Silvest89 commented 1 month ago

This way you can have a control plane node in each location for europe. (FSN, NBG, HEL)

mrclrchtr commented 1 month ago

Thanks for the feature request. I am currently carrying out some basic code refactorings. After that it should be easier to realize the feature.

TeeTeufel commented 3 weeks ago

As far as I tried to implement this feature, I think this will be a breaking change. There could be something I missed, or did wrong. https://github.com/TeeTeufel/terraform-hcloud-talos/tree/feature/nodesInDifferentRegions

Changing the datacenter of existing nodes will destroy an recreate them, this is something I expect. While recreating them, it seems that inline primary IPv4/IPv6 addresses won't be recreated. This leads to an terraform error, because primary ip addresses are location bound and can't be transferred to a different datacenter.

I will check how this will behave if primary ips are created with their own resource.

Unrelated to the creation of primary ips, changing the datacenter will probably result in different primary ips for all recreated nodes.

This enhancement probably needs a bit more research in context with autoscaling, because currently I set the datacenter location for the nodes as a list of strings.

With terraform 1.9 there should be a feature introduced to validate a variable condition with external variables. Currently, the list of strings could be longer, or shorter. https://github.com/hashicorp/terraform/issues/25609#issuecomment-2125070095

mrclrchtr commented 3 weeks ago

This module is currently not able to carry out meaningful upgrades anyway. Changing the machine config would always lead to a recreate of the server. (I often create my cluster from scratch. Simple/automatic importing of backups helps a lot.)

If updates are made, then they should be made via patches. But this is not so easy, because patches are not idempotent. They build on the current state and change it. This means that a patch can have a different effect the second time you run it.

I think at the beginning it is sufficient to define several node groups in regions when creating the cluster, similar to kube-hetzner.