search

siderolabs / talos

Talos Linux is a modern Linux distribution built for Kubernetes.
https://www.talos.dev
Mozilla Public License 2.0
6.97k stars 565 forks source link

System extensions that are 'critical' #8121

Closed btrepp closed 11 months ago

btrepp commented 11 months ago

Feature Request

Allow critical extensions (e.g networking) to run before/independent of the kubelet/talos internals. Things like networking need to be up for other commands to run, but you can get into a state in which you can't issue 'talosctl upgrade', as the cluster isn't ready, but it's not ready, because you need to run the upgrade command, thus forming a nice loop :).

Description

I have been rebuilding my cluster again using the Tailscale extension, this time trying to force more and more of the 'control plane' to be inside the Tailscale network only

        # # The `nodeIP` field is used to configure `--node-ip` flag for the kubelet.
        nodeIP:
        #     # The `validSubnets` field configures the networks to pick kubelet node IP from.
          validSubnets:
          -  100.64.0.0/10

does work, and restricts kublets into the Tailscale network interfaces on the control plane nodes.

One a new controlplane/rebuild, this can cause issues though. The control plane can't join the cluster, as the networking capability isn't up, but I need to run upgrade to get the networking extension installed :(.

Am trying with '--force' on the new node, to see if it can install the extensions so that they are ready, but it is taking awhile because it hasn't got a connection to etcd/the cluster yet.

btrepp commented 11 months ago

Okay, so the trick in the new method is to use the factory service to 'bake in' the extension, and make sure you use that in the ISO/boot.

This actually makes a heap of sense, so solves this problem. If it's critical these files exist, they should be baked into your ISO :).