kairos-io / kairos

:penguin: The immutable Linux meta-distribution for edge Kubernetes.
https://kairos.io
Apache License 2.0
989 stars 87 forks source link

feat: Add stage for cluster bootstrap (non-p2p) #2569

Open 6ixfalls opened 1 month ago

6ixfalls commented 1 month ago

Is your feature request related to a problem? Please describe.

Currently its difficult to apply manifests after the k3s server has started. I'm forced to use reconcile for this, but that's not a reliable solution because if the cluster takes more than 5 minutes to start k3s, it takes an additional 60 minutes before my manifests and scripts are run and therefore leads to a poor user experience.

This is an extension of #2518, but in this case, I'm trying to apply manifests only to the first node in the cluster, and I'm not using p2p. It seems like the boot stage isn't called after the after-install-chroot stage, which results in my steps not being executed as intended. https://github.com/6ixfalls/homelab-gitops/blob/ea6b82a1453adbf6e93638012e68151ec5aeda21/bootstrap/cloud-config.yaml#L69 If this is solvable without this feature, it would be great - maybe it could be just a configuration issue. I can also manually run kairos-agent run-stage boot or whatever stage is specified in the manifest above and my cluster is bootstrapped fine.

Describe the solution you'd like

There should be another kairos agent stage, something like cluster-ready, which is called when the k3s server or agent is started.

Describe alternatives you've considered

I've tried the reconcile stage but it will take 65 minutes after first boot before the cluster is ready.

ci-robbot commented 1 month ago

Hello 6ixfalls, thank you for bringing this issue to our attention. While your description provides some context, it seems like there is a missing part that could help us understand the problem better. Could you please provide more details on how to reproduce the issue and mention the versions of the relevant artifacts you're using? This will help us in triaging and providing a better solution to the problem.

ci-robbot commented 3 weeks ago

Hello 6ixfalls, thank you for your detailed description of the issue you're facing. In order to better assist you, could you please provide more information about the versions of Kairos and K3s you are using? Additionally, please mention any other relevant artifacts that you're using as part of your cluster setup. This information will help in understanding the situation more clearly.

I'd like to remind you that for a new issue to be properly triaged and addressed, it should meet the following requirements:

Please make sure to include this information in your issue, so that the triage process can be completed efficiently. I am a bot, an experiment by @mudler and @jimmykarily, and my purpose is to ensure that all issues meet the necessary requirements before proceeding.

Thank you for your patience and understanding.