Add documentation: How to manage the k3os configration(s) via Kubernetes operator

[FabianNiehaus]

Is your feature request related to a problem? Please describe. The documentation contains the following passage:

The /var/lib/rancher/k3os/config.yaml or config.d/* files are intended to be used at runtime. These files can be manipulated manually, through scripting, or managed with the Kubernetes operator.

Sadly, I cannot find any instructions on how to use the Kubernetes operator to manage the configurations.

Describe the solution you'd like Add documentation on how to to manage the k3os configration(s) via the Kubernetes operator to the ReadMe.

Describe alternatives you've considered If such documentation already exists, simply refer to it by a link at the corresponding place in the ReadMe.

Additional context n.a.

[annismckenzie]

I'm working on an operator over in https://github.com/annismckenzie/k3os-config-operator. 🤫 There is no operator yet, there was a mention of something in https://github.com/rancher/k3os/issues/157 but it's been over a year.

[chris13524]

@annismckenzie

https://github.com/rancher/system-upgrade-controller

https://github.com/rancher/k3os/blob/master/overlay/share/rancher/k3s/server/manifests/system-upgrade-plans/k3os-latest.yaml

I'm not sure if this integration is functional, but it seems to exist?

[annismckenzie]

That’s for upgrading k3OS to a new version and that operator is all good. But that’s not handling the node configuration (even though it could) and other features we might want.

I will take a look at that controller (I actually wanted to do that this week, just haven’t gotten around to it yet). Thanks for reminding me!

On 10. Jan 2021, at 4:44 AM, Chris Smith notifications@github.com wrote:

 @annismckenzie

https://github.com/rancher/system-upgrade-controller

https://github.com/rancher/k3os/blob/master/overlay/share/rancher/k3s/server/manifests/system-upgrade-plans/k3os-latest.yaml

I'm not sure if this integration is functional as it's lacking docs, but it seems to exist?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[vdboor]

The only approach I've found so far:

ssh rancher@k3os-1234
sudo -i
mount -o remount,rw /k3os/system
vim /k3os/system/config.yaml
mount -o remount,ro /k3os/system
reboot

I wanted to alter write_files to add a persistent mount point in /etc/fstab (applied on boot). I don't see how this could be altered at runtime by the Kubernetes operator.

[Apocrathia]

Has there been any movement on this? The lack of a Python runtime in k3os also means that we can't leverage Ansible to update the config files. I've got a script that essentially tears down the cluster, copies the new configs, and reboots everything. However, I've got a use case where I would like to be able to change where k3s is looking for the k8s control-plane, as part of my initialization manifests include allocating an external address to the k8s control-plane, and I would like for nodes to begin looking there rather than continuing to hit the same node.

[annismckenzie]

@Apocrathia As I mentioned in https://github.com/rancher/k3os/issues/644#issuecomment-757364681 I am working on an operator. It is kinda-sorta (😅) functional but I haven't gotten around to finishing it. What it is able to do is roll out updated node config files. What is missing is what @vdboor mentioned. That part would be handled by the system upgrade controller that @chris13524 linked to – it's able to do those commands on the node host and reboot.

My project has fallen by the wayside a bit because I'm currently lacking a functional k3os cluster I could test it all with. But – if you want – we could pair up and get it to work. Maybe meet on Slack or Zoom? I'd be interested (and I'd have a guinea pig with a functional k3os cluster and a drive to make it work 😬). WDYT?

[dweomer]

SUC would be a good candidate for dropping config files into /var/lib/rancher/k3os/config.d