spin won't work on RKE2

[tardich]

I'm following the instructions to install the shim on a RKE2 cluster. For it, I had to edit the values such as:

installer: hostEtcContainerdPath: /var/lib/rancher/rke2/agent/etc/containerd hostBinPath: /opt/bin

The reason of the hostBinPath is that I'm running on top of Flatcar (immutable OS) where the default /usr/bin or /usr/local/bin is on a ReadOnly partition, hence the /opt/bin where rke2 normally finds its addon binaries.

And the config.toml resides in the said path above. That said, the config.toml is being rewritten by the rke2 binary at start so losing the configuration at boot. Instead, I need to write custom configs in config.toml.tmpl so a better approach would be to be able to edit the HOST_CONTAINERD_CONFIG in order to edit the right file (which could default to config.toml but configurable)

[tardich]

Was ale to modify the helm chart to point to a config.toml.tmpl file to accomodate the RKE2 format. Now, I'm having issue with the toml binary that does not like the format of my config.toml.tmpl file, that looks like:

{{ template "base" . }}

which toml doesn't recognize, which makes me think that the entrypoint script needs to take a different approach.

Here's an idea. Let's say that I let the script know that the file is a template file (not the full config.toml). So maybe the script could just add to a temp, empty file, the required stuff for spin to work, then happen whatever I need (coming from the values file (in my case, the basic stuff stated above) and then put the result back in the config.toml.tmpl so it can then work as prescribed by RKE2 (in my case). This would make the installer way more flexible.

I'll try to create my own image from the code and adapt the sccript to perform some tests.

[jpflueger]

Hey @tardich, thanks for raising this issue!

With the recent announcement of SpinKube we will be deprecating this repository. In the near-term I would recommend using the kwasm-operator with the node-installer image from the containerd-shim-spin repository. You can find the documentation here.

If you prefer the DaemonSet approach over the kwasm-operator you should be able to adapt the DaemonSet from the kwasm-node-installer repository to accomplish the same thing. I'll update the README with a notice to avoid future confusion.

[tardich]

Thanks for the hints. I gave kwasm-operator a try. For an unknown reason, it killed all my nodes (in fact, the rke2 process) that I had to restart manually one by one. Once done, though, everything ran smoothly.

Regarding the spin-operator helm chart, I think it should include the CRD's instead of having to install them in a seperate task. And as this has to be done in two steps at the moment, the order in the documentation is not in the right order. The spin-operator cannot install if the CRD's aren't present, so the doc should talk about the CRD's BEFORE showing how to install the operator. But I really thing they should be included in the chart, as they are required.