search

spinkube / spin-operator

Spin Operator is a Kubernetes operator that empowers platform engineers to deploy Spin applications as custom resources to their Kubernetes clusters
https://www.spinkube.dev/docs/overview/
Other
185 stars 23 forks source link

Talos cluster thwarts installer with security context #203

Closed kingdonb closed 6 months ago

kingdonb commented 6 months ago

In my spin operator controller manager, I have this class of errors which I believe is responsible for the app not starting up (it remains in ContainerCreating status with some errors I will also copy below):

2024-04-14T23:03:03Z    INFO    KubeAPIWarningLogger    would violate PodSecurity "restricted:latest": allowPrivilegeEscalation != false (container "simple-spinapp" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container "simple-spinapp" must set securityContext.capabilities.drop=["ALL"]), runAsNonRoot != true (pod or container "simple-spinapp" must set securityContext.runAsNonRoot=true), seccompProfile (pod or container "simple-spinapp" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")
  Warning  FailedCreatePodSandBox  4m52s (x47 over 14m)  kubelet            Failed to create pod sandbox: rpc error: code = Unknown desc = failed to get sandbox runtime: no runtime for "spin" is configured

I'm guessing that something about the way talos bakes host images is going to require me to install the containerd-shim-spin a different way. Don't know if these errors are connected, but these two errors were the only indication of something wrong once the pod failed to come online!

Is there anyone testing talos linux successfully who has blogged or written about their journey that I might benefit from seeing?

vdice commented 6 months ago

I think talos support in the runtime-class-manager may still be in progress. (The error you've provided above, i.e. no runtime for "spin" is configured points at the shim not being installed where containerd expects it, if I recall correctly.)

There's a related issue in the shim project re: Talos support: https://github.com/spinkube/containerd-shim-spin/issues/57. It mentions a now-merged upstream contribution to Talos, but I'm not sure if explicit support still needs to be added to the rumtime manager. cc @0xE282B0 who may be able to advise.

kingdonb commented 6 months ago

It looks like it's in progress, or landed, depending on your vantage point:

yebyen@Kingdons-MacBook-Pro website % crane export ghcr.io/siderolabs/extensions:v1.7.0-beta.1 | tar x -O image-digests | grep spin
ghcr.io/siderolabs/spin:v0.13.1@sha256:f620675d41ba6e10f94e90e5184dbf21fd6f492d844d131f937fe0bcd0c73860

I'm running Talos 1.6.5 which does not have this extension published for it. 1.7.0-beta.0 was published two weeks ago, also does not have the extension. v1.7.0-beta.1 has an extension.

Looks like I'm upgrading to Talos v1.7.0-beta.1 as soon as I figure out how to cobble together an image with this extension in it from the image factory, and I'll let you know how it goes on the other side!

kingdonb commented 6 months ago

It worked! (The second try!) Once I figured out how to upgrade via talosctl upgrade, by passing --image

My spin app just sprung to life. So, the answer is to use a Talos image that has the spin extension baked in, from Image Factory. Awesome!

0xE282B0 commented 6 months ago

Hi @kingdonb, Sorry for being late 😅. As you figured out the Spin extension for Talos was merged a week ago. Great to see you checking out SpinKube! Let me know if you need anything 😊