Closed neoakris closed 7 months ago
smira
commented
2 years ago I think supporting non-runc container runtimes is a feature which should be part of Talos. We need to figure out exact path towards that, as certainly bundling every possible container runtime with Talos doesn't sound like great idea.
andrewrynhard
commented
2 years ago I think supporting non-runc container runtimes is a feature which should be part of Talos. We need to figure out exact path towards that, as certainly bundling every possible container runtime with Talos doesn't sound like great idea.
I agree. This would be a killer feature of Talos (Just look at everything suggested to get centos 7/8 doing something comparable. That is a lot of work.). I think we should decide on one and stick with it like we do with containerd.
neoakris
commented
2 years ago 100% agree that centos = crap ton of yak shaving, that I'd rather not do.
solid call on picking one and standardizing. I <3 standardization. Per this kata containers > gVisor by far in terms of compatibility and some performance scenarios. (In other performance scenarios they're close in terms of overhead. The only mild advantage gVisor had was a startup time of 0.5 sec vs kata's 1 sec, which should be perfectly acceptable to most.) https://object-storage-ca-ymq-1.vexxhost.net/swift/v1/6e4619c416ff4bd19e1c087f27a43eea/www-assets-prod/presentation-media/kata-containers-and-gvisor-a-quantitave-comparison.pdf
kata containers performance is likely to improve in the future since it supports both qemu and firecracker virtualization technologies. kata containers = nested virtualization --implies--> need hardware support, this should be extremely acceptable in most cases.
batistein
commented
2 years ago This would be very helpful. I have the use case that we want to use talos with cri-o and crun. So a way to install/build talos with another container runtime would be highly appreciated. Is there right now some workaround to achieve this?
smira
commented
2 years ago Replacing containerd with cri-o might be much more tricky for Talos, supporting different container runtime should be now available via system extensions. See https://github.com/talos-systems/extensions
OGtrilliams
commented
9 months ago Hello! My name is Treva, I'm community manager for Kata Containers. I just wanted to reach out to see if there was interest in potentially collaborating on adding Kata Containers support to Talos, as we recently got a request for the feature. If so, feel free to ping me here or email Treva [at] openinfra [dot] dev
rsmitty
commented
9 months ago Hi @OGtrilliams, thanks for reaching out. Kata support is actually already in the works, with @fidencio hacking on this from the Kata side. Excited to see this land. It'll be a very cool feature for Talos! Here's a couple of PRs so that they're linked to this high-level issue:
https://github.com/siderolabs/extensions/pull/279 https://github.com/siderolabs/talos/pull/8038
OGtrilliams
commented
9 months ago Hey @rsmitty thanks for filling me in - that's awesome! I am very excited to work alongside you all & I'm thrilled to hear that the addition is already in-works. Would you be interested in chatting about Talos & Kata Containers/Confidential Containers on an OpenInfra Live episode?
rsmitty
commented
9 months ago Sure, let's get the support landed and I'm sure we'd be happy to hop on with you all and chat :)
frezbo
commented
7 months ago
Feature Request
BLUF: Support for kata containers (qemu and firecracker vm varents), as default or configurable runtime class. Ideally in a way that can support nested virtualization. (Example: talos-os node running virtualized on KVM / Proxmox / OpenStack HyperVisor, should be able to run containers sandboxed using nested VMs)
Description
Details / Why:
cri-o & containerd can both support the following runtimes: runc kata-runtime - qemu VM kata-runtime - firecracker VM gvisor - qemu VM gvisor - ptrace sources: https://youtu.be/HpAVlnl2Z9M?t=1846 https://gvisor.dev/docs/architecture_guide/platforms/#implementations
From lots of google fu it seems that talos and the majority of alternative container optimized OS's have significant benefits in terms of updates, resource usage, and default security hardening. BUT the majority also seem to lack (or at least make it unclear if they support) container sandboxing technologies. I saw a hacker news post (https://news.ycombinator.com/item?id=24345035) for example that mentions BottleRocketOS doesn't support gvisor / kata containers, there's also https://github.com/talos-systems/talos/issues/3023, which makes it look like talos probably doesn't as well.
The only 2 container optimized OS's I'm aware of that seem to be known to support container sandboxing technologies are:
My theoretical understanding (which could very well be wrong, and is based more on reading vs in depth tinkering experience) is that if you have a malicious container running on a Talos Cluster then:
Because of this I wonder if the following combination might end up being more secure than Talos (from the perspective of a multi tenancy cluster with untrusted tenants, that might try to have a rogue container try to break out of a runc sandbox to attack another container):
Any comments on how hard / likely it would be for gVisor and Kata Container to become supported? + comments on if my theoretical understanding is way off (If they're already supported today, it'd be a great thing to advertise on the website + configure some kind of happy path flags to quick enable them.)