Closed xiaods closed 9 months ago
use kata to quickstart use vmm on k8s.
Proposed solution: provide another engine for K8E, Kata provides a one-click reference that can work directly with runc. And firecrack does not apply directly to K8S, this limits our design, so, after consideration, it is recommended to unify the management of VMM containers through Kata
follow https://github.com/kata-containers/kata-containers/issues/2207 we need let k8e containerd support devmapper plugin.
[root@ip-172-31-28-183 ~]# k8e ctr plugins list
TYPE ID PLATFORMS STATUS
io.containerd.content.v1 content - ok
io.containerd.snapshotter.v1 aufs linux/amd64 skip
io.containerd.snapshotter.v1 btrfs linux/amd64 skip
io.containerd.snapshotter.v1 devmapper linux/amd64 error
yum install device-mapper-persistent-data
systemctl enable devicemapper
devices {
dir = "/dev/mapper"
scan = [ "/dev/mapper" ]
filter = [ "a|/dev/loop|", "a|/dev/dm-|", "r/.*/" ]
}
systemctl start devicemapper
wget https://github.com/containerd/devmapper/releases/download/v1.1.1/devmapper-1.1.1-linux-amd64.tar.gz
tar -xvf devmapper-1.1.1-linux-amd64.tar.gz
containerd config default | sudo tee /etc/containerd/config.toml
systemctl restart containerd
containerd plugin ls | grep devmapper
在/var/lib/k8e/agent/etc/containerd/confg.toml.tmpl 中配置模板, 然后重启服务后k8e就会加载devmapper plugin snapshotter
[plugins."io.containerd.snapshotter.v1.devmapper"]
root_path = "/var/lib/k8e/agent/containerd"
pool_name = "devpool"
base_image_size = "8192MB"
discard_blocks = true
reference: https://github.com/containerd/containerd/blob/main/docs/snapshotters/devmapper.md
report issue on kata community: https://github.com/kata-containers/kata-containers/issues/6317
report issue on kata community: kata-containers/kata-containers#6317
no response from kata community.
又发现有的库来实现VMM libkrun is a dynamic library that allows programs to easily acquire the ability to run processes in a partially isolated environment using KVM Virtualization.
libkrun的主要目标是在单个动态库中提供所有虚拟化过程隔离所需的功能(包括VMM、模拟设备和Guest内核等),以便可以轻松地在不同命名空间之间切换,并且具有最小可能的内存占用。
我们并不试图成为通用VMM,这使得我们能够减少一些负载,并且专注于此使用案例可以将对外部支持进程和shim的需求降至最低。
我们还正在开发实验性扩展Guest内核,以打破VM和容器之间的障碍,例如透明套接字模仿(Transparent Socket Impersonation),它提供了一定程度的网络命名空间透明度,并会有更多功能推出。
总而言之,libkrun仍处于早期开发阶段,尚未准备好投入生产。因此,如果您需要短期解决方案,则QEMU或Firecracker(与Kata或其他运行时组合)可能是您需要的。
https://kuasar.io/docs/architecture/why-kuasar/ 今天看到这个,直接替换 runc,提供多 sandbox 方案。可以啊。
kuasar 正在申请 cncf sandbox
k8e 替换了 containerd , PR https://github.com/xiaods/k8e/pull/331
Expires, close it
as pure kubernetes distribution, k8e's feature is almost like k3s. we aimed to fast enterprise adoption with serverless technology.
Refs: