search

k3s-io / k3s

Lightweight Kubernetes
https://k3s.io
Apache License 2.0
28.06k stars 2.35k forks source link

single master #16

Closed pierreozoux closed 5 years ago

pierreozoux commented 5 years ago

Hello! Nice project :)

I'd like to replace yunohost with kubernetes, because the API is so much more beautiful than bash :)

In a single node scenario, is flannel still required?

Do you plan to support socket activation for the API, it would be nice :) and maybe having the controller run on Cron instead of events? Just some idea for improvements :)

Do you also support CRI-o ? I think it would further decrease the footprint on small devices.

Do you have more ideas on how to improve it?

What is the current support you provide for this side project? (Just asking, not putting any form of pressure on you, I know how it is to run hoby open source project ;) )

Thanks again for working on that, this is really cool!

ibuildthecloud commented 5 years ago

Answered in slightly different order as asked:

CRI-o is not really supported, I could if you really want it, but by default I use containerd. Containerd is significantly smaller memory footprint than docker and I can't imagine CRI-o can do much better than containerd. But just technology wise containerd is really a superior piece of tech and has more support from cloud providers (google, ibm, aws). I believe pretty much all cloud providers will eventually transition to containerd (and not cri-o which is a heavy redhatlandia project). So, I'd strongly suggest you use containerd. Plain old docker is supported too, if you care.

Flannel is not required for a single host, but some CNI driver is. In terms of memory footprint flannel adds basically nothing, I wouldn't worry about. The way k3s is packaged, flannel is embedded in the kubelet which makes the overhead negligible. Flannel can be disabled but some CNI is required. So you can disable the embedded flannel and install you own but that is more hassle if all you want is just memory. The simplest networking that k8s has is actually kubenet but it requires Docker and docker over containerd will add over 100mb of memory.

I don't support socket activation but would be will to but it has a major downside. Docker supports socket activation but in practice you never want to use it because if you restart the device docker will not know to restart all the containers. So you can't get proper supervision on reboot if it's socket activated. In practice you just want to run this all the time.

I'm always looking for ways to decrease the memory footprint. Right now I think a single node setup is around ~150mb. The easiest way to test that is to run the docker-compose setup and look at the docker stats of the server and nodes. Right now my biggest focus is everything working properly. After I know all is sane I'll try to decrease the memory further. Right now I think k3s single node is really not far off from just running docker. The biggest problem with k3s at the moment for small devices is that startup is slow and it does more writes to disk than say Docker.

This project is going to become a full Rancher Labs support project very soon, wait for an announcement :smile:

pierreozoux commented 5 years ago

Really nice, thank you for the detailed answer. it thought CRI-o would have a smaller mem footprint, but if you say containerd is all good, I'm fine too ;)

About socket activation, I was not thinking about the containers themself.

I'll precise my problem space. Imagine you self host at home. You basically install a WordPress a RocketChat and a Nextcloud. I think that k8s api is better than anything else, so that's why I'm here ;) But basically, once I installed what I wanted, I'll probably not need the k8s api for some time.

In term of solutions for this problem: We could imagine having the k8s api being behind socket activation. So whenever somebody needs to discuss with that, it boots and respond. We could also imagine the controller be started by a cron every 5 mins for instance. These are some ideas, but I don't know enough the internals to be sure that it is feasible in a nice way, and probably you have different objectives with this project.

I think that if it is 150mb overhead, it is already really nice, and I think it is a good starting point!

Thanks again a lot for sharing, I'll close the issue to keep the repo clean, but feel free to respond or discuss here :)

Have a nice day!