loft-sh / vcluster

vCluster - Create fully functional virtual Kubernetes clusters - Each vcluster runs inside a namespace of the underlying k8s cluster. It's cheaper than creating separate full-blown clusters and it offers better multi-tenancy and isolation than regular namespaces.
https://www.vcluster.com
Apache License 2.0
6.47k stars 412 forks source link

Emptydir /run/k0s creates issues for k0s #648

Closed christiaangoossens closed 2 years ago

christiaangoossens commented 2 years ago

What happened?

After the upgrade from 0.10.2 to 0.11.0, k0s won't start anymore with the following error log:

Error: failed to create /data/k0s/status.sock: directory "/run/k0s" exist, but the permission is 0777. The expected permission is 755

Comparing to an older statefulset, the new one has an emptyDir connected to /run/k0s, while the old one does not. This causes a directory to be created with the wrong chmod.

What did you expect to happen?

vcluster container should just start

How can we reproduce it (as minimally and precisely as possible)?

Install a k0s distro vcluster with version 0.11.0

Anything else we need to know?

No response

Host cluster Kubernetes version

Server Version: version.Info{Major:"1", Minor:"22", GitVersion:"v1.22.8+k0s", GitCommit:"7061dbbf75f9f82e8ab21f9be7e8ffcaae8e0d44", GitTreeState:"clean", BuildDate:"2022-03-24T22:32:09Z", GoVersion:"go1.16.15", Compiler:"gc", Platform:"linux/amd64"}

Host cluster Kubernetes distribution

k0s

vlcuster version

0.11.0

Vcluster Kubernetes distribution(k3s(default)), k8s, k0s)

k0s

OS and Arch

-

christiaangoossens commented 2 years ago

Removing this emptyDir solves the issue temporarily, until a new upgrade is done with the vcluster cli.

FabianKramm commented 2 years ago

@christiaangoossens thanks for reporting this issue! We were adding the empty dir because it caused problems in conjunction with rootless, we'll definitely check the problem though and fix it for the next version. Does this also occur on a new vcluster or only an upgraded one?

christiaangoossens commented 2 years ago

This occurs on both new and upgraded clusters with 0.11.0, I tested that a minute ago.