Document k0s persistence

[kbumsik]

Hi, I am experimenting k0smotron to implement multi-tenant k8s with remote workers.

While playing around k0smotron, one thing I noticed is that documentation on Cluster.spec.persistence is lacking, while it seems to be important for production deployment.

For example:

• What it actually does - the doc doesn't mention what it persists at all. Just saying "persistence configuration" is very confusing. It seems to mount /var/lib/k0s though.
• Recommended size and performance of the volume
  • dose it need to be high-performance? Suggested IOPS? What is the expected size of the volume? Does it keep growing?
  • In case of etcd, it is easier to figure out the optimal pvc settings. However, it is hard to figure out the hardware requirements of k0s control plane instance.
• Maybe also add why it needs a persistent volume. I thought kube-apiserver is stateless.

Since v1 is out, some insights regarding production deployments like this would really helps.

[nealf]

I'm in the same situation of not knowing whether it is needed or not for non-ephemeral clusters. If it is necessary, we found that it does not currently work in Standalone mode (at least when replicas >1) because it doesn't use the StatefuleSets persistent volume template correctly. It creates the volume template, but the VolumeMount doesn't use the template name and pods are never created because of that config error.