canonical / microk8s

MicroK8s is a small, fast, single-package Kubernetes for datacenters and the edge.
https://microk8s.io
Apache License 2.0
8.41k stars 767 forks source link

hostpath-provisioner pods are not executed in tainted nodes #4385

Open arielly-parussulo opened 7 months ago

arielly-parussulo commented 7 months ago

Summary

When I was trying to create a hostpath-provisioner PVC for my microk8s cluster I noticed that hostpath-provisioner could not create PVCs inside nodes tainted with NoSchedule effect. The problem is that when hostpath-provisioner creates a pod to create the storage path inside the node and the created pod doesn't contain a toleration to allow it to be executed in any node.

What Should Happen Instead?

hostpath-provisioned pods should be executed in any node even with NoScheduled and NoExecuted taints.

Reproduction Steps

  1. Add a node inside your microk8s cluster.
  2. Run microk8s kubectl taint test=true:NoScheduled to taint the node.
  3. Enable hostpath-storage add-ons using microk8s enable hostpath-storage.
  4. Add a pod and a PVC with tolerations:
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: test-pvc
spec:
  accessModes: [ReadWriteOnce]
---
apiVersion: v1
kind: Pod
metadata:
  name: test-nginx
spec:
  volumes:
    - name: pvc
      persistentVolumeClaim:
        claimName: test-pvc
  tolerations:
    - key: "test"
      operator: "Equal"
      value: "true"
      effect: "NoSchedule"
  containers:
    - name: nginx
      image: nginx
      ports:
        - containerPort: 80
      volumeMounts:
        - name: pvc
          mountPath: /usr/share/nginx/html
  1. Run microk8s kubectl -n kube-system get pods to find hostpath-provisioner-<node>-<hash> in Pending state.

Introspection Report

inspection-report.tar.gz

Can you suggest a fix?

hostpath-provisioner-<node>-<hash> could start wih the toleration so it could be scheduled in any node.

tolerations:
  - operator: Equals

Are you interested in contributing with a fix?

Yes.

ktsakalozos commented 7 months ago

Hi @arielly-parussulo, sorry for the late reply. The file you would need to patch is https://github.com/canonical/microk8s-core-addons/blob/main/addons/hostpath-storage/storage.yaml and your work will be available on the next MicroK8s release (1.30). Thank you.