openebs-archive / cstor-operators

Collection of OpenEBS cStor Data Engine Operators
https://openebs.io
Apache License 2.0
94 stars 69 forks source link

pod not able to mount PV after the pv was resized #212

Open survivant opened 3 years ago

survivant commented 3 years ago

I followed this guide : https://github.com/openebs/cstor-operators/blob/master/docs/tutorial/volumes/resize.md

After that I deleted the pod. The pod was never able to mount the PV. I had to scale down to 0 the deployment to fix the problem

mittachaitu commented 3 years ago

Can we get output pod describe which is not able to mount? Is it still reproducible in the latest version of OpenEBS? I quick tried to reproduce but didn't hit the issue. Can you try in your setup and close the issue if it no longer reproducible:

Steps I tried in 2.12.0 version of OpenEBS

## PVC
[sai@centos-master ~]$ kubectl get pvc
NAME      STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS     AGE
csi-pvc   Bound    pvc-b29bf7ca-c569-4493-ac95-5cd6bb5fa2c3   10Gi       RWO            openebs-csi-sc   3h17m

## Pod consuming above PVC
[sai@centos-master ~]$ kubectl get po
NAME                                READY   STATUS    RESTARTS   AGE
fio-2-deployment-6c948d6799-hknfd   1/1     Running   0          7m45s

# Expanded the volume and deleted the pod consuming the volume
[sai@centos-master ~]$ kubectl patch pvc csi-pvc --patch '{"spec": {"resources": {"requests": {"storage": "11Gi"}}}}' && kubectl delete po fio-2-deployment-6c948d6799-hknfd
persistentvolumeclaim/csi-pvc patched
pod "fio-2-deployment-6c948d6799-hknfd" deleted
^C
[sai@centos-master ~]$ kubectl get po
NAME                                READY   STATUS        RESTARTS   AGE
fio-2-deployment-6c948d6799-87sk7   1/1     Running       0          17s
fio-2-deployment-6c948d6799-hknfd   1/1     Terminating   0          8m30s

# Again deleted the pod after coming to running state
[sai@centos-master ~]$ kubectl delete po fio-2-deployment-6c948d6799-87sk7
pod "fio-2-deployment-6c948d6799-87sk7" deleted
^C
[sai@centos-master ~]$ kubectl get po -o wide
NAME                                READY   STATUS              RESTARTS   AGE   IP             NODE              NOMINATED NODE   READINESS GATES
fio-2-deployment-6c948d6799-87sk7   1/1     Terminating         0          35s   10.244.1.238   centos-worker-1   <none>           <none>
fio-2-deployment-6c948d6799-fljbp   0/1     ContainerCreating   0          11s   <none>         centos-master     <none>           <none>

# pod came to running state
[sai@centos-master ~]$ kubectl get po
NAME                                READY   STATUS    RESTARTS   AGE
fio-2-deployment-6c948d6799-fljbp   1/1     Running   0          52s