search

IBM / ibm-spectrum-scale-csi

The IBM Spectrum Scale Container Storage Interface (CSI) project enables container orchestrators, such as Kubernetes and OpenShift, to manage the life-cycle of persistent storage.
Apache License 2.0
66 stars 49 forks source link

Multiple Requests are coming to CSI for resizing for invalid value #1229

Open saurabhwani5 opened 1 week ago

saurabhwani5 commented 1 week ago

Describe the bug

Around 2733 requests have come to CSI for resizing invalid value which is not expected Normal Resizing 4m26s (x2733 over 3d23h) external-resizer spectrumscale.csi.ibm.com External resizer is resizing volume pvc-0b6af8a8-39d2-43e1-9535-465548222926

How to Reproduce?

  1. Install CSI with following images #1226 

    Operator : quay.io/ibm-spectrum-scale-dev/ibm-spectrum-scale-csi-operator@sha256:1e579401a52f4ae12eff2b0b61b83af12d37d354a80d2ee6b83b9ac6b7cc01e2
Driver : quay.io/badri_pathak/ibm-spectrum-scale-csi-driver:pvc_max_size_v1

  2. Create PVC as following :

    
[root@ocppr1226]# cat pvc.yaml
apiVersion: v1
kind: Pod
metadata:
name: csi-scale-fsetdemo-pod-2
labels:
app: nginx
spec:
containers:
- name: web-server
 image: nginx:1.22.0
 volumeMounts:
   - name: mypvc
     mountPath: /usr/share/nginx/html/scale
 ports:
 - containerPort: 80
volumes:
- name: mypvc
 `persistentVolumeClaim:`
   claimName: scale-advance-pvc-1
   readOnly: false

apiVersion: v1 kind: PersistentVolumeClaim metadata: name: scale-advance-pvc-1 spec: accessModes:

apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: ibm-spectrum-scale-csi-advance provisioner: spectrumscale.csi.ibm.com parameters: volBackendFs: "fs0" version: "2" reclaimPolicy: Delete allowVolumeExpansion: true [root@ocppr1226]# oc apply -f pvc.yaml pod/csi-scale-fsetdemo-pod-2 created persistentvolumeclaim/scale-advance-pvc-1 created storageclass.storage.k8s.io/ibm-spectrum-scale-csi-advance created [root@ocppr1226]# oc get pods NAME READY STATUS RESTARTS AGE csi-scale-fsetdemo-pod-2 1/1 Running 0 62s ibm-spectrum-scale-csi-attacher-5d58777ffc-75c6d 1/1 Running 0 34m ibm-spectrum-scale-csi-attacher-5d58777ffc-d4mpl 1/1 Running 0 34m ibm-spectrum-scale-csi-cfmnq 3/3 Running 0 34m ibm-spectrum-scale-csi-operator-75f546c65c-q8vt2 1/1 Running 0 35m ibm-spectrum-scale-csi-provisioner-7f698474dd-xmhjf 1/1 Running 0 34m ibm-spectrum-scale-csi-resizer-67d499dc54-h892q 1/1 Running 0 34m ibm-spectrum-scale-csi-snapshotter-b9b67b997-5fw2t 1/1 Running 0 34m ibm-spectrum-scale-csi-vzvt8 3/3 Running 0 34m ibm-spectrum-scale-csi-z5sl5 3/3 Running 0 34m [root@ocppr1226]# oc get pvc NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS VOLUMEATTRIBUTESCLASS AGE scale-advance-pvc-1 Bound pvc-0b6af8a8-39d2-43e1-9535-465548222926 1Gi RWX ibm-spectrum-scale-csi-advance 63s [root@ocppr1226]#


3. Increase PVC size to 9223372036854775807Gi:

apiVersion: v1 kind: PersistentVolumeClaim metadata: name: scale-advance-pvc-1 spec: accessModes:

  1. Check PVC description after 3 days

[root@api.cnsa-badripathak710.cp.fyre.ibm.com ~]# oc describe pvc Name: scale-advance-pvc-1 Namespace: ibm-spectrum-scale-csi StorageClass: ibm-spectrum-scale-csi-advance Status: Bound Volume: pvc-0b6af8a8-39d2-43e1-9535-465548222926 Labels: Annotations: pv.kubernetes.io/bind-completed: yes pv.kubernetes.io/bound-by-controller: yes volume.beta.kubernetes.io/storage-provisioner: spectrumscale.csi.ibm.com volume.kubernetes.io/storage-provisioner: spectrumscale.csi.ibm.com Finalizers: [kubernetes.io/pvc-protection] Capacity: 1Gi Access Modes: RWX VolumeMode: Filesystem Used By: csi-scale-fsetdemo-pod-2 Conditions: Type Status LastProbeTime LastTransitionTime Reason Message

Resizing True Mon, 01 Jan 0001 00:00:00 +0000 Sun, 06 Oct 2024 23:10:20 -0700 Events: Type Reason Age From Message

Normal Resizing 2m45s (x2735 over 3d23h) external-resizer spectrumscale.csi.ibm.com External resizer is resizing volume pvc-0b6af8a8-39d2-43e1-9535-465548222926 [root@api.cnsa-badripathak710.cp.fyre.ibm.com ~]#


## Expected behavior
Multiple resizing request should not come to CSI for invalid value 

### Data Collection and Debugging

/scale-csi/D.1229 mustgather.tar.gz

badri-pathak commented 1 week ago

Hi @saurabhwani5 The multiple requests are expected to come on the failure, as the VolumeExpansion will reject the request having wrong volume capacity. This is expected, so we are not able to address this issue, because handled by kubernetes itself.