search

IBM / ibm-spectrum-scale-csi

The IBM Spectrum Scale Container Storage Interface (CSI) project enables container orchestrators, such as Kubernetes and OpenShift, to manage the life-cycle of persistent storage.
Apache License 2.0
66 stars 49 forks source link

In k8s env when cg variable enabled for remoteFS gives errors #1013

Closed saurabhwani5 closed 1 year ago

saurabhwani5 commented 1 year ago

Describe the bug

When discover_cg_fileset is enabled for remoteFS it should work as disabled as per design .

How to Reproduce?

  1. Install CSI 2.10.0 with #1004 images on k8s 
    [root@saurabh6-master pr1004]# oc get pods -n ibm-spectrum-scale-csi-driver
NAME                                                  READY   STATUS    RESTARTS      AGE
ibm-spectrum-scale-csi-attacher-6df969845f-md5nq      1/1     Running   0             72m
ibm-spectrum-scale-csi-attacher-6df969845f-rwcz2      1/1     Running   0             72m
ibm-spectrum-scale-csi-operator-546c5dbf98-q5fws      1/1     Running   0             7h52m
ibm-spectrum-scale-csi-ph5b9                          3/3     Running   0             72m
ibm-spectrum-scale-csi-provisioner-d58bf488b-kbq9d    1/1     Running   1 (72m ago)   72m
ibm-spectrum-scale-csi-resizer-76d4cb574f-g254w       1/1     Running   0             72m
ibm-spectrum-scale-csi-snapshotter-569fb5dcb8-bb62k   1/1     Running   1 (63m ago)   72m
ibm-spectrum-scale-csi-vf9vd                          3/3     Running   0             72m
[root@saurabh6-master pr1004]# oc get cso -n ibm-spectrum-scale-csi-driver
NAME                     VERSION   SUCCESS
ibm-spectrum-scale-csi   2.10.0    True
  2. Create configmap as following : 
    [root@saurabh6-master pr1004]# cat cm.yaml
kind: ConfigMap
apiVersion: v1
metadata:
name: ibm-spectrum-scale-csi-config
namespace: ibm-spectrum-scale-csi-driver
data:
VAR_DRIVER_DISCOVER_CG_FILESET: ENABLED
  3. Create two CG filesets as following on any remoteFS : 
    [root@csi-518-scalegui fs1]# ls | grep test
fcaaaa2e-3e86-438b-a371-000000000000-test
fcaaaa2e-3e86-438b-a371-000000000001-test
Command to create independent fileset:
mmcrfileset fs1 31223663-891c-42dd-b826-000000000005-default --inode-space=new -t "Fileset created by IBM Container Storage Interface driver"
mmlinkfileset fs1 31223663-891c-42dd-b826-000000000005-default
  4. Create Version2 pvc , sc and pod as following : 
    
[root@saurabh6-master pr1004]# cat apply.yaml
apiVersion: v1
kind: Pod
metadata:
name: csi-scale-fsetdemo-pod
labels:
app: nginx
spec:
containers:
- name: web-server
 image: nginx
 volumeMounts:
   - name: mypvc
     mountPath: /usr/share/nginx/html/scale
 ports:
 - containerPort: 80
volumes:
- name: mypvc
 persistentVolumeClaim:
   claimName: scale-advance-pvc
   readOnly: false

apiVersion: v1 kind: PersistentVolumeClaim metadata: name: scale-advance-pvc spec: accessModes:

apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: ibm-spectrum-scale-csi-advance provisioner: spectrumscale.csi.ibm.com parameters: volBackendFs: "remotefs" version: "2" reclaimPolicy: Delete

5. Describe the pvc :

[root@saurabh6-master pr1004]# oc describe pvc Name: scale-advance-pvc Namespace: test StorageClass: ibm-spectrum-scale-csi-advance Status: Pending Volume: Labels: Annotations: volume.beta.kubernetes.io/storage-provisioner: spectrumscale.csi.ibm.com volume.kubernetes.io/storage-provisioner: spectrumscale.csi.ibm.com Finalizers: [kubernetes.io/pvc-protection] Capacity: Access Modes: VolumeMode: Filesystem Used By: csi-scale-fsetdemo-pod Events: Type Reason Age From Message

Warning ProvisioningFailed 36m persistentvolume-controller storageclass.storage.k8s.io "ibm-spectrum-scale-csi-advance" not found Warning ProvisioningFailed 7m26s (x15 over 34m) spectrumscale.csi.ibm.com_ibm-spectrum-scale-csi-provisioner-d58bf488b-kbq9d_4f957877-78fd-475d-ba41-f6859717a1c1 failed to provision volume with StorageClass "ibm-spectrum-scale-csi-advance": rpc error: code = Internal desc = conflicting filesets found [fcaaaa2e-3e86-438b-a371-000000000001-test fcaaaa2e-3e86-438b-a371-000000000000-test] Normal Provisioning 2m26s (x16 over 36m) spectrumscale.csi.ibm.com_ibm-spectrum-scale-csi-provisioner-d58bf488b-kbq9d_4f957877-78fd-475d-ba41-f6859717a1c1 External provisioner is provisioning volume for claim "test/scale-advance-pvc" Normal ExternalProvisioning 91s (x141 over 36m) persistentvolume-controller waiting for a volume to be created, either by external provisioner "spectrumscale.csi.ibm.com" or manually created by system administrator



## Expected behavior
PVC should bound for remoteFS irrespective of discover_cg_fileset is enabled or disabled 
If one fileset is present in remotefs it is using that fileset for version 2 while it is expected that it should behave as disabled where fileset should be ownCGname_namespace

And this behavior is same for OCP with remoteFS
amdabhad commented 1 year ago

Fixed the issue and image is update here: https://github.com/IBM/ibm-spectrum-scale-csi/pull/1004#issuecomment-1695580621