search

IBM / ibm-spectrum-scale-csi

The IBM Spectrum Scale Container Storage Interface (CSI) project enables container orchestrators, such as Kubernetes and OpenShift, to manage the life-cycle of persistent storage.
Apache License 2.0
66 stars 49 forks source link

Cross cluster shallow copy is not supported #1078

Closed saurabhwani5 closed 9 months ago

saurabhwani5 commented 9 months ago

Describe the bug

When we try to restore snapshot with readonlymany(shallow copy volume creation) from local filesystem to remote filesystem then that is not supported currently and vice versa. This can be improvement in shallow copy feature.

How to Reproduce?

  1. Install CSI 2.11 with driver images from #1067 :

    root@saurabhmultiguiubu-master:~# oc get pods
NAME                                                  READY   STATUS    RESTARTS       AGE
ibm-spectrum-scale-csi-attacher-869bd7ff6d-hl2c2      1/1     Running   0              154m
ibm-spectrum-scale-csi-attacher-869bd7ff6d-rcz5w      1/1     Running   0              154m
ibm-spectrum-scale-csi-bf8h4                          3/3     Running   0              154m
ibm-spectrum-scale-csi-hcrqv                          3/3     Running   0              154m
ibm-spectrum-scale-csi-operator-f9f57f54c-84xfk       1/1     Running   0              155m
ibm-spectrum-scale-csi-provisioner-c48d8df47-hbxl2    1/1     Running   0              154m
ibm-spectrum-scale-csi-resizer-54c67667c4-8p44q       1/1     Running   1 (154m ago)   154m
ibm-spectrum-scale-csi-snapshotter-6f4964bd9b-dbv2d   1/1     Running   1 (107m ago)   154m
root@saurabhmultiguiubu-master:~# oc get cso
NAME                     VERSION   SUCCESS
ibm-spectrum-scale-csi   2.11.0    True
root@saurabhmultiguiubu-master:~# oc describe pod | grep quay
Image:         quay.io/hemalatha_gajendran/driver_shallowcopy_withoutfileset
Image ID:      quay.io/hemalatha_gajendran/driver_shallowcopy_withoutfileset@sha256:8f1d9b7d5c80ca2b7c173584ca97d2587143f8e84d5c7bfa48d3618e43e9119b
Image:         quay.io/hemalatha_gajendran/driver_shallowcopy_withoutfileset
Image ID:      quay.io/hemalatha_gajendran/driver_shallowcopy_withoutfileset@sha256:8f1d9b7d5c80ca2b7c173584ca97d2587143f8e84d5c7bfa48d3618e43e9119b
Image:         quay.io/ibm-spectrum-scale-dev/ibm-spectrum-scale-csi-operator@sha256:18264e0c9c112856bc2744f7f971a4b60ecb24de57e46af4d35456dcdf8e3cbf
Image ID:      quay.io/ibm-spectrum-scale-dev/ibm-spectrum-scale-csi-operator@sha256:18264e0c9c112856bc2744f7f971a4b60ecb24de57e46af4d35456dcdf8e3cbf
  CSI_DRIVER_IMAGE:      quay.io/hemalatha_gajendran/driver_shallowcopy_withoutfileset

  2. Create PVC on local cluster :

    
k8s# cat apply.yaml
apiVersion: v1
kind: Pod
metadata:
name: csi-scale-fsetdemo-pod-1
labels:
app: nginx
spec:
containers:
- name: web-server
 image: nginx:1.22.0
 volumeMounts:
   - name: mypvc
     mountPath: /usr/share/nginx/html/scale
 ports:
 - containerPort: 80
volumes:
- name: mypvc
 persistentVolumeClaim:
   claimName: scale-pvc-1
   readOnly: false

apiVersion: v1 kind: PersistentVolumeClaim metadata: name: scale-pvc-1 spec: accessModes:

apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: ibm-spectrum-scale-csi provisioner: spectrumscale.csi.ibm.com parameters: volBackendFs: "fs1" reclaimPolicy: Delete k8s# oc apply -f apply.yaml pod/csi-scale-fsetdemo-pod-1 created persistentvolumeclaim/scale-pvc-1 created storageclass.storage.k8s.io/ibm-spectrum-scale-csi created

3. Write data inside volume:

k8s# oc get pvc NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE scale-pvc-1 Bound pvc-8b4f6188-9699-4f2a-a9ac-e9b6e6550aa6 1Gi RWX ibm-spectrum-scale-csi 98s k8s# oc get pods NAME READY STATUS RESTARTS AGE csi-scale-fsetdemo-pod-1 1/1 Running 0 102s root@csi-scale-fsetdemo-pod-1:/usr/share/nginx/html/scale# touch test{1..10} root@csi-scale-fsetdemo-pod-1:/usr/share/nginx/html/scale# ls test1 test10 test2 test3 test4 test5 test6 test7 test8 test9

4. Take snapshot of the pvc:

k8s# cat snapshot.yaml apiVersion: snapshot.storage.k8s.io/v1 kind: VolumeSnapshot metadata: name: ibm-spectrum-scale-snapshot spec: volumeSnapshotClassName: ibm-spectrum-scale-snapshotclass source: persistentVolumeClaimName: scale-pvc-1

apiVersion: snapshot.storage.k8s.io/v1 kind: VolumeSnapshotClass metadata: name: ibm-spectrum-scale-snapshotclass driver: spectrumscale.csi.ibm.com parameters: snapWindow: "30" #Optional : Time in minutes (default=30) deletionPolicy: Delete k8s# oc apply -f snapshot.yaml volumesnapshot.snapshot.storage.k8s.io/ibm-spectrum-scale-snapshot created volumesnapshotclass.snapshot.storage.k8s.io/ibm-spectrum-scale-snapshotclass created

5. Create shallow copy in remote cluster:

k8s# cat shallowcopy.yaml apiVersion: v1 kind: Pod metadata: name: csi-scale-fsetdemo-pod-remote labels: app: nginx spec: containers:

apiVersion: v1 kind: PersistentVolumeClaim metadata: name: scale-advance-pvc-remote spec: accessModes:

apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: ibm-spectrum-scale-csi-remote provisioner: spectrumscale.csi.ibm.com parameters: volBackendFs: "remotetest" reclaimPolicy: Delete k8s# oc apply -f shallowcopy.yaml pod/csi-scale-fsetdemo-pod-remote created persistentvolumeclaim/scale-advance-pvc-remote created storageclass.storage.k8s.io/ibm-spectrum-scale-csi-remote created

6. Check the shallow copy PVC description

k8s# oc get pvc NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE scale-advance-pvc-remote Pending ibm-spectrum-scale-csi-remote 3m25s scale-pvc-1 Bound pvc-8b4f6188-9699-4f2a-a9ac-e9b6e6550aa6 1Gi RWX ibm-spectrum-scale-csi 10m k8s# oc describe pvc scale-advance-pvc-remote Name: scale-advance-pvc-remote Namespace: ibm-spectrum-scale-csi-driver StorageClass: ibm-spectrum-scale-csi-remote Status: Pending Volume: Labels: Annotations: volume.beta.kubernetes.io/storage-provisioner: spectrumscale.csi.ibm.com volume.kubernetes.io/storage-provisioner: spectrumscale.csi.ibm.com Finalizers: [kubernetes.io/pvc-protection] Capacity: Access Modes: VolumeMode: Filesystem DataSource: APIGroup: snapshot.storage.k8s.io Kind: VolumeSnapshot Name: ibm-spectrum-scale-snapshot Used By: csi-scale-fsetdemo-pod-remote Events: Type Reason Age From Message

Warning ProvisioningFailed 3m34s persistentvolume-controller storageclass.storage.k8s.io "ibm-spectrum-scale-csi-remote" not found Normal Provisioning 13s (x9 over 3m23s) spectrumscale.csi.ibm.com_ibm-spectrum-scale-csi-provisioner-c48d8df47-hbxl2_d3cb0bfd-0768-460b-9e90-1f5e1f417736 External provisioner is provisioning volume for claim "ibm-spectrum-scale-csi-driver/scale-advance-pvc-remote" Normal ExternalProvisioning 8s (x15 over 3m23s) persistentvolume-controller waiting for a volume to be created, either by external provisioner "spectrumscale.csi.ibm.com" or manually created by system administrator Warning ProvisioningFailed 2s (x9 over 3m17s) spectrumscale.csi.ibm.com_ibm-spectrum-scale-csi-provisioner-c48d8df47-hbxl2_d3cb0bfd-0768-460b-9e90-1f5e1f417736 failed to provision volume with StorageClass "ibm-spectrum-scale-csi-remote": rpc error: code = Unimplemented desc = creating volume from snapshot across clusters is not supported k8s#



## Expected behavior
Shallow copy should be created in cross cluster as there is no copy operation

## Logs 

scale-csi/D.1078
csisnap.tar.gz
saurabhwani5 commented 9 months ago

we are not adding support for cross cluster shallow copy creation in CSI 2.11.0 and it will give error as creating volume from snapshot across clusters is not supported