search

IBM / ibm-spectrum-scale-csi

The IBM Spectrum Scale Container Storage Interface (CSI) project enables container orchestrators, such as Kubernetes and OpenShift, to manage the life-cycle of persistent storage.
Apache License 2.0
66 stars 49 forks source link

correct error not shown in case of volumesnapshot of dependent fileset based pvc #400

Closed vrushch closed 8 months ago

vrushch commented 3 years ago

Describe the bug

In case of volumesnapshot of PVC with dependent Storageclass , snapshotter logs do not show following error message
"Volume snapshot can only be created when source volume is independent fileset" whereas in case of LW storageclass above error message is shown. Above message should be shown as user can find out why its volumesnapshot is not in readytouse True state

To Reproduce Trying to create snapshot of PVC with Dependent StorageClass

[root@csiscale-master functional-tests]# kubectl get sc
NAME                PROVISIONER                 RECLAIMPOLICY   VOLUMEBINDINGMODE   ALLOWVOLUMEEXPANSION   AGE
sc-epic-tiger-138   spectrumscale.csi.ibm.com   Delete          Immediate           false                  88s
[root@csiscale-master functional-tests]#
[root@csiscale-master functional-tests]# kubectl describe sc sc-epic-tiger-138
Name:                  sc-epic-tiger-138
IsDefaultClass:        No
Annotations:           <none>
Provisioner:           spectrumscale.csi.ibm.com
Parameters:            clusterId=7024025311744559206,filesetType=dependent,volBackendFs=gpfs0
AllowVolumeExpansion:  <unset>
MountOptions:          <none>
ReclaimPolicy:         Delete
VolumeBindingMode:     Immediate
Events:                <none>
[root@csiscale-master functional-tests]#
[root@csiscale-master functional-tests]#
[root@csiscale-master functional-tests]# kubectl  get pvc
NAME                       STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS        AGE
pvc-youthful-peacock-133   Bound    pvc-89398cfb-5e00-4d07-92d4-7fc1c657292f   1Gi        RWX            sc-epic-tiger-138   2m27s
[root@csiscale-master functional-tests]#
[root@csiscale-master functional-tests]#
[root@csiscale-master functional-tests]# kubectl get volumesnapshotclass
NAME                         DRIVER                      DELETIONPOLICY   AGE
vsclass-eager-elephant-437   spectrumscale.csi.ibm.com   Delete           3m14s
[root@csiscale-master functional-tests]#
[root@csiscale-master functional-tests]# kubectl describe volumesnapshotclass
Name:             vsclass-eager-elephant-437
Namespace:
Labels:           <none>
Annotations:      <none>
API Version:      snapshot.storage.k8s.io/v1
Deletion Policy:  Delete
Driver:           spectrumscale.csi.ibm.com
Kind:             VolumeSnapshotClass
Metadata:
  Creation Timestamp:  2021-03-25T04:33:38Z
  Generation:          1
  Managed Fields:
    API Version:  snapshot.storage.k8s.io/v1
    Fields Type:  FieldsV1
    fieldsV1:
      f:deletionPolicy:
      f:driver:
    Manager:         OpenAPI-Generator
    Operation:       Update
    Time:            2021-03-25T04:33:38Z
  Resource Version:  11619485
  UID:               eb28cbba-c698-4faf-837f-bea408531196
Events:              <none>
[root@csiscale-master functional-tests]#
[root@csiscale-master functional-tests]# kubectl get volumesnapshot
NAME                        READYTOUSE   SOURCEPVC                  SOURCESNAPSHOTCONTENT   RESTORESIZE   SNAPSHOTCLASS                SNAPSHOTCONTENT                                    CREATIONTIME   AGE
vs-charming-dolphin-180-0   false        pvc-youthful-peacock-133                                         vsclass-eager-elephant-437   snapcontent-9f5cf798-900d-4a39-b149-0727c4a6e9dc                  3m49s
[root@csiscale-master functional-tests]#

Logs in Snapshotter pod

E0325 04:33:38.342899       1 snapshot_controller.go:105] createSnapshot for content [snapcontent-9f5cf798-900d-4a39-b149-0727c4a6e9dc]: error occurred in createSnapshotWrapper: failed to add VolumeSnapshotBeingCreated annotation on the content snapcontent-9f5cf798-900d-4a39-b149-0727c4a6e9dc: "snapshot controller failed to update snapcontent-9f5cf798-900d-4a39-b149-0727c4a6e9dc on API server: Operation cannot be fulfilled on volumesnapshotcontents.snapshot.storage.k8s.io \"snapcontent-9f5cf798-900d-4a39-b149-0727c4a6e9dc\": the object has been modified; please apply your changes to the latest version and try again"
E0325 04:33:38.342926       1 snapshot_controller_base.go:264] could not sync content "snapcontent-9f5cf798-900d-4a39-b149-0727c4a6e9dc": failed to add VolumeSnapshotBeingCreated annotation on the content snapcontent-9f5cf798-900d-4a39-b149-0727c4a6e9dc: "snapshot controller failed to update snapcontent-9f5cf798-900d-4a39-b149-0727c4a6e9dc on API server: Operation cannot be fulfilled on volumesnapshotcontents.snapshot.storage.k8s.io \"snapcontent-9f5cf798-900d-4a39-b149-0727c4a6e9dc\": the object has been modified; please apply your changes to the latest version and try again"
I0325 04:33:38.342964       1 snapshot_controller_base.go:166] Failed to sync content "snapcontent-9f5cf798-900d-4a39-b149-0727c4a6e9dc", will retry again: failed to add VolumeSnapshotBeingCreated annotation on the content snapcontent-9f5cf798-900d-4a39-b149-0727c4a6e9dc: "snapshot controller failed to update snapcontent-9f5cf798-900d-4a39-b149-0727c4a6e9dc on API server: Operation cannot be fulfilled on volumesnapshotcontents.snapshot.storage.k8s.io \"snapcontent-9f5cf798-900d-4a39-b149-0727c4a6e9dc\": the object has been modified; please apply your changes to the latest version and try again"
I0325 04:33:38.342979       1 snapshot_controller_base.go:177] syncContentByKey[snapcontent-9f5cf798-900d-4a39-b149-0727c4a6e9dc]
I0325 04:33:38.343000       1 util.go:297] storeObjectUpdate updating content "snapcontent-9f5cf798-900d-4a39-b149-0727c4a6e9dc" with version 11619490
I0325 04:33:38.343011       1 snapshot_controller.go:56] synchronizing VolumeSnapshotContent[snapcontent-9f5cf798-900d-4a39-b149-0727c4a6e9dc]
I0325 04:33:38.343016       1 snapshot_controller.go:540] Check if VolumeSnapshotContent[snapcontent-9f5cf798-900d-4a39-b149-0727c4a6e9dc] should be deleted.
I0325 04:33:38.343021       1 snapshot_controller.go:75] syncContent: Call CreateSnapshot for content snapcontent-9f5cf798-900d-4a39-b149-0727c4a6e9dc
I0325 04:33:38.343025       1 snapshot_controller.go:101] createSnapshot for content [snapcontent-9f5cf798-900d-4a39-b149-0727c4a6e9dc]: started
I0325 04:33:38.343030       1 snapshot_controller.go:267] createSnapshotWrapper: Creating snapshot for content snapcontent-9f5cf798-900d-4a39-b149-0727c4a6e9dc through the plugin ...
I0325 04:33:38.343035       1 snapshot_controller.go:180] getCSISnapshotInput for content [snapcontent-9f5cf798-900d-4a39-b149-0727c4a6e9dc]
I0325 04:33:38.343039       1 snapshot_controller.go:448] getSnapshotClass: VolumeSnapshotClassName [vsclass-eager-elephant-437]
I0325 04:33:38.343047       1 snapshot_controller.go:577] setAnnVolumeSnapshotBeingCreated: set annotation [snapshot.storage.kubernetes.io/volumesnapshot-being-created:yes] on content [snapcontent-9f5cf798-900d-4a39-b149-0727c4a6e9dc].
I0325 04:33:38.343455       1 event.go:282] Event(v1.ObjectReference{Kind:"VolumeSnapshotContent", Namespace:"", Name:"", UID:"", APIVersion:"snapshot.storage.k8s.io/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'SnapshotCreationFailed' Failed to create snapshot: failed to add VolumeSnapshotBeingCreated annotation on the content snapcontent-9f5cf798-900d-4a39-b149-0727c4a6e9dc: "snapshot controller failed to update snapcontent-9f5cf798-900d-4a39-b149-0727c4a6e9dc on API server: Operation cannot be fulfilled on volumesnapshotcontents.snapshot.storage.k8s.io \"snapcontent-9f5cf798-900d-4a39-b149-0727c4a6e9dc\": the object has been modified; please apply your changes to the latest version and try again"
I0325 04:33:38.349721       1 util.go:297] storeObjectUpdate updating content "snapcontent-9f5cf798-900d-4a39-b149-0727c4a6e9dc" with version 11619494
I0325 04:33:38.349750       1 snapshot_controller.go:592] setAnnVolumeSnapshotBeingCreated: volume snapshot content &{TypeMeta:{Kind: APIVersion:} ObjectMeta:{Name:snapcontent-9f5cf798-900d-4a39-b149-0727c4a6e9dc GenerateName: Namespace: SelfLink: UID:c24c5915-ca04-4cb9-a273-b2618bfdce9d ResourceVersion:11619494 Generation:1 CreationTimestamp:2021-03-25 04:33:38 +0000 UTC DeletionTimestamp:<nil> DeletionGracePeriodSeconds:<nil> Labels:map[] Annotations:map[snapshot.storage.kubernetes.io/volumesnapshot-being-created:yes] OwnerReferences:[] Finalizers:[snapshot.storage.kubernetes.io/volumesnapshotcontent-bound-protection] ClusterName: ManagedFields:[{Manager:csi-snapshotter Operation:Update APIVersion:snapshot.storage.k8s.io/v1 Time:2021-03-25 04:33:38 +0000 UTC FieldsType:FieldsV1 FieldsV1:{"f:metadata":{"f:annotations":{".":{},"f:snapshot.storage.kubernetes.io/volumesnapshot-being-created":{}}}}} {Manager:snapshot-controller Operation:Update APIVersion:snapshot.storage.k8s.io/v1 Time:2021-03-25 04:33:38 +0000 UTC FieldsType:FieldsV1 FieldsV1:{"f:metadata":{"f:finalizers":{".":{},"v:\"snapshot.storage.kubernetes.io/volumesnapshotcontent-bound-protection\"":{}}},"f:spec":{".":{},"f:deletionPolicy":{},"f:driver":{},"f:source":{".":{},"f:volumeHandle":{}},"f:volumeSnapshotClassName":{},"f:volumeSnapshotRef":{".":{},"f:apiVersion":{},"f:kind":{},"f:name":{},"f:namespace":{},"f:resourceVersion":{},"f:uid":{}}}}}]} Spec:{VolumeSnapshotRef:{Kind:VolumeSnapshot Namespace:ibm-spectrum-scale-csi-driver Name:vs-charming-dolphin-180-0 UID:9f5cf798-900d-4a39-b149-0727c4a6e9dc APIVersion:snapshot.storage.k8s.io/v1 ResourceVersion:11619486 FieldPath:} DeletionPolicy:Delete Driver:spectrumscale.csi.ibm.com VolumeSnapshotClassName:0xc0005c1850 Source:{VolumeHandle:0xc0005c1840 SnapshotHandle:<nil>}} Status:<nil>}
I0325 04:33:38.349989       1 snapshotter.go:56] CSI CreateSnapshot: snapshot-9f5cf798-900d-4a39-b149-0727c4a6e9dc
I0325 04:33:38.350000       1 snapshot_controller_base.go:143] enqueued "snapcontent-9f5cf798-900d-4a39-b149-0727c4a6e9dc" for sync
I0325 04:33:38.350022       1 connection.go:182] GRPC call: /csi.v1.Identity/GetPluginInfo
I0325 04:33:38.350028       1 connection.go:183] GRPC request: {}
I0325 04:33:38.350820       1 connection.go:185] GRPC response: {"name":"spectrumscale.csi.ibm.com","vendor_version":"2.2.0"}
I0325 04:33:38.350875       1 connection.go:186] GRPC error: <nil>
I0325 04:33:38.350894       1 connection.go:182] GRPC call: /csi.v1.Controller/CreateSnapshot
I0325 04:33:38.350899       1 connection.go:183] GRPC request: {"name":"snapshot-9f5cf798-900d-4a39-b149-0727c4a6e9dc","source_volume_id":"7024025311744559206;0A0B6135:5FE30B08;filesetName=pvc-89398cfb-5e00-4d07-92d4-7fc1c657292f;path=/ibm/gpfs0/spectrum-scale-csi-volume-store/.volumes/pvc-89398cfb-5e00-4d07-92d4-7fc1c657292f"}
I0325 04:33:38.385061       1 connection.go:185] GRPC response: {}
I0325 04:33:38.385229       1 connection.go:186] GRPC error: rpc error: code = FailedPrecondition desc = the version of Spectrum Scale on cluster is 5.1.0-0. Min required Spectrum Scale version for snapshot support with CSI is 5.1.1-0
I0325 04:33:38.385274       1 snapshot_controller.go:301] createSnapshotWrapper: CreateSnapshot for content snapcontent-9f5cf798-900d-4a39-b149-0727c4a6e9dc returned error: rpc error: code = FailedPrecondition desc = the version of Spectrum Scale on cluster is 5.1.0-0. Min required Spectrum Scale version for snapshot support with CSI is 5.1.1-0
I0325 04:33:38.385330       1 snapshot_controller.go:141] updateContentStatusWithEvent[snapcontent-9f5cf798-900d-4a39-b149-0727c4a6e9dc]
I0325 04:33:38.390017       1 snapshot_controller.go:165] updating VolumeSnapshotContent[snapcontent-9f5cf798-900d-4a39-b149-0727c4a6e9dc] error status failed Operation cannot be fulfilled on volumesnapshotcontents.snapshot.storage.k8s.io "snapcontent-9f5cf798-900d-4a39-b149-0727c4a6e9dc": the object has been modified; please apply your changes to the latest version and try again
E0325 04:33:38.390107       1 snapshot_controller.go:105] createSnapshot for content [snapcontent-9f5cf798-900d-4a39-b149-0727c4a6e9dc]: error occurred in createSnapshotWrapper: failed to take snapshot of the volume, 7024025311744559206;0A0B6135:5FE30B08;filesetName=pvc-89398cfb-5e00-4d07-92d4-7fc1c657292f;path=/ibm/gpfs0/spectrum-scale-csi-volume-store/.volumes/pvc-89398cfb-5e00-4d07-92d4-7fc1c657292f: %!q(<nil>)
E0325 04:33:38.390124       1 snapshot_controller_base.go:264] could not sync content "snapcontent-9f5cf798-900d-4a39-b149-0727c4a6e9dc": failed to take snapshot of the volume, 7024025311744559206;0A0B6135:5FE30B08;filesetName=pvc-89398cfb-5e00-4d07-92d4-7fc1c657292f;path=/ibm/gpfs0/spectrum-scale-csi-volume-store/.volumes/pvc-89398cfb-5e00-4d07-92d4-7fc1c657292f: %!q(<nil>)
I0325 04:33:38.390192       1 snapshot_controller_base.go:166] Failed to sync content "snapcontent-9f5cf798-900d-4a39-b149-0727c4a6e9dc", will retry again: failed to take snapshot of the volume, 7024025311744559206;0A0B6135:5FE30B08;filesetName=pvc-89398cfb-5e00-4d07-92d4-7fc1c657292f;path=/ibm/gpfs0/spectrum-scale-csi-volume-store/.volumes/pvc-89398cfb-5e00-4d07-92d4-7fc1c657292f: %!q(<nil>)
I0325 04:33:38.390211       1 snapshot_controller_base.go:177] syncContentByKey[snapcontent-9f5cf798-900d-4a39-b149-0727c4a6e9dc]
I0325 04:33:38.390271       1 util.go:297] storeObjectUpdate updating content "snapcontent-9f5cf798-900d-4a39-b149-0727c4a6e9dc" with version 11619494
I0325 04:33:38.390286       1 snapshot_controller.go:56] synchronizing VolumeSnapshotContent[snapcontent-9f5cf798-900d-4a39-b149-0727c4a6e9dc]

Trying to create snapshot of PVC with LW StorageClass

[root@csiscale-master functional-tests]# kubectl get sc
NAME                   PROVISIONER                 RECLAIMPOLICY   VOLUMEBINDINGMODE   ALLOWVOLUMEEXPANSION   AGE
sc-vigilant-wolf-918   spectrumscale.csi.ibm.com   Delete          Immediate           false                  33m
[root@csiscale-master functional-tests]#

[root@csiscale-master functional-tests]# kubectl describe sc
Name:                  sc-vigilant-wolf-918
IsDefaultClass:        No
Annotations:           <none>
Provisioner:           spectrumscale.csi.ibm.com
Parameters:            volBackendFs=gpfs0,volDirBasePath=LW
AllowVolumeExpansion:  <unset>
MountOptions:          <none>
ReclaimPolicy:         Delete
VolumeBindingMode:     Immediate
Events:                <none>
[root@csiscale-master functional-tests]#
[root@csiscale-master functional-tests]# kubectl get pvc
NAME                     STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS           AGE
pvc-frosty-chicken-705   Bound    pvc-c8584144-d4a6-4a11-9731-0fa917c9ddaf   1Gi        RWX            sc-vigilant-wolf-918   33m
[root@csiscale-master functional-tests]#
[root@csiscale-master functional-tests]#
[root@csiscale-master functional-tests]# kubectl get volumesnapshotclass
NAME                         DRIVER                      DELETIONPOLICY   AGE
vsclass-eloquent-sheep-628   spectrumscale.csi.ibm.com   Delete           32m
[root@csiscale-master functional-tests]#
[root@csiscale-master functional-tests]# kubectl describe volumesnapshotclass
Name:             vsclass-eloquent-sheep-628
Namespace:
Labels:           <none>
Annotations:      <none>
API Version:      snapshot.storage.k8s.io/v1
Deletion Policy:  Delete
Driver:           spectrumscale.csi.ibm.com
Kind:             VolumeSnapshotClass
Metadata:
  Creation Timestamp:  2021-03-25T04:51:40Z
  Generation:          1
  Managed Fields:
    API Version:  snapshot.storage.k8s.io/v1
    Fields Type:  FieldsV1
    fieldsV1:
      f:deletionPolicy:
      f:driver:
    Manager:         OpenAPI-Generator
    Operation:       Update
    Time:            2021-03-25T04:51:40Z
  Resource Version:  11623079
  UID:               57ecd447-aa69-4407-8a43-648c2ba194d6
Events:              <none>
[root@csiscale-master functional-tests]# kubectl get volumesnapshot
NAME                    READYTOUSE   SOURCEPVC                SOURCESNAPSHOTCONTENT   RESTORESIZE   SNAPSHOTCLASS                SNAPSHOTCONTENT                                    CREATIONTIME   AGE
vs-goofy-monkey-652-0   false        pvc-frosty-chicken-705                                         vsclass-eloquent-sheep-628   snapcontent-3a87b7df-5cdf-484a-a569-bfa59ae24626                  33m
[root@csiscale-master functional-tests]#

Logs in Snapshotter pod

I0325 05:26:51.674008       1 event.go:282] Event(v1.ObjectReference{Kind:"VolumeSnapshotContent", Namespace:"", Name:"", UID:"", APIVersion:"snapshot.storage.k8s.io/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'SnapshotCreationFailed' Failed to create snapshot: failed to take snapshot of the volume, 7024025311744559206;0A0B6135:5FE30B08;path=/ibm/gpfs0/spectrum-scale-csi-volume-store/.volumes/pvc-c8584144-d4a6-4a11-9731-0fa917c9ddaf: %!q(<nil>)
I0325 05:26:51.674573       1 connection.go:185] GRPC response: {"name":"spectrumscale.csi.ibm.com","vendor_version":"2.2.0"}
I0325 05:26:51.674609       1 connection.go:186] GRPC error: <nil>
I0325 05:26:51.674619       1 connection.go:182] GRPC call: /csi.v1.Controller/CreateSnapshot
I0325 05:26:51.674624       1 connection.go:183] GRPC request: {"name":"snapshot-3a87b7df-5cdf-484a-a569-bfa59ae24626","source_volume_id":"7024025311744559206;0A0B6135:5FE30B08;path=/ibm/gpfs0/spectrum-scale-csi-volume-store/.volumes/pvc-c8584144-d4a6-4a11-9731-0fa917c9ddaf"}
I0325 05:26:51.675019       1 connection.go:185] GRPC response: {}
I0325 05:26:51.675048       1 connection.go:186] GRPC error: rpc error: code = InvalidArgument desc = CreateSnapshot - volume [7024025311744559206;0A0B6135:5FE30B08;path=/ibm/gpfs0/spectrum-scale-csi-volume-store/.volumes/pvc-c8584144-d4a6-4a11-9731-0fa917c9ddaf] - Volume snapshot can only be created when source volume is independent fileset
I0325 05:26:51.675065       1 snapshot_controller.go:301] createSnapshotWrapper: CreateSnapshot for content snapcontent-3a87b7df-5cdf-484a-a569-bfa59ae24626 returned error: rpc error: code = InvalidArgument desc = CreateSnapshot - volume [7024025311744559206;0A0B6135:5FE30B08;path=/ibm/gpfs0/spectrum-scale-csi-volume-store/.volumes/pvc-c8584144-d4a6-4a11-9731-0fa917c9ddaf] - Volume snapshot can only be created when source volume is independent fileset

Expected behavior In case of volumesnapshot of PVC with dependent Storageclass , following error message should be shown "Volume snapshot can only be created when source volume is independent fileset"

Environment

# Deployment
[root@csiscale-master functional-tests]# kubectl version
Client Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.1", GitCommit:"c4d752765b3bbac2237bf87cf0b1c2e307844666", GitTreeState:"clean", BuildDate:"2020-12-18T12:09:25Z", GoVersion:"go1.15.5", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.1", GitCommit:"c4d752765b3bbac2237bf87cf0b1c2e307844666", GitTreeState:"clean", BuildDate:"2020-12-18T12:00:47Z", GoVersion:"go1.15.5", Compiler:"gc", Platform:"linux/amd64"}
[root@csiscale-master functional-tests]#
[root@csiscale-master functional-tests]# rpm -qa | grep gpfs
gpfs.msg.en_US-5.1.0-0.noarch
gpfs.java-5.1.0-0.x86_64
gpfs.license.adv-5.1.0-0.x86_64
gpfs.gss.pmcollector-5.1.0-0.el7.x86_64
gpfs.nfs-ganesha-gpfs-2.7.5-ibm066.00.el7.x86_64
gpfs.bda-integration-1.0.2-0.noarch
gpfs.smb-4.12.6_gpfs_3-1.el7.x86_64
gpfs.gpl-5.1.0-0.noarch
gpfs.docs-5.1.0-0.noarch
gpfs.compression-5.1.0-0.x86_64
gpfs.afm.cos-1.0.0-0.x86_64
gpfs.crypto-5.1.0-0.x86_64
gpfs.gss.pmsensors-5.1.0-0.el7.x86_64
gpfs.nfs-ganesha-2.7.5-ibm066.00.el7.x86_64
gpfs.pm-ganesha-10.0.0-2.el7.x86_64
gpfs.protocols-support-5.1.0-0.noarch
gpfs.base-5.1.0-0.x86_64
gpfs.gskit-8.0.55-12.x86_64
gpfs.adv-5.1.0-0.x86_64
gpfs.librdkafka-5.1.0-0.x86_64
gpfs.nfs-ganesha-utils-2.7.5-ibm066.00.el7.x86_64
[root@csiscale-master functional-tests]#
deeghuge commented 8 months ago

@saurabhwani5 please recheck this one

saurabhwani5 commented 8 months ago

we can see error logs in snapshotter pod and snapshot description as following for dependent fileset as following snapshot sidecar pods logs :

I0205 08:49:08.082328       1 snapshot_controller_base.go:162] enqueued "snapcontent-6a710a10-a399-497c-a61f-67c30d94f0f0" for sync
I0205 08:49:08.082447       1 snapshot_controller.go:648] Removed VolumeSnapshotBeingCreated annotation from volume snapshot content snapcontent-6a710a10-a399-497c-a61f-67c30d94f0f0
I0205 08:49:08.082464       1 util.go:216] storeObjectUpdate updating content "snapcontent-6a710a10-a399-497c-a61f-67c30d94f0f0" with version 22572295
I0205 08:49:08.082504       1 snapshot_controller.go:144] updateContentStatusWithEvent[snapcontent-6a710a10-a399-497c-a61f-67c30d94f0f0]
I0205 08:49:08.091859       1 util.go:216] storeObjectUpdate updating content "snapcontent-6a710a10-a399-497c-a61f-67c30d94f0f0" with version 22572296
E0205 08:49:08.091874       1 snapshot_controller.go:107] createSnapshot for content [snapcontent-6a710a10-a399-497c-a61f-67c30d94f0f0]: error occurred in createSnapshotWrapper: failed to take snapshot of the volume 0;1;4181049054023231843;263B0B0A:6589AA52;;pvc-f60f272c-da05-45bf-88b6-d24472ba107b;/mnt/fs0/primary-fileset-fs0-9999915105066266935/.volumes/pvc-f60f272c-da05-45bf-88b6-d24472ba107b: "rpc error: code = Internal desc = unable to create snapshot [snapshot-6a710a10-a399-497c-a61f-67c30d94f0f0]. Error [[EFSSG0579C Fileset snapshots can only be taken from independent filesets.]]"

snapshot description :

Status:
  Bound Volume Snapshot Content Name:  snapcontent-6a710a10-a399-497c-a61f-67c30d94f0f0
  Error:
    Message:     Failed to check and update snapshot content: failed to take snapshot of the volume 0;1;4181049054023231843;263B0B0A:6589AA52;;pvc-f60f272c-da05-45bf-88b6-d24472ba107b;/mnt/fs0/primary-fileset-fs0-9999915105066266935/.volumes/pvc-f60f272c-da05-45bf-88b6-d24472ba107b: "rpc error: code = Internal desc = unable to create snapshot [snapshot-6a710a10-a399-497c-a61f-67c30d94f0f0]. Error [[EFSSG0579C Fileset snapshots can only be taken from independent filesets.]]"
    Time:        2024-02-05T08:49:23Z
  Ready To Use:  false
Events:
  Type     Reason                  Age    From                 Message
  ----     ------                  ----   ----                 -------
  Warning  GetSnapshotClassFailed  4m38s  snapshot-controller  Failed to get snapshot class with error volumesnapshotclass.snapshot.storage.k8s.io "ibm-spectrum-scale-snapshotclass-advance" not found
  Normal   CreatingSnapshot        4m38s  snapshot-controller  Waiting for a snapshot ibm-spectrum-scale-csi/ibm-spectrum-scale-snapshot to be created by the CSI driver.
saurabhwani5 commented 8 months ago

closing as issue is fixed