search

dell / csm

Dell Container Storage Modules (CSM)
Apache License 2.0
71 stars 15 forks source link

[Qualification]: Full Coverage with OpenShift 4.14 with PowerStore 3.6.1.2 connected with ISCSI using CSI driver 2.10.1 #1501

Closed dancohen21 closed 3 days ago

dancohen21 commented 1 month ago

Operating System

OpenShift 4.14.8

Container Orchestrator

OpenShift 4.14.8

Storage Platform

PowerStore 3.6.1.2

Protocol

iSCSI

Relevant log output

[2024-10-04 12:34:32]  INFO Starting cert-csi; ver. 1.5.0
[2024-10-04 12:34:32]  INFO Suites to run with powerstore-xfs storage class:
[2024-10-04 12:34:32]  INFO 1. VolumeIoSuite {volumes: 2, volumeSize: 5Gi chains: 2-2}
[2024-10-04 12:34:32]  INFO 2. ScalingSuite {replicas: 2, volumes: 5, volumeSize: 5Gi}
[2024-10-04 12:34:32]  INFO 3. CloneVolumeSuite {pods: 2, volumes: 1, volumeSize: 5Gi}
[2024-10-04 12:34:32]  INFO 4. VolumeExpansionSuite {pods: 1, volumes: 1, size: 5Gi, expSize: 10Gi, block: false}
[2024-10-04 12:34:32]  INFO 5. VolumeExpansionSuite {pods: 1, volumes: 1, size: 5Gi, expSize: 10Gi, block: true}
[2024-10-04 12:34:32]  INFO 6. SnapSuite {snapshots: 3, volumeSize; 5Gi}
[2024-10-04 12:34:32]  INFO 7. ReplicationSuite {pods: 2, volumes: 5, volumeSize: 5Gi}
[2024-10-04 12:34:32]  INFO 8. MultiAttachSuite {pods: 5, rawBlock: true, size: 5Gi, accMode: ReadWriteMany}
[2024-10-04 12:34:32]  INFO 9. EphemeralVolumeSuite {driver: csi-powerstore.dellemc.com, podNumber: 2, volAttributes: map[arrayid:PSe69390685e6a protocol:iSCSI size:10Gi]}
[2024-10-04 12:34:32]  INFO Suites to run with powerstore-nfs-waitforfirstconsumer storage class:
[2024-10-04 12:34:32]  INFO 1. VolumeIoSuite {volumes: 2, volumeSize: 5Gi chains: 2-2}
[2024-10-04 12:34:32]  INFO 2. ScalingSuite {replicas: 2, volumes: 5, volumeSize: 5Gi}
[2024-10-04 12:34:32]  INFO 3. CloneVolumeSuite {pods: 2, volumes: 1, volumeSize: 5Gi}
[2024-10-04 12:34:32]  INFO 4. VolumeExpansionSuite {pods: 1, volumes: 1, size: 5Gi, expSize: 10Gi, block: false}
[2024-10-04 12:34:32]  INFO 5. SnapSuite {snapshots: 3, volumeSize; 5Gi}
[2024-10-04 12:34:32]  INFO 6. ReplicationSuite {pods: 2, volumes: 5, volumeSize: 5Gi}
[2024-10-04 12:34:32]  INFO 7. MultiAttachSuite {pods: 5, rawBlock: false, size: 5Gi, accMode: ReadWriteMany}
[2024-10-04 12:34:32]  INFO 8. MultiAttachSuite {pods: 5, rawBlock: false, size: 5Gi, accMode: ReadWriteOncePod}
[2024-10-04 12:34:32]  INFO 9. EphemeralVolumeSuite {driver: csi-powerstore.dellemc.com, podNumber: 2, volAttributes: map[arrayid:PSe69390685e6a nasname:nas server protocol:NFS size:100Gi]}
[2024-10-04 12:34:32]  INFO 10. CapacityTrackingSuite {DriverNamespace: powerstore, volumeSize: 5Gi, pollInterval: 2m0s}
Does it look OK? (Y)es/(n)o
-> y
[2024-10-04 12:34:37]  INFO Using EVENT observer type
[2024-10-04 12:34:37]  INFO Using default config
[2024-10-04 12:34:37]  INFO Successfully loaded config. Host: https://console-openshift-console.apps.ocpcsi.powerflex.lab.dell.com:6443
[2024-10-04 12:34:37]  INFO Created new KubeClient
[2024-10-04 12:34:37]  INFO Running 1 iteration(s)
[2024-10-04 12:34:37]  INFO     *** ITERATION NUMBER 1 ***
[2024-10-04 12:34:37]  INFO Starting CapacityTrackingSuite with powerstore-nfs-waitforfirstconsumer storage class name=CapacityTrackingSuite num=9 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:34:37]  INFO Starting MultiAttachSuite with powerstore-nfs-waitforfirstconsumer storage class name=MultiAttachSuite num=6 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:34:37]  INFO Successfully created namespace capacity-tracking-test-536c4208 name=CapacityTrackingSuite num=9 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:34:37]  INFO Starting EphemeralVolumeSuite with powerstore-xfs storage class name=EphemeralVolumeSuite num=8 sc=powerstore-xfs
[2024-10-04 12:34:37]  INFO Successfully created namespace mas-test-1fe08070 name=MultiAttachSuite num=6 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:34:37]  INFO Using default image: docker.io/centos:latest name=MultiAttachSuite num=6 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:34:37]  INFO Creating Volume name=MultiAttachSuite num=6 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:34:37]  INFO Starting ReplicationSuite with powerstore-nfs-waitforfirstconsumer storage class name=ReplicationSuite num=5 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:34:37]  INFO Successfully created namespace functional-test-11558bb6 name=EphemeralVolumeSuite num=8 sc=powerstore-xfs
[2024-10-04 12:34:37]  INFO Using default image: docker.io/centos:latest
[2024-10-04 12:34:37]  INFO Starting VolumeExpansionSuite with powerstore-xfs storage class name=VolumeExpansionSuite num=3 sc=powerstore-xfs
[2024-10-04 12:34:37]  INFO Starting SnapSuite with powerstore-nfs-waitforfirstconsumer storage class name=SnapSuite num=4 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:34:37]  INFO Starting CloneVolumeSuite with powerstore-xfs storage class name=CloneVolumeSuite num=2 sc=powerstore-xfs
[2024-10-04 12:34:37]  INFO Creating 2 pods, each with 1 volumes
[2024-10-04 12:34:37]  INFO Attaching Volume to original pod name=MultiAttachSuite num=6 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:34:37]  INFO Starting VolumeExpansionSuite with powerstore-nfs-waitforfirstconsumer storage class name=VolumeExpansionSuite num=3 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:34:37]  INFO Starting VolumeIoSuite with powerstore-xfs storage class name=VolumeIoSuite num=0 sc=powerstore-xfs
[2024-10-04 12:34:37]  INFO Successfully created namespace volume-expansion-suite-f7bec9a2 name=VolumeExpansionSuite num=3 sc=powerstore-xfs
[2024-10-04 12:34:37]  INFO Using default image: docker.io/centos:latest name=VolumeExpansionSuite num=3 sc=powerstore-xfs
[2024-10-04 12:34:37]  INFO Creating 1 pods, each with 1 volumes of size (5Gi) name=VolumeExpansionSuite num=3 sc=powerstore-xfs
[2024-10-04 12:34:37]  INFO Starting VolumeExpansionSuite with powerstore-xfs storage class name=VolumeExpansionSuite num=4 sc=powerstore-xfs
[2024-10-04 12:34:37]  INFO Starting MultiAttachSuite with powerstore-nfs-waitforfirstconsumer storage class name=MultiAttachSuite num=7 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:34:37]  INFO Starting EphemeralVolumeSuite with powerstore-nfs-waitforfirstconsumer storage class name=EphemeralVolumeSuite num=8 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:34:37]  INFO Starting ReplicationSuite with powerstore-xfs storage class name=ReplicationSuite num=6 sc=powerstore-xfs
[2024-10-04 12:34:37]  INFO Starting ScalingSuite with powerstore-xfs storage class name=ScalingSuite num=1 sc=powerstore-xfs
[2024-10-04 12:34:37]  INFO Starting CloneVolumeSuite with powerstore-nfs-waitforfirstconsumer storage class name=CloneVolumeSuite num=2 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:34:37]  INFO Successfully created namespace replication-suite-ac6ef1bb name=ReplicationSuite num=5 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:34:37]  INFO Starting VolumeIoSuite with powerstore-nfs-waitforfirstconsumer storage class name=VolumeIoSuite num=0 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:34:37]  INFO Starting ScalingSuite with powerstore-nfs-waitforfirstconsumer storage class name=ScalingSuite num=1 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:34:37]  INFO Starting SnapSuite with powerstore-xfs storage class name=SnapSuite num=5 sc=powerstore-xfs
[2024-10-04 12:34:37]  INFO Starting MultiAttachSuite with powerstore-xfs storage class name=MultiAttachSuite num=7 sc=powerstore-xfs
[2024-10-04 12:34:37]  INFO Successfully created namespace snap-test-c4dc004d name=SnapSuite num=4 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:34:38]  INFO Using default image: docker.io/centos:latest name=ReplicationSuite num=5 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:34:38]  INFO Creating 2 pods, each with 5 volumes name=ReplicationSuite num=5 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:34:38]  INFO Using default image: docker.io/centos:latest name=SnapSuite num=4 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:34:38]  INFO Creating Snapshot pod name=SnapSuite num=4 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:34:38]  INFO Successfully created namespace clonevolume-suite-5fee6322 name=CloneVolumeSuite num=2 sc=powerstore-xfs
[2024-10-04 12:34:38]  INFO Using default image: docker.io/centos:latest name=CloneVolumeSuite num=2 sc=powerstore-xfs
[2024-10-04 12:34:38]  INFO Creating 2 pods, each with 1 volumes name=CloneVolumeSuite num=2 sc=powerstore-xfs
[2024-10-04 12:34:38]  INFO Waiting for all pods in mas-test-1fe08070 to be READY name=MultiAttachSuite num=6 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:34:38]  INFO Successfully created namespace volume-expansion-suite-dadb9a84 name=VolumeExpansionSuite num=3 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:34:38]  INFO Using default image: docker.io/centos:latest name=VolumeExpansionSuite num=3 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:34:38]  INFO Creating 1 pods, each with 1 volumes of size (5Gi) name=VolumeExpansionSuite num=3 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:34:38]  INFO Found 1 topology segment(s) in csinode
[2024-10-04 12:34:38]  INFO Using default image: docker.io/centos:latest
[2024-10-04 12:34:38]  INFO Creating capacity-tracking-94579f4d storage class
I1004 12:34:38.895621  750953 request.go:697] Waited for 1.055681526s due to client-side throttling, not priority and fairness, request: POST:https://console-openshift-console.apps.ocpcsi.powerflex.lab.dell.com:6443/api/v1/namespaces
[2024-10-04 12:34:38]  INFO Waiting for CSIStorageCapacity to be CREATED for capacity-tracking-94579f4d storage class in powerstore namespace
[2024-10-04 12:34:38]  INFO Successfully created namespace volumeio-test-859a5f63 name=VolumeIoSuite num=0 sc=powerstore-xfs
[2024-10-04 12:34:38]  INFO Using default image: docker.io/centos:latest name=VolumeIoSuite num=0 sc=powerstore-xfs
[2024-10-04 12:34:38]  INFO Creating IO pod name=VolumeIoSuite num=0 sc=powerstore-xfs
[2024-10-04 12:34:39]  INFO Successfully created namespace volume-expansion-suite-d4630819 name=VolumeExpansionSuite num=4 sc=powerstore-xfs
[2024-10-04 12:34:39]  INFO Using default image: docker.io/centos:latest name=VolumeExpansionSuite num=4 sc=powerstore-xfs
[2024-10-04 12:34:39]  INFO Creating 1 pods, each with 1 volumes of size (5Gi) name=VolumeExpansionSuite num=4 sc=powerstore-xfs
[2024-10-04 12:34:39]  INFO Successfully created namespace mas-test-ac057d4a name=MultiAttachSuite num=7 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:34:39]  INFO Using default image: docker.io/centos:latest name=MultiAttachSuite num=7 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:34:39]  INFO Creating Volume name=MultiAttachSuite num=7 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:34:39]  INFO Successfully created namespace functional-test-7065f7fe name=EphemeralVolumeSuite num=8 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:34:39]  INFO Using default image: docker.io/centos:latest
[2024-10-04 12:34:39]  INFO Creating 2 pods, each with 1 volumes
[2024-10-04 12:34:40]  INFO Successfully created namespace replication-suite-5f6691ea name=ReplicationSuite num=6 sc=powerstore-xfs
[2024-10-04 12:34:40]  INFO Using default image: docker.io/centos:latest name=ReplicationSuite num=6 sc=powerstore-xfs
[2024-10-04 12:34:40]  INFO Creating 2 pods, each with 5 volumes name=ReplicationSuite num=6 sc=powerstore-xfs
[2024-10-04 12:34:40]  INFO Successfully created namespace scale-test-875985ec name=ScalingSuite num=1 sc=powerstore-xfs
[2024-10-04 12:34:40]  INFO Using default image: docker.io/centos:latest name=ScalingSuite num=1 sc=powerstore-xfs
[2024-10-04 12:34:40]  INFO Successfully created namespace clonevolume-suite-fdcbe134 name=CloneVolumeSuite num=2 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:34:40]  INFO Using default image: docker.io/centos:latest name=CloneVolumeSuite num=2 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:34:40]  INFO Creating 2 pods, each with 1 volumes name=CloneVolumeSuite num=2 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:34:40]  INFO Successfully created namespace volumeio-test-3872950e name=VolumeIoSuite num=0 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:34:40]  INFO Using default image: docker.io/centos:latest name=VolumeIoSuite num=0 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:34:40]  INFO Creating IO pod name=VolumeIoSuite num=0 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:34:40]  INFO Successfully created namespace scale-test-5d38098f name=ScalingSuite num=1 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:34:40]  INFO Using default image: docker.io/centos:latest name=ScalingSuite num=1 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:34:40]  INFO All CSIStorageCapacity are created in 2.040537028s
[2024-10-04 12:34:40]  INFO Deleting capacity-tracking-94579f4d storage class,
[2024-10-04 12:34:40]  INFO Waiting for CSIStorageCapacity to be DELETED for capacity-tracking-94579f4d storage class in powerstore namespace
[2024-10-04 12:34:41]  INFO Successfully created namespace snap-test-64494d46 name=SnapSuite num=5 sc=powerstore-xfs
[2024-10-04 12:34:41]  INFO Using default image: docker.io/centos:latest name=SnapSuite num=5 sc=powerstore-xfs
[2024-10-04 12:34:41]  INFO Creating Snapshot pod name=SnapSuite num=5 sc=powerstore-xfs
[2024-10-04 12:34:41]  INFO Successfully created namespace mas-test-bd73ab25 name=MultiAttachSuite num=7 sc=powerstore-xfs
[2024-10-04 12:34:41]  INFO Using default image: docker.io/centos:latest name=MultiAttachSuite num=7 sc=powerstore-xfs
[2024-10-04 12:34:41]  INFO Creating Volume name=MultiAttachSuite num=7 sc=powerstore-xfs
[2024-10-04 12:34:41]  INFO Created stateful set sts-scale-testmnl5l name=ScalingSuite num=1 sc=powerstore-xfs
[2024-10-04 12:34:41]  INFO Waiting for sts pods to become ready name=ScalingSuite num=1 sc=powerstore-xfs
[2024-10-04 12:34:41]  INFO Created stateful set sts-scale-testzzpdj name=ScalingSuite num=1 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:34:41]  INFO Waiting for sts pods to become ready name=ScalingSuite num=1 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:34:42]  INFO Waiting for all pods in functional-test-11558bb6 to be READY name=EphemeralVolumeSuite num=8 sc=powerstore-xfs
[2024-10-04 12:34:42]  INFO All CSIStorageCapacity are deleted in 2.017984293s
[2024-10-04 12:34:42]  INFO Updating CSIStorageCapacity for powerstore-nfs-waitforfirstconsumer storage class, setting capacity to 0
[2024-10-04 12:34:43]  INFO Creating capacity-tracking-pod-835f8d5e pod using powerstore-nfs-waitforfirstconsumer storage class
[2024-10-04 12:34:44]  INFO Waiting for all pods in volume-expansion-suite-f7bec9a2 to be READY name=VolumeExpansionSuite num=3 sc=powerstore-xfs
[2024-10-04 12:34:44]  INFO Attaching Volume to original pod name=MultiAttachSuite num=7 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:34:47]  INFO Attaching Volume to original pod name=MultiAttachSuite num=7 sc=powerstore-xfs
[2024-10-04 12:34:48]  INFO Waiting for pod iowriter-test-cswc7 to be READY name=SnapSuite num=4 sc=powerstore-nfs-waitforfirstconsumer
I1004 12:34:48.896366  750953 request.go:697] Waited for 6.584434332s due to client-side throttling, not priority and fairness, request: POST:https://console-openshift-console.apps.ocpcsi.powerflex.lab.dell.com:6443/api/v1/namespaces/clonevolume-suite-5fee6322/pods
[2024-10-04 12:34:50]  INFO Waiting for all pods in volume-expansion-suite-dadb9a84 to be READY name=VolumeExpansionSuite num=3 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:34:51]  INFO Waiting for all pods in volume-expansion-suite-d4630819 to be READY name=VolumeExpansionSuite num=4 sc=powerstore-xfs
[2024-10-04 12:34:51]  INFO Waiting for all pods in mas-test-ac057d4a to be READY name=MultiAttachSuite num=7 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:34:52]  INFO Waiting for all pods in functional-test-7065f7fe to be READY name=EphemeralVolumeSuite num=8 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:34:54]  INFO Waiting for all pods in mas-test-bd73ab25 to be READY name=MultiAttachSuite num=7 sc=powerstore-xfs
I1004 12:34:59.096634  750953 request.go:697] Waited for 5.594366761s due to client-side throttling, not priority and fairness, request: POST:https://console-openshift-console.apps.ocpcsi.powerflex.lab.dell.com:6443/api/v1/namespaces/volumeio-test-3872950e/pods
[2024-10-04 12:34:59]  INFO Waiting for pod iowriter-test-sblvh to be READY name=VolumeIoSuite num=0 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:34:59]  INFO Waiting for pod iowriter-test-78nmj to be READY name=SnapSuite num=5 sc=powerstore-xfs
[2024-10-04 12:35:00]  INFO Waiting for all pods in clonevolume-suite-5fee6322 to be READY name=CloneVolumeSuite num=2 sc=powerstore-xfs
[2024-10-04 12:35:01]  INFO Waiting for pod iowriter-test-grsrd to be READY name=VolumeIoSuite num=0 sc=powerstore-xfs
[2024-10-04 12:35:04]  INFO Waiting for all pods in clonevolume-suite-fdcbe134 to be READY name=CloneVolumeSuite num=2 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:35:05]  INFO All pods are ready in 26.766921807s name=MultiAttachSuite num=6 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:35:08]  INFO capacity-tracking-pod-835f8d5e pod is PENDING
[2024-10-04 12:35:08]  INFO Waiting for provisioner to POLL GetCapacity for powerstore-nfs-waitforfirstconsumer storage class in 2m0s
I1004 12:35:09.296278  750953 request.go:697] Waited for 6.788389828s due to client-side throttling, not priority and fairness, request: POST:https://console-openshift-console.apps.ocpcsi.powerflex.lab.dell.com:6443/api/v1/namespaces/replication-suite-5f6691ea/persistentvolumeclaims
[2024-10-04 12:35:10]  INFO All pods are ready in 27.78832789s name=EphemeralVolumeSuite num=8 sc=powerstore-xfs
[2024-10-04 12:35:10]  INFO Writing to Volume on pod-ephemeral-test-vngzf
[2024-10-04 12:35:10]  INFO Executing command: [dd if=/dev/urandom of=/data/blob.data bs=1M count=128 oflag=sync] name=EphemeralVolumeSuite num=8 sc=powerstore-xfs
[2024-10-04 12:35:10]  INFO Scaling to 2 replicas name=ScalingSuite num=1 sc=powerstore-xfs
[2024-10-04 12:35:10]  INFO Waiting for sts pods to become ready name=ScalingSuite num=1 sc=powerstore-xfs
[2024-10-04 12:35:12]  INFO All pods are ready in 27.987610811s name=VolumeExpansionSuite num=3 sc=powerstore-xfs
[2024-10-04 12:35:12]  INFO Requested 5242880 KiB, got 0 KiB -- delta is -5242880 KiB name=VolumeExpansionSuite num=3 sc=powerstore-xfs
[2024-10-04 12:35:12]  INFO Expanding the size of volumes from 5Gi to 10Gi name=VolumeExpansionSuite num=3 sc=powerstore-xfs
[2024-10-04 12:35:12]  INFO Creating new pods with original Volume attached to them name=MultiAttachSuite num=6 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:35:12]  INFO Waiting for pod iowriter-test-29k8d to be READY name=VolumeIoSuite num=0 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:35:13]  INFO All pods are ready in 21.980147281s name=VolumeExpansionSuite num=4 sc=powerstore-xfs
[2024-10-04 12:35:13]  INFO Expanding the size of volumes from 5Gi to 10Gi name=VolumeExpansionSuite num=4 sc=powerstore-xfs
[2024-10-04 12:35:13]  INFO All pods are ready in 21.579776315s name=MultiAttachSuite num=7 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:35:13]  INFO All pods are ready in 21.582188244s name=EphemeralVolumeSuite num=8 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:35:13]  INFO Writing to Volume on pod-ephemeral-test-q9qql
[2024-10-04 12:35:13]  INFO Executing command: [dd if=/dev/urandom of=/data/blob.data bs=1M count=128 oflag=sync] name=EphemeralVolumeSuite num=8 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:35:15]  INFO All pods are ready in 20.974530412s name=MultiAttachSuite num=7 sc=powerstore-xfs
[2024-10-04 12:35:16]  INFO Executing command: [dd if=/dev/urandom of=/data0/writer-0.data bs=1M count=128 oflag=sync] name=SnapSuite num=4 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:35:16]  INFO Writer originalPod: pod-ephemeral-test-vngzf
[2024-10-04 12:35:16]  INFO Executing command: [/bin/bash -c sha512sum /data/blob.data > /data/blob.sha512] name=EphemeralVolumeSuite num=8 sc=powerstore-xfs
[2024-10-04 12:35:17]  INFO All pods are ready in 27.383351371s name=VolumeExpansionSuite num=3 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:35:18]  INFO Requested 5242880 KiB, got 6815744 KiB -- delta is 1572864 KiB name=VolumeExpansionSuite num=3 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:35:18]  INFO Expanding the size of volumes from 5Gi to 10Gi name=VolumeExpansionSuite num=3 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:35:18]  INFO Scaling to 2 replicas name=ScalingSuite num=1 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:35:18]  INFO Waiting for sts pods to become ready name=ScalingSuite num=1 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:35:18]  INFO Executing command: [/bin/bash -c sha512sum -c /data/blob.sha512] name=EphemeralVolumeSuite num=8 sc=powerstore-xfs
[2024-10-04 12:35:18]  INFO Writer originalPod: pod-ephemeral-test-q9qql
[2024-10-04 12:35:18]  INFO Executing command: [/bin/bash -c sha512sum /data/blob.data > /data/blob.sha512] name=EphemeralVolumeSuite num=8 sc=powerstore-nfs-waitforfirstconsumer
I1004 12:35:19.296642  750953 request.go:697] Waited for 6.162675451s due to client-side throttling, not priority and fairness, request: POST:https://console-openshift-console.apps.ocpcsi.powerflex.lab.dell.com:6443/api/v1/namespaces/replication-suite-ac6ef1bb/persistentvolumeclaims
[2024-10-04 12:35:19]  INFO Hashes match
[2024-10-04 12:35:19]  INFO Writing to Volume on pod-ephemeral-test-4lcvm
[2024-10-04 12:35:19]  INFO Executing command: [dd if=/dev/urandom of=/data/blob.data bs=1M count=128 oflag=sync] name=EphemeralVolumeSuite num=8 sc=powerstore-xfs
[2024-10-04 12:35:20]  INFO Executing command: [/bin/bash -c sha512sum -c /data/blob.sha512] name=EphemeralVolumeSuite num=8 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:35:20]  INFO Creating new pods with original Volume attached to them name=MultiAttachSuite num=7 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:35:21]  INFO Hashes match
[2024-10-04 12:35:21]  INFO Writing to Volume on pod-ephemeral-test-8w22c
[2024-10-04 12:35:21]  INFO Executing command: [dd if=/dev/urandom of=/data/blob.data bs=1M count=128 oflag=sync] name=EphemeralVolumeSuite num=8 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:35:21]  INFO Writer pod: iowriter-test-cswc7 name=SnapSuite num=4 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:35:21]  INFO Executing command: [/bin/bash -c sha512sum /data0/writer-0.data > /data0/writer-0.sha512] name=SnapSuite num=4 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:35:21]  INFO Waiting for pod iowriter-test-qgl5m to be READY name=VolumeIoSuite num=0 sc=powerstore-xfs
[2024-10-04 12:35:22]  INFO Creating new pods with original Volume attached to them name=MultiAttachSuite num=7 sc=powerstore-xfs
[2024-10-04 12:35:25]  INFO Writer originalPod: pod-ephemeral-test-8w22c
[2024-10-04 12:35:25]  INFO Executing command: [/bin/bash -c sha512sum /data/blob.data > /data/blob.sha512] name=EphemeralVolumeSuite num=8 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:35:26]  INFO Deleting all resources in namespace volume-expansion-suite-d4630819 name=VolumeExpansionSuite num=4 sc=powerstore-xfs
[2024-10-04 12:35:26]  INFO Writer originalPod: pod-ephemeral-test-4lcvm
[2024-10-04 12:35:26]  INFO Executing command: [/bin/bash -c sha512sum /data/blob.data > /data/blob.sha512] name=EphemeralVolumeSuite num=8 sc=powerstore-xfs
[2024-10-04 12:35:26]  INFO Executing command: [/bin/bash -c sha512sum -c /data/blob.sha512] name=EphemeralVolumeSuite num=8 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:35:27]  INFO All pods are ready in 23.390320214s name=CloneVolumeSuite num=2 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:35:27]  INFO Creating new pods with cloned volumes mounted on them name=CloneVolumeSuite num=2 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:35:27]  INFO Executing command: [/bin/bash -c dd if=/dev/urandom bs=1M count=128 oflag=sync > /data0/writer-0.data] name=VolumeIoSuite num=0 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:35:28]  INFO Executing command: [/bin/bash -c sha512sum -c /data/blob.sha512] name=EphemeralVolumeSuite num=8 sc=powerstore-xfs
[2024-10-04 12:35:28]  INFO Hashes match
[2024-10-04 12:35:28]  INFO Deleting all resources in namespace functional-test-7065f7fe name=EphemeralVolumeSuite num=8 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:35:29]  INFO All pods are ready in 28.592977785s name=CloneVolumeSuite num=2 sc=powerstore-xfs
[2024-10-04 12:35:29]  INFO Creating new pods with cloned volumes mounted on them name=CloneVolumeSuite num=2 sc=powerstore-xfs
I1004 12:35:29.495754  750953 request.go:697] Waited for 5.38007809s due to client-side throttling, not priority and fairness, request: POST:https://console-openshift-console.apps.ocpcsi.powerflex.lab.dell.com:6443/api/v1/namespaces/mas-test-1fe08070/pods
[2024-10-04 12:35:29]  INFO Executing command: [dd if=/dev/urandom of=/data0/writer-0.data bs=1M count=128 oflag=sync] name=SnapSuite num=5 sc=powerstore-xfs
[2024-10-04 12:35:30]  INFO Executing command: [/bin/bash -c dd if=/dev/urandom bs=1M count=128 oflag=sync > /data0/writer-0.data] name=VolumeIoSuite num=0 sc=powerstore-xfs
[2024-10-04 12:35:31]  INFO Hashes match
[2024-10-04 12:35:31]  INFO Deleting all resources in namespace functional-test-11558bb6 name=EphemeralVolumeSuite num=8 sc=powerstore-xfs
[2024-10-04 12:35:31]  INFO Executing command: [/bin/bash -c sha512sum /data0/writer-0.data > /data0/writer-0.sha512] name=VolumeIoSuite num=0 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:35:33]  INFO Executing command: [/bin/bash -c sha512sum /data0/writer-0.data > /data0/writer-0.sha512] name=VolumeIoSuite num=0 sc=powerstore-xfs
[2024-10-04 12:35:33]  INFO Checking if pod-prov-test-tgzsg/data0 was properly resized name=VolumeExpansionSuite num=3 sc=powerstore-xfs
[2024-10-04 12:35:34]  INFO Executing command: [/bin/bash -c dd if=/dev/urandom bs=1M count=128 oflag=sync > /data0/writer-1.data] name=VolumeIoSuite num=0 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:35:37]  INFO Executing command: [/bin/bash -c sha512sum /data0/writer-1.data > /data0/writer-1.sha512] name=VolumeIoSuite num=0 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:35:38]  INFO Scaling to 0 replicas name=ScalingSuite num=1 sc=powerstore-xfs
[2024-10-04 12:35:38]  INFO Waiting for sts pods to become ready name=ScalingSuite num=1 sc=powerstore-xfs
I1004 12:35:39.695877  750953 request.go:697] Waited for 6.246015271s due to client-side throttling, not priority and fairness, request: DELETE:https://console-openshift-console.apps.ocpcsi.powerflex.lab.dell.com:6443/api/v1/namespaces/volumeio-test-3872950e/pods/iowriter-test-sblvh
[2024-10-04 12:35:39]  INFO Checking if pod-prov-test-54jcj/data0 was properly resized name=VolumeExpansionSuite num=3 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:35:40]  INFO pod-prov-test-54jcj/data0 was successfully expanded name=VolumeExpansionSuite num=3 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:35:40]  INFO Deleting all resources in namespace volume-expansion-suite-dadb9a84 name=VolumeExpansionSuite num=3 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:35:40]  INFO Creating 3 snapshots name=SnapSuite num=4 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:35:40]  INFO Waiting for Snapshot 'snap' to be READY name=SnapSuite num=4 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:35:41]  INFO Writer pod: iowriter-test-78nmj name=SnapSuite num=5 sc=powerstore-xfs
[2024-10-04 12:35:41]  INFO Executing command: [/bin/bash -c sha512sum /data0/writer-0.data > /data0/writer-0.sha512] name=SnapSuite num=5 sc=powerstore-xfs
[2024-10-04 12:35:42]  INFO Waiting for all pods in mas-test-1fe08070 to be READY name=MultiAttachSuite num=6 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:35:43]  INFO Deleting all resources in namespace scale-test-875985ec name=ScalingSuite num=1 sc=powerstore-xfs
[2024-10-04 12:35:44]  INFO Waiting until no Volume Attachments with PV  left
[2024-10-04 12:35:44]  INFO VolumeAttachment deleted
[2024-10-04 12:35:46]  INFO Waiting until no Volume Attachments with PV csivol-ffa33373a6 left
[2024-10-04 12:35:46]  INFO All pods are ready in 4.799547728s name=MultiAttachSuite num=6 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:35:46]  INFO Writing to Volume on 1st originalPod name=MultiAttachSuite num=6 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:35:46]  INFO Executing command: [dd if=/dev/urandom of=/data0/blob.data bs=1M count=128 oflag=sync] name=MultiAttachSuite num=6 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:35:47]  INFO Waiting for all pods in replication-suite-ac6ef1bb to be READY name=ReplicationSuite num=5 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:35:48]  INFO Waiting for Snapshot 'snap1' to be READY name=SnapSuite num=4 sc=powerstore-nfs-waitforfirstconsumer
I1004 12:35:49.696132  750953 request.go:697] Waited for 5.991950038s due to client-side throttling, not priority and fairness, request: GET:https://console-openshift-console.apps.ocpcsi.powerflex.lab.dell.com:6443/api/v1/namespaces/scale-test-5d38098f/pods
[2024-10-04 12:35:49]  INFO Executing command: [/bin/bash -c dd if=/dev/urandom bs=1M count=128 oflag=sync > /data0/writer-1.data] name=VolumeIoSuite num=0 sc=powerstore-xfs
[2024-10-04 12:35:50]  INFO Waiting for all pods in clonevolume-suite-fdcbe134 to be READY name=CloneVolumeSuite num=2 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:35:50]  INFO Waiting until no Volume Attachments with PV  left
[2024-10-04 12:35:50]  INFO VolumeAttachment deleted
[2024-10-04 12:35:51]  INFO Scaling to 0 replicas name=ScalingSuite num=1 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:35:51]  INFO Waiting for sts pods to become ready name=ScalingSuite num=1 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:35:51]  INFO Waiting for pod iowriter-test-gxrbc to be READY name=VolumeIoSuite num=0 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:35:51]  INFO Writer originalPod: iowriter-test-np9rv name=MultiAttachSuite num=6 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:35:51]  INFO Waiting for all pods in replication-suite-5f6691ea to be READY name=ReplicationSuite num=6 sc=powerstore-xfs
[2024-10-04 12:35:51]  INFO Executing command: [/bin/bash -c sha512sum /data0/blob.data > /data0/blob.sha512] name=MultiAttachSuite num=6 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:35:51]  INFO Executing command: [/bin/bash -c sha512sum /data0/writer-1.data > /data0/writer-1.sha512] name=VolumeIoSuite num=0 sc=powerstore-xfs
[2024-10-04 12:35:52]  INFO VolumeAttachment deleted
[2024-10-04 12:35:52]  INFO Waiting for all pods in mas-test-bd73ab25 to be READY name=MultiAttachSuite num=7 sc=powerstore-xfs
[2024-10-04 12:35:52]  INFO Checking hash sum on all of the other pods name=MultiAttachSuite num=6 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:35:52]  INFO Executing command: [/bin/bash -c sha512sum -c /data0/blob.sha512] name=MultiAttachSuite num=6 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:35:53]  INFO Waiting for all pods in clonevolume-suite-5fee6322 to be READY name=CloneVolumeSuite num=2 sc=powerstore-xfs
[2024-10-04 12:35:53]  INFO Hashes match name=MultiAttachSuite num=6 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:35:53]  INFO Executing command: [/bin/bash -c sha512sum -c /data0/blob.sha512] name=MultiAttachSuite num=6 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:35:54]  INFO Provisioner POLLED GetCapacity for powerstore-nfs-waitforfirstconsumer storage class
[2024-10-04 12:35:54]  INFO Waiting for pod capacity-tracking-pod-835f8d5e to be READY name=CapacityTrackingSuite num=9 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:35:55]  INFO Hashes match name=MultiAttachSuite num=6 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:35:55]  INFO Executing command: [/bin/bash -c sha512sum -c /data0/blob.sha512] name=MultiAttachSuite num=6 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:35:56]  INFO Waiting for pod iowriter-test-crjdx to be READY name=VolumeIoSuite num=0 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:35:56]  INFO Deleting all resources in namespace scale-test-5d38098f name=ScalingSuite num=1 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:35:57]  INFO Hashes match name=MultiAttachSuite num=6 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:35:57]  INFO Executing command: [/bin/bash -c sha512sum -c /data0/blob.sha512] name=MultiAttachSuite num=6 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:35:57]  INFO Waiting for pod iowriter-test-smcxn to be READY name=VolumeIoSuite num=0 sc=powerstore-xfs
[2024-10-04 12:35:57]  INFO All pods are ready in 5.199571914s name=MultiAttachSuite num=7 sc=powerstore-xfs
[2024-10-04 12:35:57]  INFO Executing command: [dd if=/dev/urandom of=/dev/data0 bs=1M count=128 oflag=sync] name=MultiAttachSuite num=7 sc=powerstore-xfs
[2024-10-04 12:35:58]  INFO Hashes match name=MultiAttachSuite num=6 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:35:58]  INFO Executing command: [/bin/bash -c sha512sum -c /data0/blob.sha512] name=MultiAttachSuite num=6 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:35:59]  INFO Creating 3 snapshots name=SnapSuite num=5 sc=powerstore-xfs
[2024-10-04 12:35:59]  INFO Waiting for Snapshot 'snap' to be READY name=SnapSuite num=5 sc=powerstore-xfs
[2024-10-04 12:35:59]  INFO capacity-tracking-pod-835f8d5e pod is RUNNING
[2024-10-04 12:35:59]  INFO Deleting all resources in namespace capacity-tracking-test-536c4208 name=CapacityTrackingSuite num=9 sc=powerstore-nfs-waitforfirstconsumer
I1004 12:35:59.896404  750953 request.go:697] Waited for 4.793996565s due to client-side throttling, not priority and fairness, request: GET:https://console-openshift-console.apps.ocpcsi.powerflex.lab.dell.com:6443/api/v1/namespaces/volume-expansion-suite-dadb9a84/pods
[2024-10-04 12:36:00]  INFO Namespace functional-test-7065f7fe was deleted in 31.28804493s name=EphemeralVolumeSuite num=8 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:36:00]  INFO SUCCESS: EphemeralVolumeSuite in 1m22.462049249s name=EphemeralVolumeSuite num=8 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:36:00]  INFO Hashes match name=MultiAttachSuite num=6 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:36:00]  INFO Deleting all resources in namespace mas-test-1fe08070 name=MultiAttachSuite num=6 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:36:02]  INFO Namespace functional-test-11558bb6 was deleted in 31.241808778s name=EphemeralVolumeSuite num=8 sc=powerstore-xfs
[2024-10-04 12:36:02]  INFO SUCCESS: EphemeralVolumeSuite in 1m24.793782943s name=EphemeralVolumeSuite num=8 sc=powerstore-xfs
[2024-10-04 12:36:02]  INFO Waiting for Snapshot 'snap2' to be READY name=SnapSuite num=4 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:36:03]  INFO Waiting until no Volume Attachments with PV csivol-e5c9443732 left
[2024-10-04 12:36:04]  INFO Executing command: [/bin/bash -c sha512sum -c /data0/writer-0.sha512] name=VolumeIoSuite num=0 sc=powerstore-nfs-waitforfirstconsumer
128+0 records in
128+0 records out
134217728 bytes (134 MB, 128 MiB) copied, 6.6533 s, 20.2 MB/s
[2024-10-04 12:36:04]  INFO Executing command: [dd if=/dev/data0 of=/tmp/blob.data bs=1M count=128] name=MultiAttachSuite num=7 sc=powerstore-xfs
[2024-10-04 12:36:05]  INFO Waiting for Snapshot 'snap1' to be READY name=SnapSuite num=5 sc=powerstore-xfs
128+0 records in
128+0 records out
134217728 bytes (134 MB, 128 MiB) copied, 0.413729 s, 324 MB/s
[2024-10-04 12:36:05]  INFO Executing command: [sha512sum /tmp/blob.data] name=MultiAttachSuite num=7 sc=powerstore-xfs
[2024-10-04 12:36:06]  INFO OriginalPod: iowriter-test-7246m name=MultiAttachSuite num=7 sc=powerstore-xfs
[2024-10-04 12:36:06]  INFO hash sum is:1026679d37c1d4d1152d0a87c8ef5229c0a86484b307f869f979f29e88c3e39ceceea76d2b983b27bf71d6fae3e3810c6a095e23ae81bfa60254918162d6dbff  /tmp/blob.data
 name=MultiAttachSuite num=7 sc=powerstore-xfs
[2024-10-04 12:36:06]  INFO Checking hash sum on all of the other pods name=MultiAttachSuite num=7 sc=powerstore-xfs
[2024-10-04 12:36:06]  INFO Executing command: [blockdev --flushbufs /dev/data0] name=MultiAttachSuite num=7 sc=powerstore-xfs
[2024-10-04 12:36:06]  INFO Hashes match name=VolumeIoSuite num=0 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:36:06]  INFO Executing command: [/bin/bash -c dd if=/dev/urandom bs=1M count=128 oflag=sync > /data0/writer-0.data] name=VolumeIoSuite num=0 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:36:07]  INFO Executing command: [dd if=/dev/data0 of=/tmp/blob.data bs=1M count=128] name=MultiAttachSuite num=7 sc=powerstore-xfs
[2024-10-04 12:36:07]  INFO Waiting for Snapshot 'snap2' to be READY name=SnapSuite num=5 sc=powerstore-xfs
[2024-10-04 12:36:07]  INFO Namespace volume-expansion-suite-d4630819 was deleted in 41.374494825s name=VolumeExpansionSuite num=4 sc=powerstore-xfs
[2024-10-04 12:36:09]  INFO Restoring from snap name=SnapSuite num=5 sc=powerstore-xfs
[2024-10-04 12:36:09]  INFO Creating pvc snap-restore name=SnapSuite num=5 sc=powerstore-xfs
[2024-10-04 12:36:09]  INFO Executing command: [/bin/bash -c sha512sum /data0/writer-0.data > /data0/writer-0.sha512] name=VolumeIoSuite num=0 sc=powerstore-nfs-waitforfirstconsumer
I1004 12:36:10.096508  750953 request.go:697] Waited for 5.590560737s due to client-side throttling, not priority and fairness, request: GET:https://console-openshift-console.apps.ocpcsi.powerflex.lab.dell.com:6443/api/v1/namespaces/snap-test-64494d46/persistentvolumeclaims
128+0 records in
128+0 records out
134217728 bytes (134 MB, 128 MiB) copied, 3.11984 s, 43.0 MB/s
[2024-10-04 12:36:10]  INFO Executing command: [sha512sum /tmp/blob.data] name=MultiAttachSuite num=7 sc=powerstore-xfs
[2024-10-04 12:36:10]  INFO Restoring from snap1 name=SnapSuite num=4 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:36:10]  INFO Creating pvc snap1-restore name=SnapSuite num=4 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:36:11]  INFO SUCCESS: VolumeExpansionSuite in 1m33.352593822s name=VolumeExpansionSuite num=4 sc=powerstore-xfs
[2024-10-04 12:36:11]  INFO Pod: iowriter-test-grk7g name=MultiAttachSuite num=7 sc=powerstore-xfs
[2024-10-04 12:36:11]  INFO hash sum is:1026679d37c1d4d1152d0a87c8ef5229c0a86484b307f869f979f29e88c3e39ceceea76d2b983b27bf71d6fae3e3810c6a095e23ae81bfa60254918162d6dbff  /tmp/blob.data
 name=MultiAttachSuite num=7 sc=powerstore-xfs
[2024-10-04 12:36:11]  INFO Hashes match name=MultiAttachSuite num=7 sc=powerstore-xfs
[2024-10-04 12:36:11]  INFO Executing command: [blockdev --flushbufs /dev/data0] name=MultiAttachSuite num=7 sc=powerstore-xfs
[2024-10-04 12:36:11]  INFO Executing command: [dd if=/dev/data0 of=/tmp/blob.data bs=1M count=128] name=MultiAttachSuite num=7 sc=powerstore-xfs
[2024-10-04 12:36:12]  INFO Executing command: [/bin/bash -c sha512sum -c /data0/writer-1.sha512] name=VolumeIoSuite num=0 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:36:12]  INFO Namespace scale-test-875985ec was deleted in 28.909880897s name=ScalingSuite num=1 sc=powerstore-xfs
[2024-10-04 12:36:13]  INFO SUCCESS: ScalingSuite in 1m35.514507914s name=ScalingSuite num=1 sc=powerstore-xfs
[2024-10-04 12:36:13]  INFO All pods are ready in 23.363239568s name=CloneVolumeSuite num=2 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:36:13]  INFO Deleting all resources in namespace clonevolume-suite-fdcbe134 name=CloneVolumeSuite num=2 sc=powerstore-nfs-waitforfirstconsumer
128+0 records in
128+0 records out
134217728 bytes (134 MB, 128 MiB) copied, 3.19602 s, 42.0 MB/s
[2024-10-04 12:36:14]  INFO Executing command: [sha512sum /tmp/blob.data] name=MultiAttachSuite num=7 sc=powerstore-xfs
[2024-10-04 12:36:15]  INFO VolumeAttachment deleted
[2024-10-04 12:36:15]  INFO Hashes match name=VolumeIoSuite num=0 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:36:15]  INFO Executing command: [/bin/bash -c dd if=/dev/urandom bs=1M count=128 oflag=sync > /data0/writer-1.data] name=VolumeIoSuite num=0 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:36:15]  INFO Pod: iowriter-test-vjh9g name=MultiAttachSuite num=7 sc=powerstore-xfs
[2024-10-04 12:36:15]  INFO hash sum is:1026679d37c1d4d1152d0a87c8ef5229c0a86484b307f869f979f29e88c3e39ceceea76d2b983b27bf71d6fae3e3810c6a095e23ae81bfa60254918162d6dbff  /tmp/blob.data
 name=MultiAttachSuite num=7 sc=powerstore-xfs
[2024-10-04 12:36:15]  INFO Hashes match name=MultiAttachSuite num=7 sc=powerstore-xfs
[2024-10-04 12:36:15]  INFO Executing command: [blockdev --flushbufs /dev/data0] name=MultiAttachSuite num=7 sc=powerstore-xfs
[2024-10-04 12:36:16]  INFO Executing command: [dd if=/dev/data0 of=/tmp/blob.data bs=1M count=128] name=MultiAttachSuite num=7 sc=powerstore-xfs
[2024-10-04 12:36:19]  INFO Waiting for pod snap1-restore-pod to be READY name=SnapSuite num=4 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:36:19]  INFO Executing command: [/bin/bash -c sha512sum /data0/writer-1.data > /data0/writer-1.sha512] name=VolumeIoSuite num=0 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:36:19]  INFO Namespace volume-expansion-suite-dadb9a84 was deleted in 39.680319113s name=VolumeExpansionSuite num=3 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:36:19]  INFO Waiting until no Volume Attachments with PV  left
[2024-10-04 12:36:19]  INFO VolumeAttachment deleted
[2024-10-04 12:36:20]  INFO SUCCESS: VolumeExpansionSuite in 1m42.377365692s name=VolumeExpansionSuite num=3 sc=powerstore-nfs-waitforfirstconsumer
I1004 12:36:20.296314  750953 request.go:697] Waited for 5.188000836s due to client-side throttling, not priority and fairness, request: GET:https://console-openshift-console.apps.ocpcsi.powerflex.lab.dell.com:6443/api/v1/namespaces/mas-test-ac057d4a/persistentvolumeclaims
[2024-10-04 12:36:20]  INFO Waiting for pod iowriter-test-zrccq to be READY name=VolumeIoSuite num=0 sc=powerstore-xfs
128+0 records in
128+0 records out
134217728 bytes (134 MB, 128 MiB) copied, 4.05776 s, 33.1 MB/s
[2024-10-04 12:36:20]  INFO Executing command: [sha512sum /tmp/blob.data] name=MultiAttachSuite num=7 sc=powerstore-xfs
[2024-10-04 12:36:21]  INFO Pod: iowriter-test-875qg name=MultiAttachSuite num=7 sc=powerstore-xfs
[2024-10-04 12:36:21]  INFO hash sum is:1026679d37c1d4d1152d0a87c8ef5229c0a86484b307f869f979f29e88c3e39ceceea76d2b983b27bf71d6fae3e3810c6a095e23ae81bfa60254918162d6dbff  /tmp/blob.data
 name=MultiAttachSuite num=7 sc=powerstore-xfs
[2024-10-04 12:36:21]  INFO Hashes match name=MultiAttachSuite num=7 sc=powerstore-xfs
[2024-10-04 12:36:21]  INFO Executing command: [blockdev --flushbufs /dev/data0] name=MultiAttachSuite num=7 sc=powerstore-xfs
[2024-10-04 12:36:22]  INFO Executing command: [dd if=/dev/data0 of=/tmp/blob.data bs=1M count=128] name=MultiAttachSuite num=7 sc=powerstore-xfs
[2024-10-04 12:36:22]  INFO Waiting for pod snap-restore-pod to be READY name=SnapSuite num=5 sc=powerstore-xfs
[2024-10-04 12:36:24]  INFO Executing command: [/bin/bash -c sha512sum -c /data0/writer-0.sha512] name=VolumeIoSuite num=0 sc=powerstore-xfs
128+0 records in
128+0 records out
134217728 bytes (134 MB, 128 MiB) copied, 2.59988 s, 51.6 MB/s
[2024-10-04 12:36:24]  INFO Executing command: [sha512sum /tmp/blob.data] name=MultiAttachSuite num=7 sc=powerstore-xfs
[2024-10-04 12:36:25]  INFO All pods are ready in 38.573426685s name=ReplicationSuite num=5 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:36:25]  INFO Creating a snapshot on each of the volumes name=ReplicationSuite num=5 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:36:25]  INFO Pod: iowriter-test-k8dgs name=MultiAttachSuite num=7 sc=powerstore-xfs
[2024-10-04 12:36:25]  INFO hash sum is:1026679d37c1d4d1152d0a87c8ef5229c0a86484b307f869f979f29e88c3e39ceceea76d2b983b27bf71d6fae3e3810c6a095e23ae81bfa60254918162d6dbff  /tmp/blob.data
 name=MultiAttachSuite num=7 sc=powerstore-xfs
[2024-10-04 12:36:25]  INFO Hashes match name=MultiAttachSuite num=7 sc=powerstore-xfs
[2024-10-04 12:36:25]  INFO Executing command: [blockdev --flushbufs /dev/data0] name=MultiAttachSuite num=7 sc=powerstore-xfs
[2024-10-04 12:36:26]  INFO Executing command: [dd if=/dev/data0 of=/tmp/blob.data bs=1M count=128] name=MultiAttachSuite num=7 sc=powerstore-xfs
[2024-10-04 12:36:26]  INFO All pods are ready in 33.594970767s name=CloneVolumeSuite num=2 sc=powerstore-xfs
[2024-10-04 12:36:26]  INFO Deleting all resources in namespace clonevolume-suite-5fee6322 name=CloneVolumeSuite num=2 sc=powerstore-xfs
[2024-10-04 12:36:28]  INFO Namespace mas-test-1fe08070 was deleted in 27.771816014s name=MultiAttachSuite num=6 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:36:28]  INFO Namespace scale-test-5d38098f was deleted in 31.893069865s name=ScalingSuite num=1 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:36:29]  INFO Waiting until no Volume Attachments with PV  left
[2024-10-04 12:36:29]  INFO VolumeAttachment deleted
[2024-10-04 12:36:29]  INFO Deleting all resources in namespace volumeio-test-3872950e name=VolumeIoSuite num=0 sc=powerstore-nfs-waitforfirstconsumer
128+0 records in
128+0 records out
134217728 bytes (134 MB, 128 MiB) copied, 3.56254 s, 37.7 MB/s
[2024-10-04 12:36:30]  INFO Executing command: [sha512sum /tmp/blob.data] name=MultiAttachSuite num=7 sc=powerstore-xfs
[2024-10-04 12:36:30]  INFO Hashes match name=VolumeIoSuite num=0 sc=powerstore-xfs
[2024-10-04 12:36:30]  INFO Executing command: [/bin/bash -c dd if=/dev/urandom bs=1M count=128 oflag=sync > /data0/writer-0.data] name=VolumeIoSuite num=0 sc=powerstore-xfs
I1004 12:36:30.495726  750953 request.go:697] Waited for 4.190240827s due to client-side throttling, not priority and fairness, request: GET:https://console-openshift-console.apps.ocpcsi.powerflex.lab.dell.com:6443/api/v1/namespaces/replication-suite-5f6691ea/persistentvolumeclaims
[2024-10-04 12:36:30]  INFO Namespace capacity-tracking-test-536c4208 was deleted in 31.056694609s name=CapacityTrackingSuite num=9 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:36:31]  INFO Pod: iowriter-test-f78tn name=MultiAttachSuite num=7 sc=powerstore-xfs
[2024-10-04 12:36:31]  INFO hash sum is:1026679d37c1d4d1152d0a87c8ef5229c0a86484b307f869f979f29e88c3e39ceceea76d2b983b27bf71d6fae3e3810c6a095e23ae81bfa60254918162d6dbff  /tmp/blob.data
 name=MultiAttachSuite num=7 sc=powerstore-xfs
[2024-10-04 12:36:31]  INFO Hashes match name=MultiAttachSuite num=7 sc=powerstore-xfs
[2024-10-04 12:36:31]  INFO Deleting all resources in namespace mas-test-bd73ab25 name=MultiAttachSuite num=7 sc=powerstore-xfs
[2024-10-04 12:36:31]  INFO All pods are ready in 39.586539352s name=ReplicationSuite num=6 sc=powerstore-xfs
[2024-10-04 12:36:31]  INFO Creating a snapshot on each of the volumes name=ReplicationSuite num=6 sc=powerstore-xfs
[2024-10-04 12:36:32]  INFO SUCCESS: CapacityTrackingSuite in 1m55.21606781s name=CapacityTrackingSuite num=9 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:36:32]  INFO SUCCESS: MultiAttachSuite in 1m55.23158426s name=MultiAttachSuite num=6 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:36:33]  INFO Executing command: [/bin/bash -c sha512sum /data0/writer-0.data > /data0/writer-0.sha512] name=VolumeIoSuite num=0 sc=powerstore-xfs
[2024-10-04 12:36:33]  INFO SUCCESS: ScalingSuite in 1m55.946544991s name=ScalingSuite num=1 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:36:34]  WARN pod-prov-test-tgzsg/data0 expanded within ±5% of request size name=VolumeExpansionSuite num=3 sc=powerstore-xfs
[2024-10-04 12:36:34]  INFO Deleting all resources in namespace volume-expansion-suite-f7bec9a2 name=VolumeExpansionSuite num=3 sc=powerstore-xfs
[2024-10-04 12:36:34]  INFO Checker pod: snap1-restore-pod name=SnapSuite num=4 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:36:34]  INFO Executing command: [/bin/bash -c sha512sum -c /data0/writer-0.sha512] name=SnapSuite num=4 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:36:36]  INFO Hashes match name=SnapSuite num=4 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:36:36]  INFO Deleting all resources in namespace snap-test-c4dc004d name=SnapSuite num=4 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:36:39]  INFO Namespace clonevolume-suite-fdcbe134 was deleted in 25.360985437s name=CloneVolumeSuite num=2 sc=powerstore-nfs-waitforfirstconsumer
I1004 12:36:40.496494  750953 request.go:697] Waited for 3.791799258s due to client-side throttling, not priority and fairness, request: GET:https://console-openshift-console.apps.ocpcsi.powerflex.lab.dell.com:6443/api/v1/namespaces/clonevolume-suite-5fee6322
[2024-10-04 12:36:42]  INFO Waiting until no Volume Attachments with PV csivol-ffa33373a6 left
[2024-10-04 12:36:43]  INFO SUCCESS: CloneVolumeSuite in 2m5.181193441s name=CloneVolumeSuite num=2 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:36:44]  INFO Executing command: [/bin/bash -c sha512sum -c /data0/writer-1.sha512] name=VolumeIoSuite num=0 sc=powerstore-xfs
[2024-10-04 12:36:47]  INFO Hashes match name=VolumeIoSuite num=0 sc=powerstore-xfs
[2024-10-04 12:36:47]  INFO Executing command: [/bin/bash -c dd if=/dev/urandom bs=1M count=128 oflag=sync > /data0/writer-1.data] name=VolumeIoSuite num=0 sc=powerstore-xfs
[2024-10-04 12:36:48]  INFO Checker pod: snap-restore-pod name=SnapSuite num=5 sc=powerstore-xfs
[2024-10-04 12:36:48]  INFO Executing command: [/bin/bash -c sha512sum -c /data0/writer-0.sha512] name=SnapSuite num=5 sc=powerstore-xfs
I1004 12:36:50.696119  750953 request.go:697] Waited for 2.381672638s due to client-side throttling, not priority and fairness, request: GET:https://console-openshift-console.apps.ocpcsi.powerflex.lab.dell.com:6443/api/v1/namespaces/mas-test-ac057d4a/pods
[2024-10-04 12:36:50]  INFO Deleting all resources in namespace mas-test-ac057d4a name=MultiAttachSuite num=7 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:36:52]  INFO Executing command: [/bin/bash -c sha512sum /data0/writer-1.data > /data0/writer-1.sha512] name=VolumeIoSuite num=0 sc=powerstore-xfs
[2024-10-04 12:36:52]  INFO VolumeAttachment deleted
[2024-10-04 12:36:55]  INFO Hashes match name=SnapSuite num=5 sc=powerstore-xfs
[2024-10-04 12:36:55]  INFO Deleting all resources in namespace snap-test-64494d46 name=SnapSuite num=5 sc=powerstore-xfs
[2024-10-04 12:36:56]  INFO Namespace volumeio-test-3872950e was deleted in 27.393968467s name=VolumeIoSuite num=0 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:36:58]  INFO Namespace volume-expansion-suite-f7bec9a2 was deleted in 24.228469415s name=VolumeExpansionSuite num=3 sc=powerstore-xfs
[2024-10-04 12:36:58]  INFO Waiting for the snapshots in replication-suite-ac6ef1bb to be READY name=ReplicationSuite num=5 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:37:00]  INFO Namespace mas-test-bd73ab25 was deleted in 29.124828682s name=MultiAttachSuite num=7 sc=powerstore-xfs
[2024-10-04 12:37:00]  INFO Waiting until no Volume Attachments with PV csivol-e5c9443732 left
[2024-10-04 12:37:00]  INFO VolumeAttachment deleted
[2024-10-04 12:37:00]  INFO Deleting all resources in namespace volumeio-test-859a5f63 name=VolumeIoSuite num=0 sc=powerstore-xfs
I1004 12:37:00.696411  750953 request.go:697] Waited for 3.593249636s due to client-side throttling, not priority and fairness, request: GET:https://console-openshift-console.apps.ocpcsi.powerflex.lab.dell.com:6443/api/v1/namespaces/volume-expansion-suite-f7bec9a2/persistentvolumeclaims
[2024-10-04 12:37:02]  INFO Waiting for the snapshots in replication-suite-5f6691ea to be READY name=ReplicationSuite num=6 sc=powerstore-xfs
[2024-10-04 12:37:03]  INFO SUCCESS: VolumeIoSuite in 2m25.464685606s name=VolumeIoSuite num=0 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:37:03]  INFO Namespace snap-test-c4dc004d was deleted in 27.67008502s name=SnapSuite num=4 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:37:04]  INFO SUCCESS: VolumeExpansionSuite in 2m26.191454813s name=VolumeExpansionSuite num=3 sc=powerstore-xfs
[2024-10-04 12:37:04]  INFO SUCCESS: MultiAttachSuite in 2m26.549304734s name=MultiAttachSuite num=7 sc=powerstore-xfs
[2024-10-04 12:37:04]  INFO Namespace clonevolume-suite-5fee6322 was deleted in 37.990665677s name=CloneVolumeSuite num=2 sc=powerstore-xfs
[2024-10-04 12:37:08]  INFO SUCCESS: SnapSuite in 2m30.633297201s name=SnapSuite num=4 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:37:09]  INFO SUCCESS: CloneVolumeSuite in 2m31.452355255s name=CloneVolumeSuite num=2 sc=powerstore-xfs
[2024-10-04 12:37:11]  INFO Namespace mas-test-ac057d4a was deleted in 20.39283945s name=MultiAttachSuite num=7 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:37:11]  INFO SUCCESS: MultiAttachSuite in 2m33.767954365s name=MultiAttachSuite num=7 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:37:12]  INFO Creating new pods with replicated volumes mounted on them name=ReplicationSuite num=6 sc=powerstore-xfs
[2024-10-04 12:37:12]  INFO Creating new pods with replicated volumes mounted on them name=ReplicationSuite num=5 sc=powerstore-nfs-waitforfirstconsumer
I1004 12:37:15.096363  750953 request.go:697] Waited for 1.189755511s due to client-side throttling, not priority and fairness, request: GET:https://console-openshift-console.apps.ocpcsi.powerflex.lab.dell.com:6443/api/v1/namespaces/snap-test-64494d46/pods
[2024-10-04 12:37:17]  INFO Namespace snap-test-64494d46 was deleted in 22.477560794s name=SnapSuite num=5 sc=powerstore-xfs
[2024-10-04 12:37:19]  INFO SUCCESS: SnapSuite in 2m41.159462053s name=SnapSuite num=5 sc=powerstore-xfs
[2024-10-04 12:37:19]  INFO Waiting for all pods in replication-suite-5f6691ea to be READY name=ReplicationSuite num=6 sc=powerstore-xfs
[2024-10-04 12:37:19]  INFO Namespace volumeio-test-859a5f63 was deleted in 19.051408602s name=VolumeIoSuite num=0 sc=powerstore-xfs
[2024-10-04 12:37:20]  INFO Waiting for all pods in replication-suite-ac6ef1bb to be READY name=ReplicationSuite num=5 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:37:20]  INFO SUCCESS: VolumeIoSuite in 2m42.789618094s name=VolumeIoSuite num=0 sc=powerstore-xfs
[2024-10-04 12:37:44]  INFO All pods are ready in 24.808114479s name=ReplicationSuite num=6 sc=powerstore-xfs
[2024-10-04 12:37:44]  INFO The replication group name from pvc is   name=ReplicationSuite num=6 sc=powerstore-xfs
[2024-10-04 12:37:44]  INFO Deleting all resources in namespace replication-suite-5f6691ea name=ReplicationSuite num=6 sc=powerstore-xfs
[2024-10-04 12:37:46]  INFO All pods are ready in 26.206593808s name=ReplicationSuite num=5 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:37:46]  INFO The replication group name from pvc is   name=ReplicationSuite num=5 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:37:46]  INFO Deleting all resources in namespace replication-suite-ac6ef1bb name=ReplicationSuite num=5 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:38:37]  INFO Namespace replication-suite-5f6691ea was deleted in 53.491484s name=ReplicationSuite num=6 sc=powerstore-xfs
[2024-10-04 12:38:37]  INFO Deletion callback function is not empty, calling it name=ReplicationSuite num=6 sc=powerstore-xfs
[2024-10-04 12:38:37]  INFO Deleting local RG name=ReplicationSuite num=6 sc=powerstore-xfs
[2024-10-04 12:38:37]  WARN error when deleting local RG: no matches for kind "DellCSIReplicationGroup" in version "replication.storage.dell.com/v1" name=ReplicationSuite num=6 sc=powerstore-xfs
[2024-10-04 12:38:37]  INFO Sleeping for 1 minute... name=ReplicationSuite num=6 sc=powerstore-xfs
[2024-10-04 12:38:42]  INFO Namespace replication-suite-ac6ef1bb was deleted in 55.485506751s name=ReplicationSuite num=5 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:38:42]  INFO Deletion callback function is not empty, calling it name=ReplicationSuite num=5 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:38:42]  INFO Deleting local RG name=ReplicationSuite num=5 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:38:42]  WARN error when deleting local RG: no matches for kind "DellCSIReplicationGroup" in version "replication.storage.dell.com/v1" name=ReplicationSuite num=5 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:38:42]  INFO Sleeping for 1 minute... name=ReplicationSuite num=5 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:39:42]  INFO SUCCESS: ReplicationSuite in 5m5.059430433s name=ReplicationSuite num=6 sc=powerstore-xfs
[2024-10-04 12:39:44]  INFO SUCCESS: ReplicationSuite in 5m6.624440043s name=ReplicationSuite num=5 sc=powerstore-nfs-waitforfirstconsumer
[2024-10-04 12:39:44]  INFO Started generating reports...
[2024-10-04 12:39:59]  INFO Avg time of a run:   102.53s
[2024-10-04 12:39:59]  INFO Avg time of a del:   38.21s
[2024-10-04 12:39:59]  INFO Avg time of all:     145.24s
[2024-10-04 12:39:59]  INFO During this run 100.0% of suites succeeded

Additional Details

19 of 19 tests succeeded.

cert-csi release 1.5.0 PowerStore 3.6.1.2 connected with ISCSI OpenShift 4.14 using csm-operator 1.5.1 and csi-powerstore 2.10.1 using the powerstore sample on https://dell.github.io/csm-docs/docs/support/cert-csi/

in order to pass all tests, the powerstore csm instance was the only CSI driver that could be deployed. (not documented, possibly a bug in cert-csi) and this label needed to be set in the powerstore CSIDriver (not documented): oc label csidriver csi-powerstore.dellemc.com security.openshift.io/csi-ephemeral-volume-profile=restricted and per the documentation: the NFS storage class used for testing must use volume binding mode WaitForFirstConsumer

[2024-10-04 12:39:59] INFO During this run 100.0% of suites succeeded

dancohen21 commented 1 month ago

the other report related lines....

Collecting metrics 9 / 9 [------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% ? p/s Collecting metrics 10 / 10 [----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% ? p/s [2024-10-04 12:39:44] INFO Started generating reports... Collecting metrics 9 / 9 [------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% ? p/s Generating plots [2024-10-04 12:39:44] ERROR no EntityNumberMetrics provided [2024-10-04 12:39:44] ERROR no EntityNumberMetrics provided 9 / 9 [------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% 2 p/s [2024-10-04 12:39:48] ERROR no EntityMetrics provided [2024-10-04 12:39:48] ERROR no EntityMetrics provided [2024-10-04 12:39:48] WARN No ResourceUsageMetrics provided [2024-10-04 12:39:48] ERROR no ResourceUsageMetrics provided [2024-10-04 12:39:48] WARN no EntityNumberMetrics provided [2024-10-04 12:39:48] ERROR no EntityNumberMetrics provided report-test-run-24573041: Name: test-run-24573041 Host: https://console-openshift-console.apps.ocpcsi.powerflex.lab.dell.com:6443 StorageClass: powerstore-xfs Minimum and Maximum EntityOverTime charts:

/root/.cert-csi/reports/test-run-24573041/PodsCreatingOverTime.png

/root/.cert-csi/reports/test-run-24573041/PodsReadyOverTime.png

/root/.cert-csi/reports/test-run-24573041/PodsTerminatingOverTime.png

/root/.cert-csi/reports/test-run-24573041/PvcsCreatingOverTime.png

/root/.cert-csi/reports/test-run-24573041/PvcsBoundOverTime.png

Tests:

  1. TestCase: EphemeralVolumeSuite Started: 2024-10-04 12:34:37.687804351 -0400 -0400 Ended: 2024-10-04 12:36:02.507089372 -0400 -0400 Result: SUCCESS

        Stage metrics:
            PodCreation:
                Avg: 19.194715414s
                Min: 16.833401067s
                Max: 21.556029761s
                Histogram:
/root/.cert-csi/reports/test-run-24573041/EphemeralVolumeSuite1/PodCreation.png
                BoxPlot:
/root/.cert-csi/reports/test-run-24573041/EphemeralVolumeSuite1/PodCreation_boxplot.png
            PodDeletion:
                Avg: 2.933359136s
                Min: 2.812451225s
                Max: 3.054267048s
                Histogram:
/root/.cert-csi/reports/test-run-24573041/EphemeralVolumeSuite1/PodDeletion.png
                BoxPlot:
/root/.cert-csi/reports/test-run-24573041/EphemeralVolumeSuite1/PodDeletion_boxplot.png
                EntityNumberOverTime:
/root/.cert-csi/reports/test-run-24573041/EphemeralVolumeSuite1/EntityNumberOverTime.png

  2. TestCase: VolumeExpansionSuite Started: 2024-10-04 12:34:37.689685365 -0400 -0400 Ended: 2024-10-04 12:37:03.984773055 -0400 -0400 Result: SUCCESS

        Stage metrics:
            PVCAttachment:
                Avg: 3.237283404s
                Min: 3.237283404s
                Max: 3.237283404s
                Histogram:
/root/.cert-csi/reports/test-run-24573041/VolumeExpansionSuite2/PVCAttachment.png
                BoxPlot:
/root/.cert-csi/reports/test-run-24573041/VolumeExpansionSuite2/PVCAttachment_boxplot.png
            PVCBind:
                Avg: 565.088839ms
                Min: 565.088839ms
                Max: 565.088839ms
                Histogram:
/root/.cert-csi/reports/test-run-24573041/VolumeExpansionSuite2/PVCBind.png
                BoxPlot:
/root/.cert-csi/reports/test-run-24573041/VolumeExpansionSuite2/PVCBind_boxplot.png
            PVCCreation:
                Avg: 8.161312899s
                Min: 8.161312899s
                Max: 8.161312899s
                Histogram:
/root/.cert-csi/reports/test-run-24573041/VolumeExpansionSuite2/PVCCreation.png
                BoxPlot:
/root/.cert-csi/reports/test-run-24573041/VolumeExpansionSuite2/PVCCreation_boxplot.png
            PVCDeletion:
                Avg: 2.601177563s
                Min: 2.601177563s
                Max: 2.601177563s
                Histogram:
/root/.cert-csi/reports/test-run-24573041/VolumeExpansionSuite2/PVCDeletion.png
                BoxPlot:
/root/.cert-csi/reports/test-run-24573041/VolumeExpansionSuite2/PVCDeletion_boxplot.png
            PVCUnattachment:
                Avg: 1.988836296s
                Min: 1.988836296s
                Max: 1.988836296s
                Histogram:
/root/.cert-csi/reports/test-run-24573041/VolumeExpansionSuite2/PVCUnattachment.png
                BoxPlot:
/root/.cert-csi/reports/test-run-24573041/VolumeExpansionSuite2/PVCUnattachment_boxplot.png
            PodCreation:
                Avg: 22.184205474s
                Min: 22.184205474s
                Max: 22.184205474s
                Histogram:
/root/.cert-csi/reports/test-run-24573041/VolumeExpansionSuite2/PodCreation.png
                BoxPlot:
/root/.cert-csi/reports/test-run-24573041/VolumeExpansionSuite2/PodCreation_boxplot.png
            PodDeletion:
                Avg: 2.262804924s
                Min: 2.262804924s
                Max: 2.262804924s
                Histogram:
/root/.cert-csi/reports/test-run-24573041/VolumeExpansionSuite2/PodDeletion.png
                BoxPlot:
/root/.cert-csi/reports/test-run-24573041/VolumeExpansionSuite2/PodDeletion_boxplot.png
                EntityNumberOverTime:
/root/.cert-csi/reports/test-run-24573041/VolumeExpansionSuite2/EntityNumberOverTime.png

  3. TestCase: CloneVolumeSuite Started: 2024-10-04 12:34:37.689246073 -0400 -0400 Ended: 2024-10-04 12:37:09.261046118 -0400 -0400 Result: SUCCESS

        Stage metrics:
            PVCAttachment:
                Avg: 4.133211914s
                Min: 598.41602ms
                Max: 14.009846804s
                Histogram:
/root/.cert-csi/reports/test-run-24573041/CloneVolumeSuite3/PVCAttachment.png
                BoxPlot:
/root/.cert-csi/reports/test-run-24573041/CloneVolumeSuite3/PVCAttachment_boxplot.png
            PVCBind:
                Avg: 832.950188ms
                Min: 283.884363ms
                Max: 1.591478419s
                Histogram:
/root/.cert-csi/reports/test-run-24573041/CloneVolumeSuite3/PVCBind.png
                BoxPlot:
/root/.cert-csi/reports/test-run-24573041/CloneVolumeSuite3/PVCBind_boxplot.png
            PVCCreation:
                Avg: 10.34130723s
                Min: 6.432540108s
                Max: 20.71938461s
                Histogram:
/root/.cert-csi/reports/test-run-24573041/CloneVolumeSuite3/PVCCreation.png
                BoxPlot:
/root/.cert-csi/reports/test-run-24573041/CloneVolumeSuite3/PVCCreation_boxplot.png
            PVCDeletion:
                Avg: 1.488688535s
                Min: 984.398673ms
                Max: 2.007138262s
                Histogram:
/root/.cert-csi/reports/test-run-24573041/CloneVolumeSuite3/PVCDeletion.png
                BoxPlot:
/root/.cert-csi/reports/test-run-24573041/CloneVolumeSuite3/PVCDeletion_boxplot.png
            PVCUnattachment:
                Avg: 957.050354ms
                Min: 439.894266ms
                Max: 1.510557156s
                Histogram:
/root/.cert-csi/reports/test-run-24573041/CloneVolumeSuite3/PVCUnattachment.png
                BoxPlot:
/root/.cert-csi/reports/test-run-24573041/CloneVolumeSuite3/PVCUnattachment_boxplot.png
            PodCreation:
                Avg: 25.543391054s
                Min: 22.469636759s
                Max: 31.240813155s
                Histogram:
/root/.cert-csi/reports/test-run-24573041/CloneVolumeSuite3/PodCreation.png
                BoxPlot:
/root/.cert-csi/reports/test-run-24573041/CloneVolumeSuite3/PodCreation_boxplot.png
            PodDeletion:
                Avg: 2.099468377s
                Min: 1.574403532s
                Max: 2.633970398s
                Histogram:
/root/.cert-csi/reports/test-run-24573041/CloneVolumeSuite3/PodDeletion.png
                BoxPlot:
/root/.cert-csi/reports/test-run-24573041/CloneVolumeSuite3/PodDeletion_boxplot.png
                EntityNumberOverTime:
/root/.cert-csi/reports/test-run-24573041/CloneVolumeSuite3/EntityNumberOverTime.png

  4. TestCase: VolumeIoSuite Started: 2024-10-04 12:34:37.689473631 -0400 -0400 Ended: 2024-10-04 12:37:20.626280586 -0400 -0400 Result: SUCCESS

        Stage metrics:
            PVCBind:
                Avg: 1.277155396s
                Min: 734.578211ms
                Max: 1.819732582s
                Histogram:
/root/.cert-csi/reports/test-run-24573041/VolumeIoSuite4/PVCBind.png
                BoxPlot:
/root/.cert-csi/reports/test-run-24573041/VolumeIoSuite4/PVCBind_boxplot.png
            PVCCreation:
                Avg: 20.3881792s
                Min: 18.645799203s
                Max: 22.130559197s
                Histogram:
/root/.cert-csi/reports/test-run-24573041/VolumeIoSuite4/PVCCreation.png
                BoxPlot:
/root/.cert-csi/reports/test-run-24573041/VolumeIoSuite4/PVCCreation_boxplot.png
            PVCDeletion:
                Avg: 38.670933ms
                Min: 26.285815ms
                Max: 51.056052ms
                Histogram:
/root/.cert-csi/reports/test-run-24573041/VolumeIoSuite4/PVCDeletion.png
                BoxPlot:
/root/.cert-csi/reports/test-run-24573041/VolumeIoSuite4/PVCDeletion_boxplot.png
            PVCUnattachment:
                Avg: 56.715828323s
                Min: 51.268666679s
                Max: 1m2.162989967s
                Histogram:
/root/.cert-csi/reports/test-run-24573041/VolumeIoSuite4/PVCUnattachment.png
                BoxPlot:
/root/.cert-csi/reports/test-run-24573041/VolumeIoSuite4/PVCUnattachment_boxplot.png
            PodCreation:
                Avg: 23.680231806s
                Min: 22.374141787s
                Max: 25.999975569s
                Histogram:
/root/.cert-csi/reports/test-run-24573041/VolumeIoSuite4/PodCreation.png
                BoxPlot:
/root/.cert-csi/reports/test-run-24573041/VolumeIoSuite4/PodCreation_boxplot.png
            PodDeletion:
                Avg: 2.15532427s
                Min: 1.650991749s
                Max: 2.652946386s
                Histogram:
/root/.cert-csi/reports/test-run-24573041/VolumeIoSuite4/PodDeletion.png
                BoxPlot:
/root/.cert-csi/reports/test-run-24573041/VolumeIoSuite4/PodDeletion_boxplot.png
                EntityNumberOverTime:
/root/.cert-csi/reports/test-run-24573041/VolumeIoSuite4/EntityNumberOverTime.png

  5. TestCase: VolumeExpansionSuite Started: 2024-10-04 12:34:37.689749852 -0400 -0400 Ended: 2024-10-04 12:36:11.197082369 -0400 -0400 Result: SUCCESS

        Stage metrics:
            PVCAttachment:
                Avg: 1.316054579s
                Min: 1.316054579s
                Max: 1.316054579s
                Histogram:
/root/.cert-csi/reports/test-run-24573041/VolumeExpansionSuite5/PVCAttachment.png
                BoxPlot:
/root/.cert-csi/reports/test-run-24573041/VolumeExpansionSuite5/PVCAttachment_boxplot.png
            PVCBind:
                Avg: 2.229195722s
                Min: 2.229195722s
                Max: 2.229195722s
                Histogram:
/root/.cert-csi/reports/test-run-24573041/VolumeExpansionSuite5/PVCBind.png
                BoxPlot:
/root/.cert-csi/reports/test-run-24573041/VolumeExpansionSuite5/PVCBind_boxplot.png
            PVCCreation:
                Avg: 8.178829503s
                Min: 8.178829503s
                Max: 8.178829503s
                Histogram:
/root/.cert-csi/reports/test-run-24573041/VolumeExpansionSuite5/PVCCreation.png
                BoxPlot:
/root/.cert-csi/reports/test-run-24573041/VolumeExpansionSuite5/PVCCreation_boxplot.png
            PVCDeletion:
                Avg: 1.058138039s
                Min: 1.058138039s
                Max: 1.058138039s
                Histogram:
/root/.cert-csi/reports/test-run-24573041/VolumeExpansionSuite5/PVCDeletion.png
                BoxPlot:
/root/.cert-csi/reports/test-run-24573041/VolumeExpansionSuite5/PVCDeletion_boxplot.png
            PVCUnattachment:
                Avg: 2.483354249s
                Min: 2.483354249s
                Max: 2.483354249s
                Histogram:
/root/.cert-csi/reports/test-run-24573041/VolumeExpansionSuite5/PVCUnattachment.png
                BoxPlot:
/root/.cert-csi/reports/test-run-24573041/VolumeExpansionSuite5/PVCUnattachment_boxplot.png
            PodCreation:
                Avg: 21.093937141s
                Min: 21.093937141s
                Max: 21.093937141s
                Histogram:
/root/.cert-csi/reports/test-run-24573041/VolumeExpansionSuite5/PodCreation.png
                BoxPlot:
/root/.cert-csi/reports/test-run-24573041/VolumeExpansionSuite5/PodCreation_boxplot.png
            PodDeletion:
                Avg: 1.866667596s
                Min: 1.866667596s
                Max: 1.866667596s
                Histogram:
/root/.cert-csi/reports/test-run-24573041/VolumeExpansionSuite5/PodDeletion.png
                BoxPlot:
/root/.cert-csi/reports/test-run-24573041/VolumeExpansionSuite5/PodDeletion_boxplot.png
                EntityNumberOverTime:
/root/.cert-csi/reports/test-run-24573041/VolumeExpansionSuite5/EntityNumberOverTime.png

  6. TestCase: ReplicationSuite Started: 2024-10-04 12:34:37.689702162 -0400 -0400 Ended: 2024-10-04 12:39:42.915966371 -0400 -0400 Result: SUCCESS

        Stage metrics:
            PVCAttachment:
                Avg: 4.861058906s
                Min: 1.240078355s
                Max: 15.447572767s
                Histogram:
/root/.cert-csi/reports/test-run-24573041/ReplicationSuite6/PVCAttachment.png
                BoxPlot:
/root/.cert-csi/reports/test-run-24573041/ReplicationSuite6/PVCAttachment_boxplot.png
            PVCBind:
                Avg: 758.28324ms
                Min: 250.489471ms
                Max: 2.136654672s
                Histogram:
/root/.cert-csi/reports/test-run-24573041/ReplicationSuite6/PVCBind.png
                BoxPlot:
/root/.cert-csi/reports/test-run-24573041/ReplicationSuite6/PVCBind_boxplot.png
            PVCCreation:
                Avg: 14.833966223s
                Min: 2.12677197s
                Max: 42.893961055s
                Histogram:
/root/.cert-csi/reports/test-run-24573041/ReplicationSuite6/PVCCreation.png
                BoxPlot:
/root/.cert-csi/reports/test-run-24573041/ReplicationSuite6/PVCCreation_boxplot.png
            PVCDeletion:
                Avg: 5.124497849s
                Min: 4.698928424s
                Max: 5.458085714s
                Histogram:
/root/.cert-csi/reports/test-run-24573041/ReplicationSuite6/PVCDeletion.png
                BoxPlot:
/root/.cert-csi/reports/test-run-24573041/ReplicationSuite6/PVCDeletion_boxplot.png
            PVCUnattachment:
                Avg: 10.171722425s
                Min: 408.127577ms
                Max: 15.156136286s
                Histogram:
/root/.cert-csi/reports/test-run-24573041/ReplicationSuite6/PVCUnattachment.png
                BoxPlot:
/root/.cert-csi/reports/test-run-24573041/ReplicationSuite6/PVCUnattachment_boxplot.png
            PodCreation:
                Avg: 26.69846549s
                Min: 23.099896926s
                Max: 34.978628522s
                Histogram:
/root/.cert-csi/reports/test-run-24573041/ReplicationSuite6/PodCreation.png
                BoxPlot:
/root/.cert-csi/reports/test-run-24573041/ReplicationSuite6/PodCreation_boxplot.png
            PodDeletion:
                Avg: 3.863552313s
                Min: 3.619242547s
                Max: 4.013578886s
                Histogram:
/root/.cert-csi/reports/test-run-24573041/ReplicationSuite6/PodDeletion.png
                BoxPlot:
/root/.cert-csi/reports/test-run-24573041/ReplicationSuite6/PodDeletion_boxplot.png
                EntityNumberOverTime:
/root/.cert-csi/reports/test-run-24573041/ReplicationSuite6/EntityNumberOverTime.png

  7. TestCase: ScalingSuite Started: 2024-10-04 12:34:37.689487997 -0400 -0400 Ended: 2024-10-04 12:36:13.381294001 -0400 -0400 Result: SUCCESS

        Stage metrics:
            PVCAttachment:
                Avg: 2.921902627s
                Min: 1.708824081s
                Max: 4.002384434s
                Histogram:
/root/.cert-csi/reports/test-run-24573041/ScalingSuite7/PVCAttachment.png
                BoxPlot:
/root/.cert-csi/reports/test-run-24573041/ScalingSuite7/PVCAttachment_boxplot.png
            PVCBind:
                Avg: 778.375349ms
                Min: 515.621397ms
                Max: 1.404951618s
                Histogram:
/root/.cert-csi/reports/test-run-24573041/ScalingSuite7/PVCBind.png
                BoxPlot:
/root/.cert-csi/reports/test-run-24573041/ScalingSuite7/PVCBind_boxplot.png
            PVCCreation:
                Avg: 4.369169614s
                Min: 2.979908565s
                Max: 5.605603883s
                Histogram:
/root/.cert-csi/reports/test-run-24573041/ScalingSuite7/PVCCreation.png
                BoxPlot:
/root/.cert-csi/reports/test-run-24573041/ScalingSuite7/PVCCreation_boxplot.png
            PVCDeletion:
                Avg: 109.315277ms
                Min: 32.193829ms
                Max: 224.122486ms
                Histogram:
/root/.cert-csi/reports/test-run-24573041/ScalingSuite7/PVCDeletion.png
                BoxPlot:
/root/.cert-csi/reports/test-run-24573041/ScalingSuite7/PVCDeletion_boxplot.png
            PVCUnattachment:
                Avg: 4.534937099s
                Min: 2.479457307s
                Max: 6.099121713s
                Histogram:
/root/.cert-csi/reports/test-run-24573041/ScalingSuite7/PVCUnattachment.png
                BoxPlot:
/root/.cert-csi/reports/test-run-24573041/ScalingSuite7/PVCUnattachment_boxplot.png
            PodCreation:
                Avg: 25.920953591s
                Min: 24.779862276s
                Max: 27.062044906s
                Histogram:
/root/.cert-csi/reports/test-run-24573041/ScalingSuite7/PodCreation.png
                BoxPlot:
/root/.cert-csi/reports/test-run-24573041/ScalingSuite7/PodCreation_boxplot.png
            PodDeletion:
                Avg: 2.940808049s
                Min: 1.866474178s
                Max: 4.015141921s
                Histogram:
/root/.cert-csi/reports/test-run-24573041/ScalingSuite7/PodDeletion.png
                BoxPlot:
/root/.cert-csi/reports/test-run-24573041/ScalingSuite7/PodDeletion_boxplot.png
                EntityNumberOverTime:
/root/.cert-csi/reports/test-run-24573041/ScalingSuite7/EntityNumberOverTime.png

  8. TestCase: SnapSuite Started: 2024-10-04 12:34:37.689704885 -0400 -0400 Ended: 2024-10-04 12:37:19.054550451 -0400 -0400 Result: SUCCESS

        Stage metrics:
            PVCAttachment:
                Avg: 1.320166727s
                Min: 1.042561035s
                Max: 1.59777242s
                Histogram:
/root/.cert-csi/reports/test-run-24573041/SnapSuite8/PVCAttachment.png
                BoxPlot:
/root/.cert-csi/reports/test-run-24573041/SnapSuite8/PVCAttachment_boxplot.png
            PVCBind:
                Avg: 1.279564355s
                Min: 762.753113ms
                Max: 1.796375598s
                Histogram:
/root/.cert-csi/reports/test-run-24573041/SnapSuite8/PVCBind.png
                BoxPlot:
/root/.cert-csi/reports/test-run-24573041/SnapSuite8/PVCBind_boxplot.png
            PVCCreation:
                Avg: 11.896859718s
                Min: 9.96718104s
                Max: 13.826538397s
                Histogram:
/root/.cert-csi/reports/test-run-24573041/SnapSuite8/PVCCreation.png
                BoxPlot:
/root/.cert-csi/reports/test-run-24573041/SnapSuite8/PVCCreation_boxplot.png
            PVCDeletion:
                Avg: 1.396286992s
                Min: 24.750762ms
                Max: 2.767823222s
                Histogram:
/root/.cert-csi/reports/test-run-24573041/SnapSuite8/PVCDeletion.png
                BoxPlot:
/root/.cert-csi/reports/test-run-24573041/SnapSuite8/PVCDeletion_boxplot.png
            PVCUnattachment:
                Avg: 6.043416624s
                Min: 512.585375ms
                Max: 11.574247873s
                Histogram:
/root/.cert-csi/reports/test-run-24573041/SnapSuite8/PVCUnattachment.png
                BoxPlot:
/root/.cert-csi/reports/test-run-24573041/SnapSuite8/PVCUnattachment_boxplot.png
            PodCreation:
                Avg: 23.946894079s
                Min: 23.750437867s
                Max: 24.143350291s
                Histogram:
/root/.cert-csi/reports/test-run-24573041/SnapSuite8/PodCreation.png
                BoxPlot:
/root/.cert-csi/reports/test-run-24573041/SnapSuite8/PodCreation_boxplot.png
            PodDeletion:
                Avg: 1.48187303s
                Min: 1.400637083s
                Max: 1.563108978s
                Histogram:
/root/.cert-csi/reports/test-run-24573041/SnapSuite8/PodDeletion.png
                BoxPlot:
/root/.cert-csi/reports/test-run-24573041/SnapSuite8/PodDeletion_boxplot.png
                EntityNumberOverTime:
/root/.cert-csi/reports/test-run-24573041/SnapSuite8/EntityNumberOverTime.png

  9. TestCase: MultiAttachSuite Started: 2024-10-04 12:34:37.689727139 -0400 -0400 Ended: 2024-10-04 12:37:04.453488828 -0400 -0400 Result: SUCCESS

        Stage metrics:
            PVCAttachment:
                Avg: 1.288947423s
                Min: 1.288947423s
                Max: 1.288947423s
                Histogram:
/root/.cert-csi/reports/test-run-24573041/MultiAttachSuite9/PVCAttachment.png
                BoxPlot:
/root/.cert-csi/reports/test-run-24573041/MultiAttachSuite9/PVCAttachment_boxplot.png
            PVCBind:
                Avg: 706.63899ms
                Min: 706.63899ms
                Max: 706.63899ms
                Histogram:
/root/.cert-csi/reports/test-run-24573041/MultiAttachSuite9/PVCBind.png
                BoxPlot:
/root/.cert-csi/reports/test-run-24573041/MultiAttachSuite9/PVCBind_boxplot.png
            PVCCreation:
                Avg: 48.152330244s
                Min: 48.152330244s
                Max: 48.152330244s
                Histogram:
/root/.cert-csi/reports/test-run-24573041/MultiAttachSuite9/PVCCreation.png
                BoxPlot:
/root/.cert-csi/reports/test-run-24573041/MultiAttachSuite9/PVCCreation_boxplot.png
            PVCDeletion:
                Avg: 4.094899293s
                Min: 4.094899293s
                Max: 4.094899293s
                Histogram:
/root/.cert-csi/reports/test-run-24573041/MultiAttachSuite9/PVCDeletion.png
                BoxPlot:
/root/.cert-csi/reports/test-run-24573041/MultiAttachSuite9/PVCDeletion_boxplot.png
            PVCUnattachment:
                Avg: 1.349612996s
                Min: 1.349612996s
                Max: 1.349612996s
                Histogram:
/root/.cert-csi/reports/test-run-24573041/MultiAttachSuite9/PVCUnattachment.png
                BoxPlot:
/root/.cert-csi/reports/test-run-24573041/MultiAttachSuite9/PVCUnattachment_boxplot.png
            PodCreation:
                Avg: 10.257955844s
                Min: 3.163360749s
                Max: 20.869931342s
                Histogram:
/root/.cert-csi/reports/test-run-24573041/MultiAttachSuite9/PodCreation.png
                BoxPlot:
/root/.cert-csi/reports/test-run-24573041/MultiAttachSuite9/PodCreation_boxplot.png
            PodDeletion:
                Avg: 2.467656243s
                Min: 1.912648824s
                Max: 3.11774029s
                Histogram:
/root/.cert-csi/reports/test-run-24573041/MultiAttachSuite9/PodDeletion.png
                BoxPlot:
/root/.cert-csi/reports/test-run-24573041/MultiAttachSuite9/PodDeletion_boxplot.png
                EntityNumberOverTime:
/root/.cert-csi/reports/test-run-24573041/MultiAttachSuite9/EntityNumberOverTime.png

Collecting metrics 10 / 10 [----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% ? p/s Generating plots 7 / 10 [---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------->___] 70.00% 2 p/s[2024-10-04 12:39:51] ERROR no EntityNumberMetrics provided [2024-10-04 12:39:51] ERROR no EntityNumberMetrics provided 10 / 10 [----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% 3 p/s [2024-10-04 12:39:53] ERROR MinMaxEoT: No EntityNumberMetrics provided in EphemeralVolumeSuite7 [2024-10-04 12:39:53] WARN No ResourceUsageMetrics provided [2024-10-04 12:39:53] ERROR no ResourceUsageMetrics provided [2024-10-04 12:39:53] ERROR IterTimes: No EntityNumberMetrics provided in EphemeralVolumeSuite7 report-test-run-8b6f4843: Name: test-run-8b6f4843 Host: https://console-openshift-console.apps.ocpcsi.powerflex.lab.dell.com:6443 StorageClass: powerstore-nfs-waitforfirstconsumer Minimum and Maximum EntityOverTime charts:

/root/.cert-csi/reports/test-run-8b6f4843/PodsCreatingOverTime.png

/root/.cert-csi/reports/test-run-8b6f4843/PodsReadyOverTime.png

/root/.cert-csi/reports/test-run-8b6f4843/PodsTerminatingOverTime.png

/root/.cert-csi/reports/test-run-8b6f4843/PvcsCreatingOverTime.png

/root/.cert-csi/reports/test-run-8b6f4843/PvcsBoundOverTime.png

Tests:

  1. TestCase: CapacityTrackingSuite Started: 2024-10-04 12:34:37.687516239 -0400 -0400 Ended: 2024-10-04 12:36:32.896942472 -0400 -0400 Result: SUCCESS

        Stage metrics:
            PVCAttachment:
                Avg: 751.671751ms
                Min: 751.671751ms
                Max: 751.671751ms
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/CapacityTrackingSuite1/PVCAttachment.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/CapacityTrackingSuite1/PVCAttachment_boxplot.png
            PVCBind:
                Avg: 13.991156213s
                Min: 13.991156213s
                Max: 13.991156213s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/CapacityTrackingSuite1/PVCBind.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/CapacityTrackingSuite1/PVCBind_boxplot.png
            PVCCreation:
                Avg: 14.991432904s
                Min: 14.991432904s
                Max: 14.991432904s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/CapacityTrackingSuite1/PVCCreation.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/CapacityTrackingSuite1/PVCCreation_boxplot.png
            PVCDeletion:
                Avg: 2.668733533s
                Min: 2.668733533s
                Max: 2.668733533s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/CapacityTrackingSuite1/PVCDeletion.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/CapacityTrackingSuite1/PVCDeletion_boxplot.png
            PVCUnattachment:
                Avg: 1.669407306s
                Min: 1.669407306s
                Max: 1.669407306s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/CapacityTrackingSuite1/PVCUnattachment.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/CapacityTrackingSuite1/PVCUnattachment_boxplot.png
            PodCreation:
                Avg: 15.292047567s
                Min: 15.292047567s
                Max: 15.292047567s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/CapacityTrackingSuite1/PodCreation.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/CapacityTrackingSuite1/PodCreation_boxplot.png
            PodDeletion:
                Avg: 2.356053194s
                Min: 2.356053194s
                Max: 2.356053194s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/CapacityTrackingSuite1/PodDeletion.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/CapacityTrackingSuite1/PodDeletion_boxplot.png
                EntityNumberOverTime:
/root/.cert-csi/reports/test-run-8b6f4843/CapacityTrackingSuite1/EntityNumberOverTime.png

  2. TestCase: MultiAttachSuite Started: 2024-10-04 12:34:37.689126065 -0400 -0400 Ended: 2024-10-04 12:36:32.924458054 -0400 -0400 Result: SUCCESS

        Stage metrics:
            PVCAttachment:
                Avg: 630.806704ms
                Min: 630.806704ms
                Max: 630.806704ms
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/MultiAttachSuite2/PVCAttachment.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/MultiAttachSuite2/PVCAttachment_boxplot.png
            PVCBind:
                Avg: 8.577451373s
                Min: 8.577451373s
                Max: 8.577451373s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/MultiAttachSuite2/PVCBind.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/MultiAttachSuite2/PVCBind_boxplot.png
            PVCCreation:
                Avg: 46.896870663s
                Min: 46.896870663s
                Max: 46.896870663s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/MultiAttachSuite2/PVCCreation.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/MultiAttachSuite2/PVCCreation_boxplot.png
            PVCDeletion:
                Avg: 2.366033405s
                Min: 2.366033405s
                Max: 2.366033405s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/MultiAttachSuite2/PVCDeletion.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/MultiAttachSuite2/PVCDeletion_boxplot.png
            PVCUnattachment:
                Avg: 756.539536ms
                Min: 756.539536ms
                Max: 756.539536ms
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/MultiAttachSuite2/PVCUnattachment.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/MultiAttachSuite2/PVCUnattachment_boxplot.png
            PodCreation:
                Avg: 8.997402352s
                Min: 3.841571255s
                Max: 23.735200742s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/MultiAttachSuite2/PodCreation.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/MultiAttachSuite2/PodCreation_boxplot.png
            PodDeletion:
                Avg: 2.500929049s
                Min: 2.024359634s
                Max: 3.235093529s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/MultiAttachSuite2/PodDeletion.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/MultiAttachSuite2/PodDeletion_boxplot.png
                EntityNumberOverTime:
/root/.cert-csi/reports/test-run-8b6f4843/MultiAttachSuite2/EntityNumberOverTime.png

  3. TestCase: ReplicationSuite Started: 2024-10-04 12:34:37.689201432 -0400 -0400 Ended: 2024-10-04 12:39:44.388543181 -0400 -0400 Result: SUCCESS

        Stage metrics:
            PVCAttachment:
                Avg: 3.821428774s
                Min: 869.002831ms
                Max: 11.315248474s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/ReplicationSuite3/PVCAttachment.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/ReplicationSuite3/PVCAttachment_boxplot.png
            PVCBind:
                Avg: 20.908383412s
                Min: 8.54482872s
                Max: 45.199747071s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/ReplicationSuite3/PVCBind.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/ReplicationSuite3/PVCBind_boxplot.png
            PVCCreation:
                Avg: 27.22932091s
                Min: 10.992115438s
                Max: 57.060543855s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/ReplicationSuite3/PVCCreation.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/ReplicationSuite3/PVCCreation_boxplot.png
            PVCDeletion:
                Avg: 348.00648ms
                Min: 111.5971ms
                Max: 611.637886ms
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/ReplicationSuite3/PVCDeletion.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/ReplicationSuite3/PVCDeletion_boxplot.png
            PVCUnattachment:
                Avg: 8.268226907s
                Min: 1.23367879s
                Max: 12.705474558s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/ReplicationSuite3/PVCUnattachment.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/ReplicationSuite3/PVCUnattachment_boxplot.png
            PodCreation:
                Avg: 26.968298821s
                Min: 20.057533562s
                Max: 34.20463142s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/ReplicationSuite3/PodCreation.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/ReplicationSuite3/PodCreation_boxplot.png
            PodDeletion:
                Avg: 2.508734668s
                Min: 1.846622152s
                Max: 3.690412628s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/ReplicationSuite3/PodDeletion.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/ReplicationSuite3/PodDeletion_boxplot.png
                EntityNumberOverTime:
/root/.cert-csi/reports/test-run-8b6f4843/ReplicationSuite3/EntityNumberOverTime.png

  4. TestCase: SnapSuite Started: 2024-10-04 12:34:37.689177129 -0400 -0400 Ended: 2024-10-04 12:37:08.435812543 -0400 -0400 Result: SUCCESS

        Stage metrics:
            PVCAttachment:
                Avg: 1.134678671s
                Min: 397.83468ms
                Max: 1.871522662s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/SnapSuite4/PVCAttachment.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/SnapSuite4/PVCAttachment_boxplot.png
            PVCBind:
                Avg: 13.949529247s
                Min: 12.917692156s
                Max: 14.981366338s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/SnapSuite4/PVCBind.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/SnapSuite4/PVCBind_boxplot.png
            PVCCreation:
                Avg: 15.599203984s
                Min: 13.669956379s
                Max: 17.528451589s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/SnapSuite4/PVCCreation.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/SnapSuite4/PVCCreation_boxplot.png
            PVCDeletion:
                Avg: 990.54237ms
                Min: 29.401729ms
                Max: 1.951683011s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/SnapSuite4/PVCDeletion.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/SnapSuite4/PVCDeletion_boxplot.png
            PVCUnattachment:
                Avg: 491.60832ms
                Min: 269.166654ms
                Max: 714.049987ms
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/SnapSuite4/PVCUnattachment.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/SnapSuite4/PVCUnattachment_boxplot.png
            PodCreation:
                Avg: 18.590780993s
                Min: 14.410337579s
                Max: 22.771224407s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/SnapSuite4/PodCreation.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/SnapSuite4/PodCreation_boxplot.png
            PodDeletion:
                Avg: 2.540745986s
                Min: 2.13095443s
                Max: 2.950537542s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/SnapSuite4/PodDeletion.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/SnapSuite4/PodDeletion_boxplot.png
                EntityNumberOverTime:
/root/.cert-csi/reports/test-run-8b6f4843/SnapSuite4/EntityNumberOverTime.png

  5. TestCase: VolumeExpansionSuite Started: 2024-10-04 12:34:37.68766529 -0400 -0400 Ended: 2024-10-04 12:36:20.206314669 -0400 -0400 Result: SUCCESS

        Stage metrics:
            PVCAttachment:
                Avg: 2.006547623s
                Min: 2.006547623s
                Max: 2.006547623s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/VolumeExpansionSuite5/PVCAttachment.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/VolumeExpansionSuite5/PVCAttachment_boxplot.png
            PVCBind:
                Avg: 14.983559134s
                Min: 14.983559134s
                Max: 14.983559134s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/VolumeExpansionSuite5/PVCBind.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/VolumeExpansionSuite5/PVCBind_boxplot.png
            PVCCreation:
                Avg: 17.076041862s
                Min: 17.076041862s
                Max: 17.076041862s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/VolumeExpansionSuite5/PVCCreation.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/VolumeExpansionSuite5/PVCCreation_boxplot.png
            PVCDeletion:
                Avg: 2.91110793s
                Min: 2.91110793s
                Max: 2.91110793s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/VolumeExpansionSuite5/PVCDeletion.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/VolumeExpansionSuite5/PVCDeletion_boxplot.png
            PVCUnattachment:
                Avg: 10.454575812s
                Min: 10.454575812s
                Max: 10.454575812s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/VolumeExpansionSuite5/PVCUnattachment.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/VolumeExpansionSuite5/PVCUnattachment_boxplot.png
            PodCreation:
                Avg: 22.082186658s
                Min: 22.082186658s
                Max: 22.082186658s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/VolumeExpansionSuite5/PodCreation.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/VolumeExpansionSuite5/PodCreation_boxplot.png
            PodDeletion:
                Avg: 1.653746363s
                Min: 1.653746363s
                Max: 1.653746363s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/VolumeExpansionSuite5/PodDeletion.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/VolumeExpansionSuite5/PodDeletion_boxplot.png
                EntityNumberOverTime:
/root/.cert-csi/reports/test-run-8b6f4843/VolumeExpansionSuite5/EntityNumberOverTime.png

  6. TestCase: MultiAttachSuite Started: 2024-10-04 12:34:37.689226039 -0400 -0400 Ended: 2024-10-04 12:37:11.603155635 -0400 -0400 Result: SUCCESS

        Stage metrics:
            PVCAttachment:
                Avg: 713.361887ms
                Min: 713.361887ms
                Max: 713.361887ms
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/MultiAttachSuite6/PVCAttachment.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/MultiAttachSuite6/PVCAttachment_boxplot.png
            PVCBind:
                Avg: 18.360088032s
                Min: 18.360088032s
                Max: 18.360088032s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/MultiAttachSuite6/PVCBind.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/MultiAttachSuite6/PVCBind_boxplot.png
            PVCCreation:
                Avg: 19.795622612s
                Min: 19.795622612s
                Max: 19.795622612s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/MultiAttachSuite6/PVCCreation.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/MultiAttachSuite6/PVCCreation_boxplot.png
            PVCDeletion:
                Avg: 2.19058085s
                Min: 2.19058085s
                Max: 2.19058085s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/MultiAttachSuite6/PVCDeletion.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/MultiAttachSuite6/PVCDeletion_boxplot.png
            PVCUnattachment:
                Avg: 369.453431ms
                Min: 369.453431ms
                Max: 369.453431ms
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/MultiAttachSuite6/PVCUnattachment.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/MultiAttachSuite6/PVCUnattachment_boxplot.png
            PodDeletion:
                Avg: 222.513404ms
                Min: 3.620669ms
                Max: 1.30029054s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/MultiAttachSuite6/PodDeletion.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/MultiAttachSuite6/PodDeletion_boxplot.png
                EntityNumberOverTime:
/root/.cert-csi/reports/test-run-8b6f4843/MultiAttachSuite6/EntityNumberOverTime.png

  7. TestCase: EphemeralVolumeSuite Started: 2024-10-04 12:34:37.689654242 -0400 -0400 Ended: 2024-10-04 12:36:00.316199943 -0400 -0400 Result: SUCCESS

        Stage metrics:
            PodCreation:
                Avg: 16.162724546s
                Min: 16.13541781s
                Max: 16.190031282s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/EphemeralVolumeSuite7/PodCreation.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/EphemeralVolumeSuite7/PodCreation_boxplot.png
            PodDeletion:
                Avg: 7.781542281s
                Min: 7.205850782s
                Max: 8.35723378s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/EphemeralVolumeSuite7/PodDeletion.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/EphemeralVolumeSuite7/PodDeletion_boxplot.png
                EntityNumberOverTime:
/root/.cert-csi/reports/test-run-8b6f4843/EphemeralVolumeSuite7/EntityNumberOverTime.png

  8. TestCase: CloneVolumeSuite Started: 2024-10-04 12:34:37.68940025 -0400 -0400 Ended: 2024-10-04 12:36:43.055901344 -0400 -0400 Result: SUCCESS

        Stage metrics:
            PVCAttachment:
                Avg: 3.542858601s
                Min: 869.246455ms
                Max: 10.567904135s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/CloneVolumeSuite8/PVCAttachment.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/CloneVolumeSuite8/PVCAttachment_boxplot.png
            PVCBind:
                Avg: 14.088944241s
                Min: 12.750291382s
                Max: 16.355616657s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/CloneVolumeSuite8/PVCBind.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/CloneVolumeSuite8/PVCBind_boxplot.png
            PVCCreation:
                Avg: 18.19032995s
                Min: 14.17852033s
                Max: 24.066669068s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/CloneVolumeSuite8/PVCCreation.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/CloneVolumeSuite8/PVCCreation_boxplot.png
            PVCDeletion:
                Avg: 1.950150932s
                Min: 1.317335307s
                Max: 2.181042371s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/CloneVolumeSuite8/PVCDeletion.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/CloneVolumeSuite8/PVCDeletion_boxplot.png
            PVCUnattachment:
                Avg: 389.661369ms
                Min: 194.559316ms
                Max: 544.463118ms
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/CloneVolumeSuite8/PVCUnattachment.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/CloneVolumeSuite8/PVCUnattachment_boxplot.png
            PodCreation:
                Avg: 17.936776271s
                Min: 13.308363729s
                Max: 21.994192319s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/CloneVolumeSuite8/PodCreation.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/CloneVolumeSuite8/PodCreation_boxplot.png
            PodDeletion:
                Avg: 2.213069309s
                Min: 1.595496595s
                Max: 2.470374283s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/CloneVolumeSuite8/PodDeletion.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/CloneVolumeSuite8/PodDeletion_boxplot.png
                EntityNumberOverTime:
/root/.cert-csi/reports/test-run-8b6f4843/CloneVolumeSuite8/EntityNumberOverTime.png

  9. TestCase: VolumeIoSuite Started: 2024-10-04 12:34:37.689369543 -0400 -0400 Ended: 2024-10-04 12:37:03.35215891 -0400 -0400 Result: SUCCESS

        Stage metrics:
            PVCBind:
                Avg: 21.372622089s
                Min: 20.432406702s
                Max: 22.312837476s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/VolumeIoSuite9/PVCBind.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/VolumeIoSuite9/PVCBind_boxplot.png
            PVCCreation:
                Avg: 22.757326597s
                Min: 21.056758652s
                Max: 24.457894542s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/VolumeIoSuite9/PVCCreation.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/VolumeIoSuite9/PVCCreation_boxplot.png
            PVCDeletion:
                Avg: 43.299976ms
                Min: 34.300494ms
                Max: 52.299459ms
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/VolumeIoSuite9/PVCDeletion.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/VolumeIoSuite9/PVCDeletion_boxplot.png
            PVCUnattachment:
                Avg: 36.684611471s
                Min: 32.17693862s
                Max: 41.192284322s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/VolumeIoSuite9/PVCUnattachment.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/VolumeIoSuite9/PVCUnattachment_boxplot.png
            PodCreation:
                Avg: 16.738193415s
                Min: 10.695745258s
                Max: 22.358716257s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/VolumeIoSuite9/PodCreation.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/VolumeIoSuite9/PodCreation_boxplot.png
            PodDeletion:
                Avg: 1.874529961s
                Min: 1.26175163s
                Max: 2.213538424s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/VolumeIoSuite9/PodDeletion.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/VolumeIoSuite9/PodDeletion_boxplot.png
                EntityNumberOverTime:
/root/.cert-csi/reports/test-run-8b6f4843/VolumeIoSuite9/EntityNumberOverTime.png

  10. TestCase: ScalingSuite Started: 2024-10-04 12:34:37.689386007 -0400 -0400 Ended: 2024-10-04 12:36:33.83682364 -0400 -0400 Result: SUCCESS

        Stage metrics:
            PVCAttachment:
                Avg: 1.594739351s
                Min: 1.143052326s
                Max: 2.197054939s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/ScalingSuite10/PVCAttachment.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/ScalingSuite10/PVCAttachment_boxplot.png
            PVCBind:
                Avg: 13.41748631s
                Min: 9.040850746s
                Max: 14.801462701s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/ScalingSuite10/PVCBind.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/ScalingSuite10/PVCBind_boxplot.png
            PVCCreation:
                Avg: 16.331865249s
                Min: 15.404106487s
                Max: 17.374790908s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/ScalingSuite10/PVCCreation.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/ScalingSuite10/PVCCreation_boxplot.png
            PVCDeletion:
                Avg: 153.322385ms
                Min: 52.168708ms
                Max: 280.077538ms
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/ScalingSuite10/PVCDeletion.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/ScalingSuite10/PVCDeletion_boxplot.png
            PVCUnattachment:
                Avg: 10.10410353s
                Min: 8.701869968s
                Max: 11.066056592s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/ScalingSuite10/PVCUnattachment.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/ScalingSuite10/PVCUnattachment_boxplot.png
            PodCreation:
                Avg: 28.124376651s
                Min: 25.78857683s
                Max: 30.460176473s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/ScalingSuite10/PodCreation.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/ScalingSuite10/PodCreation_boxplot.png
            PodDeletion:
                Avg: 2.336812261s
                Min: 1.925177571s
                Max: 2.748446951s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/ScalingSuite10/PodDeletion.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/ScalingSuite10/PodDeletion_boxplot.png
                EntityNumberOverTime:
/root/.cert-csi/reports/test-run-8b6f4843/ScalingSuite10/EntityNumberOverTime.png

[2024-10-04 12:39:53] INFO Started generating reports... [2024-10-04 12:39:53] ERROR Couldn't get test run by name test-run-24573041 [2024-10-04 12:39:53] WARN unable to collect metrics for the test run: test-run-24573041 Collecting metrics 10 / 10 [----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% ? p/s Generating plots 6 / 10 [--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------->__] 60.00% 2 p/s[2024-10-04 12:39:56] ERROR no EntityNumberMetrics provided [2024-10-04 12:39:56] ERROR no EntityNumberMetrics provided 10 / 10 [----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% 2 p/s [2024-10-04 12:39:58] ERROR MinMaxEoT: No EntityNumberMetrics provided in EphemeralVolumeSuite7 [2024-10-04 12:39:58] WARN No ResourceUsageMetrics provided [2024-10-04 12:39:58] ERROR no ResourceUsageMetrics provided [2024-10-04 12:39:58] ERROR IterTimes: No EntityNumberMetrics provided in EphemeralVolumeSuite7 report-test-run-8b6f4843: Name: test-run-8b6f4843 Host: https://console-openshift-console.apps.ocpcsi.powerflex.lab.dell.com:6443 StorageClass: powerstore-nfs-waitforfirstconsumer Minimum and Maximum EntityOverTime charts:

/root/.cert-csi/reports/test-run-8b6f4843/PodsCreatingOverTime.png

/root/.cert-csi/reports/test-run-8b6f4843/PodsReadyOverTime.png

/root/.cert-csi/reports/test-run-8b6f4843/PodsTerminatingOverTime.png

/root/.cert-csi/reports/test-run-8b6f4843/PvcsCreatingOverTime.png

/root/.cert-csi/reports/test-run-8b6f4843/PvcsBoundOverTime.png

Tests:

  1. TestCase: CapacityTrackingSuite Started: 2024-10-04 12:34:37.687516239 -0400 -0400 Ended: 2024-10-04 12:36:32.896942472 -0400 -0400 Result: SUCCESS

        Stage metrics:
            PVCAttachment:
                Avg: 751.671751ms
                Min: 751.671751ms
                Max: 751.671751ms
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/CapacityTrackingSuite1/PVCAttachment.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/CapacityTrackingSuite1/PVCAttachment_boxplot.png
            PVCBind:
                Avg: 13.991156213s
                Min: 13.991156213s
                Max: 13.991156213s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/CapacityTrackingSuite1/PVCBind.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/CapacityTrackingSuite1/PVCBind_boxplot.png
            PVCCreation:
                Avg: 14.991432904s
                Min: 14.991432904s
                Max: 14.991432904s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/CapacityTrackingSuite1/PVCCreation.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/CapacityTrackingSuite1/PVCCreation_boxplot.png
            PVCDeletion:
                Avg: 2.668733533s
                Min: 2.668733533s
                Max: 2.668733533s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/CapacityTrackingSuite1/PVCDeletion.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/CapacityTrackingSuite1/PVCDeletion_boxplot.png
            PVCUnattachment:
                Avg: 1.669407306s
                Min: 1.669407306s
                Max: 1.669407306s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/CapacityTrackingSuite1/PVCUnattachment.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/CapacityTrackingSuite1/PVCUnattachment_boxplot.png
            PodCreation:
                Avg: 15.292047567s
                Min: 15.292047567s
                Max: 15.292047567s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/CapacityTrackingSuite1/PodCreation.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/CapacityTrackingSuite1/PodCreation_boxplot.png
            PodDeletion:
                Avg: 2.356053194s
                Min: 2.356053194s
                Max: 2.356053194s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/CapacityTrackingSuite1/PodDeletion.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/CapacityTrackingSuite1/PodDeletion_boxplot.png
                EntityNumberOverTime:
/root/.cert-csi/reports/test-run-8b6f4843/CapacityTrackingSuite1/EntityNumberOverTime.png

  2. TestCase: MultiAttachSuite Started: 2024-10-04 12:34:37.689126065 -0400 -0400 Ended: 2024-10-04 12:36:32.924458054 -0400 -0400 Result: SUCCESS

        Stage metrics:
            PVCAttachment:
                Avg: 630.806704ms
                Min: 630.806704ms
                Max: 630.806704ms
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/MultiAttachSuite2/PVCAttachment.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/MultiAttachSuite2/PVCAttachment_boxplot.png
            PVCBind:
                Avg: 8.577451373s
                Min: 8.577451373s
                Max: 8.577451373s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/MultiAttachSuite2/PVCBind.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/MultiAttachSuite2/PVCBind_boxplot.png
            PVCCreation:
                Avg: 46.896870663s
                Min: 46.896870663s
                Max: 46.896870663s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/MultiAttachSuite2/PVCCreation.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/MultiAttachSuite2/PVCCreation_boxplot.png
            PVCDeletion:
                Avg: 2.366033405s
                Min: 2.366033405s
                Max: 2.366033405s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/MultiAttachSuite2/PVCDeletion.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/MultiAttachSuite2/PVCDeletion_boxplot.png
            PVCUnattachment:
                Avg: 756.539536ms
                Min: 756.539536ms
                Max: 756.539536ms
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/MultiAttachSuite2/PVCUnattachment.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/MultiAttachSuite2/PVCUnattachment_boxplot.png
            PodCreation:
                Avg: 8.997402352s
                Min: 3.841571255s
                Max: 23.735200742s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/MultiAttachSuite2/PodCreation.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/MultiAttachSuite2/PodCreation_boxplot.png
            PodDeletion:
                Avg: 2.500929049s
                Min: 2.024359634s
                Max: 3.235093529s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/MultiAttachSuite2/PodDeletion.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/MultiAttachSuite2/PodDeletion_boxplot.png
                EntityNumberOverTime:
/root/.cert-csi/reports/test-run-8b6f4843/MultiAttachSuite2/EntityNumberOverTime.png

  3. TestCase: ReplicationSuite Started: 2024-10-04 12:34:37.689201432 -0400 -0400 Ended: 2024-10-04 12:39:44.388543181 -0400 -0400 Result: SUCCESS

        Stage metrics:
            PVCAttachment:
                Avg: 3.821428774s
                Min: 869.002831ms
                Max: 11.315248474s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/ReplicationSuite3/PVCAttachment.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/ReplicationSuite3/PVCAttachment_boxplot.png
            PVCBind:
                Avg: 20.908383412s
                Min: 8.54482872s
                Max: 45.199747071s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/ReplicationSuite3/PVCBind.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/ReplicationSuite3/PVCBind_boxplot.png
            PVCCreation:
                Avg: 27.22932091s
                Min: 10.992115438s
                Max: 57.060543855s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/ReplicationSuite3/PVCCreation.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/ReplicationSuite3/PVCCreation_boxplot.png
            PVCDeletion:
                Avg: 348.00648ms
                Min: 111.5971ms
                Max: 611.637886ms
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/ReplicationSuite3/PVCDeletion.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/ReplicationSuite3/PVCDeletion_boxplot.png
            PVCUnattachment:
                Avg: 8.268226907s
                Min: 1.23367879s
                Max: 12.705474558s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/ReplicationSuite3/PVCUnattachment.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/ReplicationSuite3/PVCUnattachment_boxplot.png
            PodCreation:
                Avg: 26.968298821s
                Min: 20.057533562s
                Max: 34.20463142s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/ReplicationSuite3/PodCreation.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/ReplicationSuite3/PodCreation_boxplot.png
            PodDeletion:
                Avg: 2.508734668s
                Min: 1.846622152s
                Max: 3.690412628s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/ReplicationSuite3/PodDeletion.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/ReplicationSuite3/PodDeletion_boxplot.png
                EntityNumberOverTime:
/root/.cert-csi/reports/test-run-8b6f4843/ReplicationSuite3/EntityNumberOverTime.png

  4. TestCase: SnapSuite Started: 2024-10-04 12:34:37.689177129 -0400 -0400 Ended: 2024-10-04 12:37:08.435812543 -0400 -0400 Result: SUCCESS

        Stage metrics:
            PVCAttachment:
                Avg: 1.134678671s
                Min: 397.83468ms
                Max: 1.871522662s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/SnapSuite4/PVCAttachment.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/SnapSuite4/PVCAttachment_boxplot.png
            PVCBind:
                Avg: 13.949529247s
                Min: 12.917692156s
                Max: 14.981366338s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/SnapSuite4/PVCBind.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/SnapSuite4/PVCBind_boxplot.png
            PVCCreation:
                Avg: 15.599203984s
                Min: 13.669956379s
                Max: 17.528451589s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/SnapSuite4/PVCCreation.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/SnapSuite4/PVCCreation_boxplot.png
            PVCDeletion:
                Avg: 990.54237ms
                Min: 29.401729ms
                Max: 1.951683011s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/SnapSuite4/PVCDeletion.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/SnapSuite4/PVCDeletion_boxplot.png
            PVCUnattachment:
                Avg: 491.60832ms
                Min: 269.166654ms
                Max: 714.049987ms
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/SnapSuite4/PVCUnattachment.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/SnapSuite4/PVCUnattachment_boxplot.png
            PodCreation:
                Avg: 18.590780993s
                Min: 14.410337579s
                Max: 22.771224407s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/SnapSuite4/PodCreation.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/SnapSuite4/PodCreation_boxplot.png
            PodDeletion:
                Avg: 2.540745986s
                Min: 2.13095443s
                Max: 2.950537542s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/SnapSuite4/PodDeletion.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/SnapSuite4/PodDeletion_boxplot.png
                EntityNumberOverTime:
/root/.cert-csi/reports/test-run-8b6f4843/SnapSuite4/EntityNumberOverTime.png

  5. TestCase: VolumeExpansionSuite Started: 2024-10-04 12:34:37.68766529 -0400 -0400 Ended: 2024-10-04 12:36:20.206314669 -0400 -0400 Result: SUCCESS

        Stage metrics:
            PVCAttachment:
                Avg: 2.006547623s
                Min: 2.006547623s
                Max: 2.006547623s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/VolumeExpansionSuite5/PVCAttachment.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/VolumeExpansionSuite5/PVCAttachment_boxplot.png
            PVCBind:
                Avg: 14.983559134s
                Min: 14.983559134s
                Max: 14.983559134s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/VolumeExpansionSuite5/PVCBind.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/VolumeExpansionSuite5/PVCBind_boxplot.png
            PVCCreation:
                Avg: 17.076041862s
                Min: 17.076041862s
                Max: 17.076041862s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/VolumeExpansionSuite5/PVCCreation.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/VolumeExpansionSuite5/PVCCreation_boxplot.png
            PVCDeletion:
                Avg: 2.91110793s
                Min: 2.91110793s
                Max: 2.91110793s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/VolumeExpansionSuite5/PVCDeletion.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/VolumeExpansionSuite5/PVCDeletion_boxplot.png
            PVCUnattachment:
                Avg: 10.454575812s
                Min: 10.454575812s
                Max: 10.454575812s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/VolumeExpansionSuite5/PVCUnattachment.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/VolumeExpansionSuite5/PVCUnattachment_boxplot.png
            PodCreation:
                Avg: 22.082186658s
                Min: 22.082186658s
                Max: 22.082186658s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/VolumeExpansionSuite5/PodCreation.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/VolumeExpansionSuite5/PodCreation_boxplot.png
            PodDeletion:
                Avg: 1.653746363s
                Min: 1.653746363s
                Max: 1.653746363s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/VolumeExpansionSuite5/PodDeletion.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/VolumeExpansionSuite5/PodDeletion_boxplot.png
                EntityNumberOverTime:
/root/.cert-csi/reports/test-run-8b6f4843/VolumeExpansionSuite5/EntityNumberOverTime.png

  6. TestCase: MultiAttachSuite Started: 2024-10-04 12:34:37.689226039 -0400 -0400 Ended: 2024-10-04 12:37:11.603155635 -0400 -0400 Result: SUCCESS

        Stage metrics:
            PVCAttachment:
                Avg: 713.361887ms
                Min: 713.361887ms
                Max: 713.361887ms
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/MultiAttachSuite6/PVCAttachment.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/MultiAttachSuite6/PVCAttachment_boxplot.png
            PVCBind:
                Avg: 18.360088032s
                Min: 18.360088032s
                Max: 18.360088032s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/MultiAttachSuite6/PVCBind.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/MultiAttachSuite6/PVCBind_boxplot.png
            PVCCreation:
                Avg: 19.795622612s
                Min: 19.795622612s
                Max: 19.795622612s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/MultiAttachSuite6/PVCCreation.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/MultiAttachSuite6/PVCCreation_boxplot.png
            PVCDeletion:
                Avg: 2.19058085s
                Min: 2.19058085s
                Max: 2.19058085s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/MultiAttachSuite6/PVCDeletion.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/MultiAttachSuite6/PVCDeletion_boxplot.png
            PVCUnattachment:
                Avg: 369.453431ms
                Min: 369.453431ms
                Max: 369.453431ms
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/MultiAttachSuite6/PVCUnattachment.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/MultiAttachSuite6/PVCUnattachment_boxplot.png
            PodDeletion:
                Avg: 222.513404ms
                Min: 3.620669ms
                Max: 1.30029054s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/MultiAttachSuite6/PodDeletion.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/MultiAttachSuite6/PodDeletion_boxplot.png
                EntityNumberOverTime:
/root/.cert-csi/reports/test-run-8b6f4843/MultiAttachSuite6/EntityNumberOverTime.png

  7. TestCase: EphemeralVolumeSuite Started: 2024-10-04 12:34:37.689654242 -0400 -0400 Ended: 2024-10-04 12:36:00.316199943 -0400 -0400 Result: SUCCESS

        Stage metrics:
            PodCreation:
                Avg: 16.162724546s
                Min: 16.13541781s
                Max: 16.190031282s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/EphemeralVolumeSuite7/PodCreation.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/EphemeralVolumeSuite7/PodCreation_boxplot.png
            PodDeletion:
                Avg: 7.781542281s
                Min: 7.205850782s
                Max: 8.35723378s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/EphemeralVolumeSuite7/PodDeletion.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/EphemeralVolumeSuite7/PodDeletion_boxplot.png
                EntityNumberOverTime:
/root/.cert-csi/reports/test-run-8b6f4843/EphemeralVolumeSuite7/EntityNumberOverTime.png

  8. TestCase: CloneVolumeSuite Started: 2024-10-04 12:34:37.68940025 -0400 -0400 Ended: 2024-10-04 12:36:43.055901344 -0400 -0400 Result: SUCCESS

        Stage metrics:
            PVCAttachment:
                Avg: 3.542858601s
                Min: 869.246455ms
                Max: 10.567904135s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/CloneVolumeSuite8/PVCAttachment.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/CloneVolumeSuite8/PVCAttachment_boxplot.png
            PVCBind:
                Avg: 14.088944241s
                Min: 12.750291382s
                Max: 16.355616657s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/CloneVolumeSuite8/PVCBind.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/CloneVolumeSuite8/PVCBind_boxplot.png
            PVCCreation:
                Avg: 18.19032995s
                Min: 14.17852033s
                Max: 24.066669068s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/CloneVolumeSuite8/PVCCreation.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/CloneVolumeSuite8/PVCCreation_boxplot.png
            PVCDeletion:
                Avg: 1.950150932s
                Min: 1.317335307s
                Max: 2.181042371s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/CloneVolumeSuite8/PVCDeletion.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/CloneVolumeSuite8/PVCDeletion_boxplot.png
            PVCUnattachment:
                Avg: 389.661369ms
                Min: 194.559316ms
                Max: 544.463118ms
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/CloneVolumeSuite8/PVCUnattachment.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/CloneVolumeSuite8/PVCUnattachment_boxplot.png
            PodCreation:
                Avg: 17.936776271s
                Min: 13.308363729s
                Max: 21.994192319s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/CloneVolumeSuite8/PodCreation.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/CloneVolumeSuite8/PodCreation_boxplot.png
            PodDeletion:
                Avg: 2.213069309s
                Min: 1.595496595s
                Max: 2.470374283s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/CloneVolumeSuite8/PodDeletion.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/CloneVolumeSuite8/PodDeletion_boxplot.png
                EntityNumberOverTime:
/root/.cert-csi/reports/test-run-8b6f4843/CloneVolumeSuite8/EntityNumberOverTime.png

  9. TestCase: VolumeIoSuite Started: 2024-10-04 12:34:37.689369543 -0400 -0400 Ended: 2024-10-04 12:37:03.35215891 -0400 -0400 Result: SUCCESS

        Stage metrics:
            PVCBind:
                Avg: 21.372622089s
                Min: 20.432406702s
                Max: 22.312837476s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/VolumeIoSuite9/PVCBind.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/VolumeIoSuite9/PVCBind_boxplot.png
            PVCCreation:
                Avg: 22.757326597s
                Min: 21.056758652s
                Max: 24.457894542s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/VolumeIoSuite9/PVCCreation.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/VolumeIoSuite9/PVCCreation_boxplot.png
            PVCDeletion:
                Avg: 43.299976ms
                Min: 34.300494ms
                Max: 52.299459ms
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/VolumeIoSuite9/PVCDeletion.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/VolumeIoSuite9/PVCDeletion_boxplot.png
            PVCUnattachment:
                Avg: 36.684611471s
                Min: 32.17693862s
                Max: 41.192284322s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/VolumeIoSuite9/PVCUnattachment.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/VolumeIoSuite9/PVCUnattachment_boxplot.png
            PodCreation:
                Avg: 16.738193415s
                Min: 10.695745258s
                Max: 22.358716257s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/VolumeIoSuite9/PodCreation.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/VolumeIoSuite9/PodCreation_boxplot.png
            PodDeletion:
                Avg: 1.874529961s
                Min: 1.26175163s
                Max: 2.213538424s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/VolumeIoSuite9/PodDeletion.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/VolumeIoSuite9/PodDeletion_boxplot.png
                EntityNumberOverTime:
/root/.cert-csi/reports/test-run-8b6f4843/VolumeIoSuite9/EntityNumberOverTime.png

  10. TestCase: ScalingSuite Started: 2024-10-04 12:34:37.689386007 -0400 -0400 Ended: 2024-10-04 12:36:33.83682364 -0400 -0400 Result: SUCCESS

        Stage metrics:
            PVCAttachment:
                Avg: 1.594739351s
                Min: 1.143052326s
                Max: 2.197054939s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/ScalingSuite10/PVCAttachment.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/ScalingSuite10/PVCAttachment_boxplot.png
            PVCBind:
                Avg: 13.41748631s
                Min: 9.040850746s
                Max: 14.801462701s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/ScalingSuite10/PVCBind.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/ScalingSuite10/PVCBind_boxplot.png
            PVCCreation:
                Avg: 16.331865249s
                Min: 15.404106487s
                Max: 17.374790908s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/ScalingSuite10/PVCCreation.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/ScalingSuite10/PVCCreation_boxplot.png
            PVCDeletion:
                Avg: 153.322385ms
                Min: 52.168708ms
                Max: 280.077538ms
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/ScalingSuite10/PVCDeletion.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/ScalingSuite10/PVCDeletion_boxplot.png
            PVCUnattachment:
                Avg: 10.10410353s
                Min: 8.701869968s
                Max: 11.066056592s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/ScalingSuite10/PVCUnattachment.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/ScalingSuite10/PVCUnattachment_boxplot.png
            PodCreation:
                Avg: 28.124376651s
                Min: 25.78857683s
                Max: 30.460176473s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/ScalingSuite10/PodCreation.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/ScalingSuite10/PodCreation_boxplot.png
            PodDeletion:
                Avg: 2.336812261s
                Min: 1.925177571s
                Max: 2.748446951s
                Histogram:
/root/.cert-csi/reports/test-run-8b6f4843/ScalingSuite10/PodDeletion.png
                BoxPlot:
/root/.cert-csi/reports/test-run-8b6f4843/ScalingSuite10/PodDeletion_boxplot.png
                EntityNumberOverTime:
/root/.cert-csi/reports/test-run-8b6f4843/ScalingSuite10/EntityNumberOverTime.png

[2024-10-04 12:39:59] ERROR Can't generate reports; error=test runs: [test-run-24573041] not found