Closed saurabhwani5 closed 10 months ago
Even though we have the fix for one part for power8, CSI driver further fails to mount or unmount the volumes as the library used by CSI driver makes the os calls which fail again with power8 + UBI9 combination.
Closing this as UBI9+power8 causes other issues and we have dropped support of power8 from CSI 2.10 onward.
Describe the bug
When tried to create PVC and pods on Power 8 env , pod mount fails with error and kubectl exec doesn't work for driver pod:
How to Reproduce?
apiVersion: v1 kind: PersistentVolumeClaim metadata: name: scale-advance-pvc-1 spec: accessModes:
apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: ibm-spectrum-scale-csi-advance provisioner: spectrumscale.csi.ibm.com parameters: volBackendFs: "fs1" version: "2" reclaimPolicy: Delete
[root@power8-master ~]# oc get pvc -w NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE scale-advance-pvc-1 Pending ibm-spectrum-scale-csi-advance 5s scale-advance-pvc-1 Pending pvc-f2a417f6-34e0-417d-bf86-c967370a35ff 0 ibm-spectrum-scale-csi-advance 56s scale-advance-pvc-1 Bound pvc-f2a417f6-34e0-417d-bf86-c967370a35ff 1Gi RWX ibm-spectrum-scale-csi-advance 56s [root@power8-master ~]# oc get pods NAME READY STATUS RESTARTS AGE csi-scale-fsetdemo-pod-1 0/1 ContainerCreating 0 101s ibm-spectrum-scale-csi-attacher-759c9d7bd4-86mrd 1/1 Running 0 4m56s ibm-spectrum-scale-csi-attacher-759c9d7bd4-rctd6 1/1 Running 0 4m56s ibm-spectrum-scale-csi-m4v9t 3/3 Running 0 4m56s ibm-spectrum-scale-csi-operator-764cc4489f-md2r8 1/1 Running 0 5m41s ibm-spectrum-scale-csi-provisioner-58b4d75cfd-sbg9h 1/1 Running 0 4m56s ibm-spectrum-scale-csi-resizer-6577bb5ddd-zkzxf 1/1 Running 0 4m56s ibm-spectrum-scale-csi-snapshotter-85b4fcf68c-fzzc7 1/1 Running 0 4m56s ibm-spectrum-scale-csi-sqjcg 3/3 Running 0 4m56s
[root@power8-master ~]# oc describe pod csi-scale-fsetdemo-pod-1 Name: csi-scale-fsetdemo-pod-1 Namespace: ibm-spectrum-scale-csi-driver Priority: 0 Service Account: default Node: power8-worker-1./10.11.23.199 Start Time: Tue, 21 Nov 2023 23:15:32 -0800 Labels: app=nginx Annotations:
Status: Pending
IP:
IPs:
Containers:
web-server:
Container ID:
Image: nginx
Image ID:
Port: 80/TCP
Host Port: 0/TCP
State: Waiting
Reason: ContainerCreating
Ready: False
Restart Count: 0
Environment:
Mounts:
/usr/share/nginx/html/scale from mypvc (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-vhpg2 (ro)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
mypvc:
Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
ClaimName: scale-advance-pvc-1
ReadOnly: false
kube-api-access-vhpg2:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: 3607
ConfigMapName: kube-root-ca.crt
ConfigMapOptional:
DownwardAPI: true
QoS Class: BestEffort
Node-Selectors:
Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message
Warning FailedScheduling 2m46s default-scheduler 0/3 nodes are available: persistentvolumeclaim "scale-advance-pvc-1" not found. preemption: 0/3 nodes are available: 3 Preemption is not helpful for scheduling.. Warning FailedScheduling 2m44s default-scheduler 0/3 nodes are available: pod has unbound immediate PersistentVolumeClaims. preemption: 0/3 nodes are available: 3 Preemption is not helpful for scheduling.. Warning FailedScheduling 109s default-scheduler 0/3 nodes are available: pod has unbound immediate PersistentVolumeClaims. preemption: 0/3 nodes are available: 3 Preemption is not helpful for scheduling.. Normal Scheduled 105s default-scheduler Successfully assigned ibm-spectrum-scale-csi-driver/csi-scale-fsetdemo-pod-1 to power8-worker-1. Normal SuccessfulAttachVolume 96s attachdetach-controller AttachVolume.Attach succeeded for volume "pvc-f2a417f6-34e0-417d-bf86-c967370a35ff" Warning FailedMount 31s (x8 over 95s) kubelet MountVolume.SetUp failed for volume "pvc-f2a417f6-34e0-417d-bf86-c967370a35ff" : rpc error: code = Unknown desc = checkGpfsType: the path [/ibm/fs1/6fdb5da2-1a50-44b0-bfc9-225beaf8f605-ibm-spectrum-scale-csi-driver/pvc-f2a417f6-34e0-417d-bf86-c967370a35ff] is not a valid gpfs path
[root@power8-master ~]# oc exec -it ibm-spectrum-scale-csi-sqjcg -- bash Fatal glibc error: CPU lacks ISA 3.00 support (POWER9 or later required) command terminated with exit code 127 [root@power8-master ~]# oc exec -it ibm-spectrum-scale-csi-sqjcg -- sh Fatal glibc error: CPU lacks ISA 3.00 support (POWER9 or later required) command terminated with exit code 127