search

hpe-storage / csi-driver

A Container Storage Interface (CSI) driver from HPE
https://scod.hpedev.io
Apache License 2.0
57 stars 53 forks source link

PVC becoming Readonly #360

Open kulkarma opened 9 months ago

kulkarma commented 9 months ago

Environment: RHOCP 4.10.37 Compute Platform: RH OSP 16.4 Backend Storage: Nimble AFA60 OR also on Cinder REDHAT CEPH. Problem on both Backends HPE CSI Driver backendType: nimble-csi and also on Built-in Kubernetes Cinder volume. Problem Description: During deployment of HPE Applicatio randomly we get into the state of the PVC becoming readonly failing to deploy application. Below is the error.

kubelet Unable to attach or mount volumes: unmounted volumes. kubelet MountVolume.MountDevice failed for volume kubelet Unable to attach or mount volumes: unmounted volumes=[hpe-udm-cmod-encrypt-data-volume], unattached volumes=[kube-api-access-w7l7b hpe-udm-cmod-encrypt-config-volume hpe-udm-cmod-encrypt-secrets-volume hpe-udm-cmod-encrypt-data-volume]: timed out waiting for the condition

kubelet MountVolume.MountDevice failed for volume "pvc-071a74a7-bc92-4e26-b2e4-be99fe768d78" : cannot mount unformatted disk /dev/disk/by-id/virtio-46340d4a-81ca-4ea3-9 as we are manipulating it in read-only mode

We did not find any solutions or articles from RH. Hence wanted to know if there is anything we have to configure. During running of the application there is no problem observed and it this is only observed during deployment.

Regards Madhu

datamattsson commented 9 months ago

In general, when volumes become read-only is because the device has been missing for a period of time and you may have issues with your storage networks.

I don't think I've seen the driver interact with "virtio" devices before though, could you describe the environment? Protocols, virtual/physical, StorageClass and PVCs etc?