Expander.NodeExpand failed to expand the volume

[Rico556]

Hi, I tried resizing pv and got stuck in the file system resizing of the volume on node, seems that something is missing on my worker node.

Here is the error log I found:

• POD:

  MountVolume.MountDevice failed while expanding volume for volume "$volume-id" : Expander.NodeExpand failed to expand the volume : rpc error: code = Internal desc = Failed to find mount file system /var/lib/kubelet/plugins/kubernetes.io/csi/pv/$volume-id/globalmount: executable file not found in $PATH

• PVC:

  Normal   FileSystemResizeRequired  50m (x2 over 158m)  external-resizer cinder.csi.openstack.org  Require file system resize of volume on node

• Cinder plugin container:

  I0301 06:31:59.236703       1 nodeserver.go:541] NodeExpandVolume: called with args {"capacity_range":{"required_bytes":51539607552},"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/pv/$volume-id/globalmount","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":1}},"volume_id":"6ca713bc-7b3f-4485-8810-3deaacafbbd8","volume_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/pv/$volume-id/globalmount"}
  E0301 06:31:59.308265       1 utils.go:92] [ID:902218] GRPC error: rpc error: code = Internal desc = Failed to find mount file system /var/lib/kubelet/plugins/kubernetes.io/csi/pv/$volume-id: executable file not found in $PATH

:~> kubectl version Client Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.6", GitCommit:"51351e76484eb80accdd7702d9bd03652ad830b3", GitTreeState:"clean", BuildDate:"2022-05-20T08:49:23Z", GoVersion:"go1.17.9", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.6", GitCommit:"51351e76484eb80accdd7702d9bd03652ad830b3", GitTreeState:"clean", BuildDate:"2022-05-20T08:46:07Z", GoVersion:"go1.17.9", Compiler:"gc", Platform:"linux/amd64"}