[manila-csi-plugin] csi.sock: connect: no such file or directory

[emreberber]

/kind bug

What happened:

When deploying the openstack-manila-csi Helm package, the controller and node-plugin pods stuck in an Error state.

~ ❯ kubectl logs manila-csi-openstack-manila-csi-controllerplugin-0 -c cephfs-nodeplugin                                                                                   
I1106 17:12:02.818479       1 driver.go:127] Driver: cephfs.manila.csi.openstack.org
I1106 17:12:02.818546       1 driver.go:128] Driver version: 0.9.0@v1.31.0
I1106 17:12:02.818552       1 driver.go:129] CSI spec version: 1.8.0
I1106 17:12:02.818558       1 driver.go:132] Operating on CEPHFS shares
I1106 17:12:02.818564       1 driver.go:137] Topology awareness disabled
I1106 17:12:02.818571       1 driver.go:159] Providing controller service
I1106 17:12:02.818577       1 driver.go:221] Enabling controller service capability: CREATE_DELETE_VOLUME
I1106 17:12:02.818582       1 driver.go:221] Enabling controller service capability: CREATE_DELETE_SNAPSHOT
I1106 17:12:02.818585       1 driver.go:221] Enabling controller service capability: EXPAND_VOLUME
I1106 17:12:02.818590       1 driver.go:240] Enabling volume access mode: MULTI_NODE_MULTI_WRITER
I1106 17:12:02.818595       1 driver.go:240] Enabling volume access mode: MULTI_NODE_SINGLE_WRITER
I1106 17:12:02.818599       1 driver.go:240] Enabling volume access mode: MULTI_NODE_READER_ONLY
I1106 17:12:02.818603       1 driver.go:240] Enabling volume access mode: SINGLE_NODE_WRITER
I1106 17:12:02.818607       1 driver.go:240] Enabling volume access mode: SINGLE_NODE_READER_ONLY
I1106 17:12:02.818612       1 driver.go:184] Providing node service
I1106 17:12:02.818789       1 common.go:111] Probing CSI driver for readiness
I1106 17:12:02.819558       1 builder.go:59] [ID:1] FWD GRPC error: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial unix /var/lib/kubelet/plugins/cephfs.csi.ceph.com/csi.sock: connect: no such file or directory"
F1106 17:12:02.819644       1 main.go:111] Driver node service initialization failed: failed to initialize proxied CSI driver: probe failed: CSI driver probe failed: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial unix /var/lib/kubelet/plugins/cephfs.csi.ceph.com/csi.sock: connect: no such file or directory"

How to reproduce it:

helm upgrade  --install manila-csi cpo/openstack-manila-csi

Anything else we need to know?:

When I try with version 1.30.0, there are no issues. It deploys successfully.

Environment: Kubernetes ❯ 1.26.6 manila-csi-plugin ❯ v1.31.0

I can't find any Kubernetes compatibility matrix. If there is one available, could you please share it? Has anyone encountered and resolved this issue?

[kayrus]

@emreberber are you going to use manila CSI plugin with the Cephfs driver? Then you need to run the https://github.com/ceph/ceph-csi and point it to the --fwdendpoint=$(FWD_CSI_ENDPOINT).

basically manila CSI driver reads requests from the --endpoint=$(CSI_ENDPOINT), modifies these requests, and forwards them to NFS CSI plugin or Cephfs CSI plugin.

[emreberber]

Thanks for your comment

I found out later. It was necessary to deploy Ceph CSI along with Manila CSI. I could not see if this requirement is written in the document.