[iscsi-tools] Please support multipath

[Twanislas]

Hi team,

It looks like multipath-tools is not available in the iscsi-tools extension. I tried to search for multipath references everywhere, to no avail.

When mounting iscsi target trying to use multipath (using democratic-csi), I get failed to discover multipath device. However, I can see that the two disk references are successfully mounted :

{"host":"kubeworker03","level":"info","message":"new request - driver: ControllerSynologyDriver method: NodeStageVolume call: {\"metadata\":{\"x-forwarded-host\":[\"localhost\"],\"user-agent\":[\"grpc-go/1.49.0\"]},\"request\":{\"publish_context\":{},\"secrets\":\"redacted\",\"volume_context\":{\"lun\":\"1\",\"node_attach_driver\":\"iscsi\",\"portal\":\"\",\"portals\":\"nas01-iscsi0.i,nas01-iscsi1.i\",\"provisioner_driver\":\"synology-iscsi\",\"storage.kubernetes.io/csiProvisionerIdentity\":\"1679431778041-8081-synology-iscsi\",\"interface\":\"\",\"iqn\":\"iqn.2000-01.com.synology:csi.k8s-pvc-83c2d084-36e9-4b31-86f2-bca6df03770d\"},\"volume_id\":\"pvc-83c2d084-36e9-4b31-86f2-bca6df03770d\",\"staging_target_path\":\"/var/lib/kubelet/plugins/kubernetes.io/csi/synology-iscsi/4db6d710ff1e3fb3e3cd58f0a4fb57b0c962a6dd773ba289f292dbf2b48d2c61/globalmount\",\"volume_capability\":{\"access_mode\":{\"mode\":\"SINGLE_NODE_MULTI_WRITER\"},\"mount\":{\"mount_flags\":[],\"fs_type\":\"\",\"volume_mount_group\":\"\"},\"access_type\":\"mount\"}},\"cancelled\":false}","service":"democratic-csi","timestamp":"2023-03-21T21:31:11.735Z"}
executing iscsi command: iscsiadm -m node -T iqn.2000-01.com.synology:csi.k8s-pvc-83c2d084-36e9-4b31-86f2-bca6df03770d -p nas01-iscsi0.i:3260 -o new
executing iscsi command: iscsiadm -m node -T iqn.2000-01.com.synology:csi.k8s-pvc-83c2d084-36e9-4b31-86f2-bca6df03770d -p nas01-iscsi0.i:3260 -o update --name node.startup --value manual
executing iscsi command: iscsiadm -m node -T iqn.2000-01.com.synology:csi.k8s-pvc-83c2d084-36e9-4b31-86f2-bca6df03770d -p nas01-iscsi0.i:3260 -l
executing iscsi command: iscsiadm -m session
executing iscsi command: iscsiadm -m session -r 5 --rescan
executing filesystem command: realpath /dev/disk/by-path/ip-nas01-iscsi0.i:3260-iscsi-iqn.2000-01.com.synology:csi.k8s-pvc-83c2d084-36e9-4b31-86f2-bca6df03770d-lun-1
{"host":"kubeworker03","level":"info","message":"successfully logged into portal nas01-iscsi0.i:3260 and created device /dev/disk/by-path/ip-nas01-iscsi0.i:3260-iscsi-iqn.2000-01.com.synology:csi.k8s-pvc-83c2d084-36e9-4b31-86f2-bca6df03770d-lun-1 with realpath /dev/sdf","service":"democratic-csi","timestamp":"2023-03-21T21:31:11.948Z"}
executing iscsi command: iscsiadm -m node -T iqn.2000-01.com.synology:csi.k8s-pvc-83c2d084-36e9-4b31-86f2-bca6df03770d -p nas01-iscsi1.i:3260 -o new
executing iscsi command: iscsiadm -m node -T iqn.2000-01.com.synology:csi.k8s-pvc-83c2d084-36e9-4b31-86f2-bca6df03770d -p nas01-iscsi1.i:3260 -o update --name node.startup --value manual
executing iscsi command: iscsiadm -m node -T iqn.2000-01.com.synology:csi.k8s-pvc-83c2d084-36e9-4b31-86f2-bca6df03770d -p nas01-iscsi1.i:3260 -l
executing iscsi command: iscsiadm -m session
executing iscsi command: iscsiadm -m session -r 6 --rescan
executing filesystem command: realpath /dev/disk/by-path/ip-nas01-iscsi1.i:3260-iscsi-iqn.2000-01.com.synology:csi.k8s-pvc-83c2d084-36e9-4b31-86f2-bca6df03770d-lun-1
{"host":"kubeworker03","level":"info","message":"successfully logged into portal nas01-iscsi1.i:3260 and created device /dev/disk/by-path/ip-nas01-iscsi1.i:3260-iscsi-iqn.2000-01.com.synology:csi.k8s-pvc-83c2d084-36e9-4b31-86f2-bca6df03770d-lun-1 with realpath /dev/sdg","service":"democratic-csi","timestamp":"2023-03-21T21:31:12.196Z"}
executing filesystem command: sh -c for file in $(ls -la /dev/mapper/* | grep "\->" | grep -oP "\-> .+" | grep -oP " .+"); do echo $(F=$(echo $file | grep -oP "[a-z0-9-]+");echo $F":"$(ls "/sys/block/${F}/slaves/");); done;
executing filesystem command: sh -c for file in $(ls -la /dev/mapper/* | grep "\->" | grep -oP "\-> .+" | grep -oP " .+"); do echo $(F=$(echo $file | grep -oP "[a-z0-9-]+");echo $F":"$(ls "/sys/block/${F}/slaves/");); done;
{"host":"kubeworker03","level":"error","message":"handler error - driver: ControllerSynologyDriver method: NodeStageVolume error: {\"name\":\"GrpcError\",\"code\":2,\"message\":\"failed to discover multipath device\"}","service":"democratic-csi","timestamp":"2023-03-21T21:31:14.221Z"}

On the democratic-csi side, nsenter support for the multipath executable needs to be added as well (see the original PR for supporting iscsiadm here : https://github.com/democratic-csi/democratic-csi/pull/225)

Thanks for taking this request into consideration :)

Cheers !

[frezbo]

it seems it depends on shell being present on host?

executing filesystem command: sh -c for file in $(ls -la /dev/mapper/* | grep "\->" | grep -oP "\-> .+" | grep -oP " .+"); do echo $(F=$(echo $file | grep -oP "[a-z0-9-]+");echo $F":"$(ls "/sys/block/${F}/slaves/");); done;
executing filesystem command: sh -c for file in $(ls -la /dev/mapper/* | grep "\->" | grep -oP "\-> .+" | grep -oP " .+"); do echo $(F=$(echo $file | grep -oP "[a-z0-9-]+");echo $F":"$(ls "/sys/block/${F}/slaves/");); done;

unless these commands are executed in the csi container

[evilhamsterman]

I'd like to bump this, many external storage arrays require multipath to manage the paths for proper redundancy. The HPE csi-driver for instance has a hard requirement on multipath being available.

[gsalisbury]

This is also a needed for the NetApp trident ontap-san backend as it requires multipathd.