multipath

[gionag]

What is the current state of it ? it is supposed to work ? I have mainly used the reccomendations here https://github.com/TheGrandWazoo/freenas-proxmox/issues/39

at the moment, by adding find_multipaths yes to multipath.conf seems to populate correctly the bindings and wwid file, but only after the first start of the machine. if the machine has never run, the output of multipath -ll doesn't show the mapping.

Furthermore, upon deletion of the vm, the lun is delete on the truenas side, but the output of multipath -ll still shows the mapping as "faulty". this is what is shown after i deleted two vm's

root@test:/etc/multipath# multipath -ll
mpatha (36589cfc000000f837ab189f04d9f0584) dm-0 TrueNAS,iSCSI Disk
size=4.0G features='1 queue_if_no_path' hwhandler='1 alua' wp=rw
`-+- policy='round-robin 0' prio=0 status=enabled
  |- 3:0:0:0 sdd 8:48 failed faulty running
  `- 4:0:0:0 sdc 8:32 failed faulty running
mpathb (36589cfc000000de308d40b7e80c7ae1e) dm-1 TrueNAS,iSCSI Disk
size=7.0G features='1 queue_if_no_path' hwhandler='1 alua' wp=rw
`-+- policy='round-robin 0' prio=0 status=enabled
  |- 3:0:0:1 sde 8:64 failed faulty running
  `- 4:0:0:1 sdf 8:80 failed faulty running

that's my /etc/pve/storage.cfg

``` zfs: sany blocksize 8k iscsiprovider freenas pool tank/vmdisks portal 10.100.100.1 target iqn.2005-10.org.freenas.ctl:pve content images freenas_apiv4_host 10.100.100.1 freenas_password password freenas_use_ssl 0 freenas_user root nowritecache 0 sparse 0 ```

that's my /etc/iscsi/iscsid.conf

``` iscsid.startup = /bin/systemctl start iscsid.socket node.startup = automatic node.leading_login = No node.session.timeo.replacement_timeout = 15 node.conn[0].timeo.login_timeout = 15 node.conn[0].timeo.logout_timeout = 15 node.conn[0].timeo.noop_out_interval = 5 node.conn[0].timeo.noop_out_timeout = 5 node.session.err_timeo.abort_timeout = 15 node.session.err_timeo.lu_reset_timeout = 30 node.session.err_timeo.tgt_reset_timeout = 30 node.session.err_timeo.host_reset_timeout = 60 node.session.initial_login_retry_max = 8 node.session.cmds_max = 128 node.session.queue_depth = 32 node.session.xmit_thread_priority = -20 node.session.iscsi.InitialR2T = No node.session.iscsi.ImmediateData = Yes node.session.iscsi.FirstBurstLength = 262144 node.session.iscsi.MaxBurstLength = 16776192 node.conn[0].iscsi.MaxRecvDataSegmentLength = 262144 node.conn[0].iscsi.MaxXmitDataSegmentLength = 0 discovery.sendtargets.iscsi.MaxRecvDataSegmentLength = 32768 node.session.nr_sessions = 1 node.session.reopen_max = 0 node.session.iscsi.FastAbort = Yes node.session.scan = auto ```

that's my /etc/multipath.conf

``` defaults { polling_interval 2 path_selector "round-robin 0" path_grouping_policy multibus uid_attribute ID_SERIAL rr_min_io 100 failback immediate no_path_retry queue user_friendly_names yes find_multipaths yes } blacklist { devnode "sd[a-b]$" } ```

can someone, maybe @TheGrandWazoo settle once and for all what we have to do in order to be able to use multipath with this plugin ?

thanks.

[TheGrandWazoo]

I have been trying myself for the last several months since I reconfigured my network to try multipath but I thought it might be my flat network configuration. Maybe it needs to be revisited? Also, I am assuming you tried the instructions https://pve.proxmox.com/wiki/ISCSI_Multipath?

[TheGrandWazoo]

And this was submitted by a user of the plugin so I have not attempted to use it until the past few months.

[gionag]

I have been trying myself for the last several months since I reconfigured my network to try multipath but I thought it might be my flat network configuration. Maybe it needs to be revisited? Also, I am assuming you tried the instructions https://pve.proxmox.com/wiki/ISCSI_Multipath?

yes... i have followed the instructions there... not the part where i manually add the wwids... as this is not our case. the system should "detect" automatically that. find_multipaths yes option seems to do that.

i have seen down further on the proxmox iscsi multipath docs that they added udev rules to handle a device change and do a rescan / deletion

# /etc/udev/rules.d/99-reduxio.rules
 
 SUBSYSTEM=="block" , ACTION=="change", ATTRS{model}=="TCAS", ATTRS{vendor}=="REDUXIO", RUN+="/bin/sh -c '/usr/sbin/iscsiadm -m session -R '"
SUBSYSTEM=="block" , ACTION=="change", ATTRS{model}=="TCAS", ATTRS{vendor}=="REDUXIO", ATTR{size}=="0", RUN+="/bin/sh -c 'echo 1 > /sys$DEVPATH/../../delete '"
SUBSYSTEM=="block" , ACTION=="change", ATTRS{model}=="TCAS", ATTRS{vendor}=="REDUXIO", RUN+="/bin/sh -c 'service multipathd reload || service multipath-tools reload ' "
SUBSYSTEM=="block" , ACTION=="change", ATTRS{model}=="TCAS", ATTRS{vendor}=="REDUXIO", RUN+="/bin/sh -c '/usr/sbin/multipath -r $DEVNAME '"

i can try that....

one question : where can i see if a VM is using the multipath mapper path ( dev/mapper/mpathX ), instead of the single disk ? in the vm configuration file i can only see reference of the storage provider... not the underlaying physical resource. I want to be sure that the mod on your plugin properly succed in assigned the correct multipath device for the vm

thanks for all the effortss

[gionag]

have you ever succeded in multipathing ?

[TheGrandWazoo]

Multipath is not part of the TrueNAS plugin. It was added to the Proxmox ZFSPlugin.pm via a patch file from a contributor to this project. Looking at the initial commits it was developed for ProxmoxVE 5. Looking at the builtin ISCSIPlugin.pm it seems there were some changes to it due to Kernel changes over the years. I am not sure if these or a portion of these changes can be added to ZFSPlugin.pm at the moment.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[boomshankerx]

Has there been any more progress made here? I just moved over from VMWare to proxmox and am looking for a working multipath iSCSI configuration.