Support for multipath storage and ISCSI LUNs

dnoland1 commented 5 years ago

Request Type Feature Request

Description Support for multipath storage and ISCSI. The use case is having the ability to deploy Pure Storage. Prefer to have this capability in the default Rancher OS console, but having to switch to the ubuntu console is acceptable. Would want to use in conjunction with vSphere node driver and horizontal cluster autoscaling using the Rancher OS ISO.

Jason-ZW commented 5 years ago

Graph:

+----------------+                                         +----------------------------+
| [iSCSI Target] |                                         |  [ROS multipathd service]  |
|    server01    +--------------------+--------------------+    [ROS iSCSI Initiator]   |
|  172.31.41.48  |                                         |          x.x.x.x           |
+----------------+                                         +----------------------------+

Description:

Currently, RancherOS support for multipathd still requires some manual work. The following is just a simple example of how to use multipathd in ROS. User can customize it by going into the multipathd container and changing the configuration /etc/multipath.conf & /etc/multipath/wwids and restarting the container.

Environment:

AWS ROS v1.5.4 (using default console) Iscsi Multipathd Missing-udev-rules

ubuntu (iscsi server):

$ apt-get update && apt-get install -y tgt
$ mkdir /var/lib/iscsi_disks
$ dd if=/dev/zero of=/var/lib/iscsi_disks/disk01.img count=0 bs=1 seek=10G
$ vi /etc/tgt/conf.d/target01.conf
<target iqn.2019-10.com.think-show.iscsi:server.target01>
    backing-store /var/lib/iscsi_disks/disk01.img
</target>
$ systemctl restart tgt
$ tgtadm --mode target --op show
Target 1: iqn.2019-10.com.think-show.iscsi:server.target01
    System information:
        Driver: iscsi
        State: ready
    I_T nexus information:
    LUN information:
        LUN: 0
            Type: controller
            SCSI ID: IET     00010000
            SCSI SN: beaf10
            Size: 0 MB, Block size: 1
            Online: Yes
            Removable media: No
            Prevent removal: No
            Readonly: No
            SWP: No
            Thin-provisioning: No
            Backing store type: null
            Backing store path: None
            Backing store flags:
        LUN: 1
            Type: disk
            SCSI ID: IET     00010001
            SCSI SN: beaf11
            Size: 10737 MB, Block size: 512
            Online: Yes
            Removable media: No
            Prevent removal: No
            Readonly: No
            SWP: No
            Thin-provisioning: No
            Backing store type: rdwr
            Backing store path: /var/lib/iscsi_disks/disk01.img
            Backing store flags:
    Account information:
    ACL information:
        ALL

ros (iscsi initiator & multipathd service)

$ ros service enable open-iscsi
$ ros service up open-iscsi
$ iscsiadm -m discovery -t sendtargets -p 172.31.41.48
$ iscsiadm -m node --login
Logging in to [iface: default, target: iqn.2019-10.com.think-show.iscsi:server.target01, portal: 172.31.40.104,3260] (multiple)
Login to [iface: default, target: iqn.2019-10.com.think-show.iscsi:server.target01, portal: 172.31.40.104,3260] successful.
$ iscsiadm -m session -o show
tcp: [1] 172.31.40.104:3260,1 iqn.2019-10.com.think-show.iscsi:server.target01
$ ros service enable kernel-extras
$ ros service up kernel-extras
$ modprobe multipath
$ modprobe dm-multipath
$ modprobe dm-round-robin
$ ros c set rancher.repositories.zhaozy.url https://raw.githubusercontent.com/Jason-ZW/os-services/add-multipathd
$ system-docker exec -it udev bash
  $ wget -O /lib/udev/rules.d/55-dm.rules https://raw.githubusercontent.com/Jason-ZW/missing-udev-rules/master/55-dm.rules
  $ exit
$ ros service enable multipathd
$ ros service up multipathd
$ system-docker exec -it multipathd sh
  $ multipath -F
  $ multipath -v3
  $ multipath -ll
  RMD (360000000000000000e00000000010001) dm-0 IET,VIRTUAL-DISK
  size=10G features='0' hwhandler='0' wp=rw
  `-+- policy='service-time 0' prio=1 status=enabled
    `- 2:0:0:1 sda 8:0 active ready running
  $ ros udev-settle
  $ ls /dev/mapper
  control  rmd

Rancheroo commented 4 years ago

is the multipathd required for the ros (iscsi initiator & multipathd service) approach?

I managed to discovery the LUNS as /dev/sdb and /dev/sdc and format them however they would not mount.
What is the recommended method to make these iscsi LUNs persist after a reboot?

Thank you

Rancheroo commented 4 years ago

I just found that the mounts should be setup in cloudconfig https://rancher.com/docs/os/v1.x/en/installation/storage/additional-mounts/

:+1:

rancher / os