search

rackerlabs / genestack

Where Flex cloud brings infrastructures to where you are.
https://docs.rackspacecloud.com/
Apache License 2.0
30 stars 33 forks source link

Using LVM iSCSI (TGT) and Multipathd causes identical serial numbers to be created #451

Closed timolow closed 1 month ago

timolow commented 1 month ago

When using multiple LVM iSCSI nodes with TGT and creating identically sized volumes being mounted to the same compute node causes incorrect grouping of paths by multipath, leading to attachment issues. This may be due to identical serial numbers being created.

single 10gb volume being attached to instance

360000000000000000e00000000010001 dm-0 IET,VIRTUAL-DISK
size=10G features='0' hwhandler='0' wp=rw
`-+- policy='queue-length 0' prio=1 status=enabled
  |- 2:0:0:1 sda 8:0  active ready running
  `- 3:0:0:1 sdb 8:16 active ready running

two 10gb volumes being attached to instance from two different lvm nodes

360000000000000000e00000000010001 dm-0 IET,VIRTUAL-DISK
size=10G features='0' hwhandler='0' wp=rw
`-+- policy='queue-length 0' prio=1 status=enabled
  |- 2:0:0:1 sda 8:0  active ready running
  |- 3:0:0:1 sdb 8:16 active ready running
  |- 4:0:0:1 sdc 8:32 active ready running
  `- 5:0:0:1 sdd 8:48 active ready running

TGT view

root@genestack-storage1:~# tgtadm --mode target --op show
Target 1: iqn.2010-10.org.openstack:a19767d8-31bc-4396-a62f-3b35fd57185f
    System information:
        Driver: iscsi
        State: ready
    I_T nexus information:
        I_T nexus: 1
            Initiator: iqn.2004-10.com.ubuntu:01:ccf450ecfaa2 alias: genestack-compute1.lab.underworld.local
            Connection: 0
                IP Address: 10.1.2.212
        I_T nexus: 2
            Initiator: iqn.2004-10.com.ubuntu:01:ccf450ecfaa2 alias: genestack-compute1.lab.underworld.local
            Connection: 0
                IP Address: 10.1.1.212
    LUN information:
        LUN: 0
            Type: controller
            SCSI ID: IET     00010000
            SCSI SN: beaf10
            Size: 0 MB, Block size: 1
            Online: Yes
            Removable media: No
            Prevent removal: No
            Readonly: No
            SWP: No
            Thin-provisioning: No
            Backing store type: null
            Backing store path: None
            Backing store flags: 
        LUN: 1
            Type: disk
            SCSI ID: IET     00010001
            SCSI SN: beaf11
            Size: 10737 MB, Block size: 512
            Online: Yes
            Removable media: No
            Prevent removal: No
            Readonly: No
            SWP: No
            Thin-provisioning: No
            Backing store type: rdwr
            Backing store path: /dev/cinder-volumes-1/a19767d8-31bc-4396-a62f-3b35fd57185f
            Backing store flags: 
    Account information:
        Xzyajwk9N8Ej4dxxLoVa
    ACL information:
        ALL
root@genestack-storage2:~# tgtadm --mode target --op show
Target 1: iqn.2010-10.org.openstack:d2540e1d-7e86-4462-aa80-4e9c6a7cc382
    System information:
        Driver: iscsi
        State: ready
    I_T nexus information:
        I_T nexus: 1
            Initiator: iqn.2004-10.com.ubuntu:01:ccf450ecfaa2 alias: genestack-compute1.lab.underworld.local
            Connection: 0
                IP Address: 10.1.1.212
        I_T nexus: 2
            Initiator: iqn.2004-10.com.ubuntu:01:ccf450ecfaa2 alias: genestack-compute1.lab.underworld.local
            Connection: 0
                IP Address: 10.1.2.212
    LUN information:
        LUN: 0
            Type: controller
            SCSI ID: IET     00010000
            SCSI SN: beaf10
            Size: 0 MB, Block size: 1
            Online: Yes
            Removable media: No
            Prevent removal: No
            Readonly: No
            SWP: No
            Thin-provisioning: No
            Backing store type: null
            Backing store path: None
            Backing store flags: 
        LUN: 1
            Type: disk
            SCSI ID: IET     00010001
            SCSI SN: beaf11
            Size: 10737 MB, Block size: 512
            Online: Yes
            Removable media: No
            Prevent removal: No
            Readonly: No
            SWP: No
            Thin-provisioning: No
            Backing store type: rdwr
            Backing store path: /dev/cinder-volumes-1/d2540e1d-7e86-4462-aa80-4e9c6a7cc382
            Backing store flags: 
    Account information:
        jM3E5C8recy5GrnPZ5Vf
    ACL information:
        ALL

when creating different size volumes multipath does not map correctly:

root@genestack-compute1:~# multipath -ll
360000000000000000e00000000010001 dm-0 IET,VIRTUAL-DISK
size=10G features='0' hwhandler='0' wp=rw
`-+- policy='queue-length 0' prio=1 status=active
  |- 2:0:0:1 sda 8:0  active ready running
  |- 3:0:0:1 sdb 8:16 active ready running
  |- 4:0:0:1 sdc 8:32 active ready running
  `- 5:0:0:1 sdd 8:48 active ready running
360000000000000000e00000000020001 dm-1 IET,VIRTUAL-DISK
size=10G features='0' hwhandler='0' wp=rw
`-+- policy='queue-length 0' prio=1 status=active
  |- 7:0:0:1 sdf 8:80 active ready running
  `- 6:0:0:1 sde 8:64 active ready running
root@genestack-compute1:~# multipath -v3 -d
14118.330354 | set open fds limit to 1048576/1048576
14118.330419 | loading //lib/multipath/libchecktur.so checker
14118.330603 | checker tur: message table size = 3
14118.330641 | loading //lib/multipath/libprioconst.so prioritizer
14118.330834 | _init_foreign: foreign library "nvme" is not enabled
14118.337105 | sda: size = 20971520
14118.337264 | sda: vendor = IET
14118.337301 | sda: product = VIRTUAL-DISK
14118.337334 | sda: rev = 0001
14118.337956 | sda: h:b:t:l = 2:0:0:1
14118.338289 | sda: tgt_node_name = iqn.2010-10.org.openstack:a19767d8-31bc-4396-a62f-3b35fd57185f
14118.338309 | sda: uid_attribute = ID_SERIAL (setting: storage device configuration)
14118.338314 | sda: recheck_wwid = 1 (setting: multipath.conf defaults/devices section)
14118.338539 | sda: 10240 cyl, 64 heads, 32 sectors/track, start at 0
14118.338558 | sda: vpd_vendor_id = 0 "undef" (setting: multipath internal)
14118.338582 | sda: serial = beaf11
14118.338596 | sda: detect_checker = yes (setting: multipath internal)
14118.339353 | sda: path_checker = tur (setting: multipath internal)
14118.339375 | sda: checker timeout = 30 s (setting: kernel sysfs)
14118.339806 | sda: tur state = up
14118.339827 | sda: uid = 360000000000000000e00000000010001 (udev)
14118.339840 | sda: detect_prio = yes (setting: multipath internal)
14118.339847 | sda: prio = const (setting: storage device configuration)
14118.339851 | sda: prio args = "" (setting: storage device configuration)
14118.339856 | sda: const prio = 1
14118.340090 | sdb: size = 20971520
14118.340297 | sdb: vendor = IET
14118.340335 | sdb: product = VIRTUAL-DISK
14118.340471 | sdb: rev = 0001
14118.341144 | sdb: h:b:t:l = 3:0:0:1
14118.341563 | sdb: tgt_node_name = iqn.2010-10.org.openstack:a19767d8-31bc-4396-a62f-3b35fd57185f
14118.341664 | sdb: uid_attribute = ID_SERIAL (setting: storage device configuration)
14118.341751 | sdb: recheck_wwid = 1 (setting: multipath.conf defaults/devices section)
14118.341987 | sdb: 10240 cyl, 64 heads, 32 sectors/track, start at 0
14118.342083 | sdb: vpd_vendor_id = 0 "undef" (setting: multipath internal)
14118.342175 | sdb: serial = beaf11
14118.342251 | sdb: detect_checker = yes (setting: multipath internal)
14118.342915 | sdb: path_checker = tur (setting: multipath internal)
14118.343016 | sdb: checker timeout = 30 s (setting: kernel sysfs)
14118.343471 | sdb: tur state = up
14118.343571 | sdb: uid = 360000000000000000e00000000010001 (udev)
14118.343642 | sdb: detect_prio = yes (setting: multipath internal)
14118.343714 | sdb: prio = const (setting: storage device configuration)
14118.343785 | sdb: prio args = "" (setting: storage device configuration)
14118.343881 | sdb: const prio = 1
14118.344198 | sdc: size = 20971520
14118.344401 | sdc: vendor = IET
14118.344485 | sdc: product = VIRTUAL-DISK
14118.344616 | sdc: rev = 0001
14118.345371 | sdc: h:b:t:l = 4:0:0:1
14118.345759 | sdc: tgt_node_name = iqn.2010-10.org.openstack:d2540e1d-7e86-4462-aa80-4e9c6a7cc382
14118.345779 | sdc: uid_attribute = ID_SERIAL (setting: storage device configuration)
14118.345783 | sdc: recheck_wwid = 1 (setting: multipath.conf defaults/devices section)
14118.346010 | sdc: 10240 cyl, 64 heads, 32 sectors/track, start at 0
14118.346030 | sdc: vpd_vendor_id = 0 "undef" (setting: multipath internal)
14118.346100 | sdc: serial = beaf11
14118.346117 | sdc: detect_checker = yes (setting: multipath internal)
14118.346842 | sdc: path_checker = tur (setting: multipath internal)
14118.346861 | sdc: checker timeout = 30 s (setting: kernel sysfs)
14118.347357 | sdc: tur state = up
14118.347376 | sdc: uid = 360000000000000000e00000000010001 (udev)
14118.347382 | sdc: detect_prio = yes (setting: multipath internal)
14118.347438 | sdc: prio = const (setting: storage device configuration)
14118.347445 | sdc: prio args = "" (setting: storage device configuration)
14118.347569 | sdc: const prio = 1
14118.347798 | sdd: size = 20971520
14118.348059 | sdd: vendor = IET
14118.348183 | sdd: product = VIRTUAL-DISK
14118.348363 | sdd: rev = 0001
14118.349198 | sdd: h:b:t:l = 5:0:0:1
14118.349656 | sdd: tgt_node_name = iqn.2010-10.org.openstack:d2540e1d-7e86-4462-aa80-4e9c6a7cc382
14118.349759 | sdd: uid_attribute = ID_SERIAL (setting: storage device configuration)
14118.349830 | sdd: recheck_wwid = 1 (setting: multipath.conf defaults/devices section)
14118.350139 | sdd: 10240 cyl, 64 heads, 32 sectors/track, start at 0
14118.350159 | sdd: vpd_vendor_id = 0 "undef" (setting: multipath internal)
14118.350227 | sdd: serial = beaf11
14118.350245 | sdd: detect_checker = yes (setting: multipath internal)
14118.351001 | sdd: path_checker = tur (setting: multipath internal)
14118.351020 | sdd: checker timeout = 30 s (setting: kernel sysfs)
14118.354617 | sdd: tur state = up
14118.354725 | sdd: uid = 360000000000000000e00000000010001 (udev)
14118.354798 | sdd: detect_prio = yes (setting: multipath internal)
14118.354872 | sdd: prio = const (setting: storage device configuration)
14118.354973 | sdd: prio args = "" (setting: storage device configuration)
14118.354981 | sdd: const prio = 1
14118.355188 | sde: size = 20971520
14118.355443 | sde: vendor = IET
14118.355574 | sde: product = VIRTUAL-DISK
14118.355703 | sde: rev = 0001
14118.356573 | sde: h:b:t:l = 6:0:0:1
14118.357084 | sde: tgt_node_name = iqn.2010-10.org.openstack:bd24d8da-a581-4395-af27-91e585b31218
14118.357212 | sde: uid_attribute = ID_SERIAL (setting: storage device configuration)
14118.357303 | sde: recheck_wwid = 1 (setting: multipath.conf defaults/devices section)
14118.357594 | sde: 10240 cyl, 64 heads, 32 sectors/track, start at 0
14118.357713 | sde: vpd_vendor_id = 0 "undef" (setting: multipath internal)
14118.357813 | sde: serial = beaf21
14118.357835 | sde: detect_checker = yes (setting: multipath internal)
14118.358511 | sde: path_checker = tur (setting: multipath internal)
14118.358530 | sde: checker timeout = 30 s (setting: kernel sysfs)
14118.358880 | sde: tur state = up
14118.358898 | sde: uid = 360000000000000000e00000000020001 (udev)
14118.358904 | sde: detect_prio = yes (setting: multipath internal)
14118.358909 | sde: prio = const (setting: storage device configuration)
14118.358913 | sde: prio args = "" (setting: storage device configuration)
14118.358970 | sde: const prio = 1
14118.359163 | sdf: size = 20971520
14118.359359 | sdf: vendor = IET
14118.359491 | sdf: product = VIRTUAL-DISK
14118.359573 | sdf: rev = 0001
14118.360347 | sdf: h:b:t:l = 7:0:0:1
14118.361205 | sdf: tgt_node_name = iqn.2010-10.org.openstack:bd24d8da-a581-4395-af27-91e585b31218
14118.361227 | sdf: uid_attribute = ID_SERIAL (setting: storage device configuration)
14118.361232 | sdf: recheck_wwid = 1 (setting: multipath.conf defaults/devices section)
14118.361456 | sdf: 10240 cyl, 64 heads, 32 sectors/track, start at 0
14118.361475 | sdf: vpd_vendor_id = 0 "undef" (setting: multipath internal)
14118.361545 | sdf: serial = beaf21
14118.361562 | sdf: detect_checker = yes (setting: multipath internal)
14118.362048 | sdf: path_checker = tur (setting: multipath internal)
14118.362070 | sdf: checker timeout = 30 s (setting: kernel sysfs)
14118.362518 | sdf: tur state = up
14118.362539 | sdf: uid = 360000000000000000e00000000020001 (udev)
14118.362548 | sdf: detect_prio = yes (setting: multipath internal)
14118.362557 | sdf: prio = const (setting: storage device configuration)
14118.362672 | sdf: prio args = "" (setting: storage device configuration)
14118.363579 | sdf: const prio = 1
14118.363886 | sdh: size = 75497472
14118.364142 | sdh: vendor = IET
14118.364183 | sdh: product = VIRTUAL-DISK
14118.364217 | sdh: rev = 0001
14118.364981 | sdh: h:b:t:l = 8:0:0:1
14118.365318 | sdh: tgt_node_name = iqn.2010-10.org.openstack:9f62e564-ed73-46d0-b0e0-2089ac63918e
14118.365338 | sdh: uid_attribute = ID_SERIAL (setting: storage device configuration)
14118.365342 | sdh: recheck_wwid = 1 (setting: multipath.conf defaults/devices section)
14118.366392 | sdh: 36864 cyl, 64 heads, 32 sectors/track, start at 0
14118.366411 | sdh: vpd_vendor_id = 0 "undef" (setting: multipath internal)
14118.366438 | sdh: serial = beaf21
14118.366453 | sdh: detect_checker = yes (setting: multipath internal)
14118.366882 | sdh: path_checker = tur (setting: multipath internal)
14118.366900 | sdh: checker timeout = 30 s (setting: kernel sysfs)
14118.367305 | sdh: tur state = up
14118.367323 | sdh: uid = 360000000000000000e00000000020001 (udev)
14118.367329 | sdh: detect_prio = yes (setting: multipath internal)
14118.367343 | sdh: prio = const (setting: storage device configuration)
14118.367356 | sdh: prio args = "" (setting: storage device configuration)
14118.367361 | sdh: const prio = 1
14118.367517 | sdg: size = 75497472
14118.367656 | sdg: vendor = IET
14118.367692 | sdg: product = VIRTUAL-DISK
14118.367724 | sdg: rev = 0001
14118.368429 | sdg: h:b:t:l = 9:0:0:1
14118.368753 | sdg: tgt_node_name = iqn.2010-10.org.openstack:9f62e564-ed73-46d0-b0e0-2089ac63918e
14118.368771 | sdg: uid_attribute = ID_SERIAL (setting: storage device configuration)
14118.368776 | sdg: recheck_wwid = 1 (setting: multipath.conf defaults/devices section)
14118.368944 | sdg: 36864 cyl, 64 heads, 32 sectors/track, start at 0
14118.368962 | sdg: vpd_vendor_id = 0 "undef" (setting: multipath internal)
14118.368983 | sdg: serial = beaf21
14118.368997 | sdg: detect_checker = yes (setting: multipath internal)
14118.369486 | sdg: path_checker = tur (setting: multipath internal)
14118.369505 | sdg: checker timeout = 30 s (setting: kernel sysfs)
14118.369875 | sdg: tur state = up
14118.369892 | sdg: uid = 360000000000000000e00000000020001 (udev)
14118.369897 | sdg: detect_prio = yes (setting: multipath internal)
14118.369902 | sdg: prio = const (setting: storage device configuration)
14118.369923 | sdg: prio args = "" (setting: storage device configuration)
14118.369928 | sdg: const prio = 1
14118.370030 | rbd0: device node name blacklisted
14118.370133 | xvda: device node name blacklisted
14118.370403 | xvdb: device node name blacklisted
14118.370591 | loop0: device node name blacklisted
14118.370686 | loop1: device node name blacklisted
14118.370763 | loop2: device node name blacklisted
14118.370839 | loop3: device node name blacklisted
14118.370931 | loop4: device node name blacklisted
14118.371007 | loop5: device node name blacklisted
14118.371096 | loop6: device node name blacklisted
14118.371183 | loop7: device node name blacklisted
14118.371285 | nbd0: device node name blacklisted
14118.371370 | nbd1: device node name blacklisted
14118.371459 | nbd10: device node name blacklisted
14118.371549 | nbd11: device node name blacklisted
14118.371640 | nbd12: device node name blacklisted
14118.371730 | nbd13: device node name blacklisted
14118.371806 | nbd14: device node name blacklisted
14118.371884 | nbd15: device node name blacklisted
14118.371973 | nbd2: device node name blacklisted
14118.372102 | nbd3: device node name blacklisted
14118.372193 | nbd4: device node name blacklisted
14118.372284 | nbd5: device node name blacklisted
14118.372372 | nbd6: device node name blacklisted
14118.372463 | nbd7: device node name blacklisted
14118.372540 | nbd8: device node name blacklisted
14118.372616 | nbd9: device node name blacklisted
14118.372696 | dm-0: device node name blacklisted
14118.372782 | dm-1: device node name blacklisted
===== paths list =====
uuid                              hcil    dev dev_t pri dm_st chk_st vend/prod/rev    dev_st 
360000000000000000e00000000010001 2:0:0:1 sda 8:0   1   undef undef  IET,VIRTUAL-DISK unknown
360000000000000000e00000000010001 3:0:0:1 sdb 8:16  1   undef undef  IET,VIRTUAL-DISK unknown
360000000000000000e00000000010001 4:0:0:1 sdc 8:32  1   undef undef  IET,VIRTUAL-DISK unknown
360000000000000000e00000000010001 5:0:0:1 sdd 8:48  1   undef undef  IET,VIRTUAL-DISK unknown
360000000000000000e00000000020001 6:0:0:1 sde 8:64  1   undef undef  IET,VIRTUAL-DISK unknown
360000000000000000e00000000020001 7:0:0:1 sdf 8:80  1   undef undef  IET,VIRTUAL-DISK unknown
360000000000000000e00000000020001 8:0:0:1 sdh 8:112 1   undef undef  IET,VIRTUAL-DISK unknown
360000000000000000e00000000020001 9:0:0:1 sdg 8:96  1   undef undef  IET,VIRTUAL-DISK unknown
14118.374590 | multipath-tools v0.8.8 (03/12, 2021)
14118.374620 | libdevmapper version 1.02.175
14118.374790 | kernel device mapper v4.45.0
14118.374823 | DM multipath kernel driver v1.14.0
14118.374991 | sda: size = 20971520
14118.375012 | sda: vendor = IET
14118.375017 | sda: product = VIRTUAL-DISK
14118.375037 | sda: rev = 0001
14118.375655 | sda: h:b:t:l = 2:0:0:1
14118.375792 | sda: tgt_node_name = iqn.2010-10.org.openstack:a19767d8-31bc-4396-a62f-3b35fd57185f
14118.375830 | sda: 10240 cyl, 64 heads, 32 sectors/track, start at 0
14118.375845 | sda: vpd_vendor_id = 0 "undef" (setting: multipath internal)
14118.375866 | sda: serial = beaf11
14118.376463 | sda: tur state = up
14118.376513 | sdb: size = 20971520
14118.376532 | sdb: vendor = IET
14118.376537 | sdb: product = VIRTUAL-DISK
14118.376549 | sdb: rev = 0001
14118.377113 | sdb: h:b:t:l = 3:0:0:1
14118.377280 | sdb: tgt_node_name = iqn.2010-10.org.openstack:a19767d8-31bc-4396-a62f-3b35fd57185f
14118.377321 | sdb: 10240 cyl, 64 heads, 32 sectors/track, start at 0
14118.377337 | sdb: vpd_vendor_id = 0 "undef" (setting: multipath internal)
14118.377359 | sdb: serial = beaf11
14118.377784 | sdb: tur state = up
14118.377872 | sdc: size = 20971520
14118.377900 | sdc: vendor = IET
14118.377922 | sdc: product = VIRTUAL-DISK
14118.377967 | sdc: rev = 0001
14118.378576 | sdc: h:b:t:l = 4:0:0:1
14118.378748 | sdc: tgt_node_name = iqn.2010-10.org.openstack:d2540e1d-7e86-4462-aa80-4e9c6a7cc382
14118.378799 | sdc: 10240 cyl, 64 heads, 32 sectors/track, start at 0
14118.378838 | sdc: vpd_vendor_id = 0 "undef" (setting: multipath internal)
14118.378899 | sdc: serial = beaf11
14118.379346 | sdc: tur state = up
14118.379393 | sdd: size = 20971520
14118.379411 | sdd: vendor = IET
14118.379416 | sdd: product = VIRTUAL-DISK
14118.379437 | sdd: rev = 0001
14118.380041 | sdd: h:b:t:l = 5:0:0:1
14118.380175 | sdd: tgt_node_name = iqn.2010-10.org.openstack:d2540e1d-7e86-4462-aa80-4e9c6a7cc382
14118.380211 | sdd: 10240 cyl, 64 heads, 32 sectors/track, start at 0
14118.380234 | sdd: vpd_vendor_id = 0 "undef" (setting: multipath internal)
14118.380295 | sdd: serial = beaf11
14118.380677 | sdd: tur state = up
14118.380761 | sdf: size = 20971520
14118.380786 | sdf: vendor = IET
14118.380811 | sdf: product = VIRTUAL-DISK
14118.380831 | sdf: rev = 0001
14118.381392 | sdf: h:b:t:l = 7:0:0:1
14118.381527 | sdf: tgt_node_name = iqn.2010-10.org.openstack:bd24d8da-a581-4395-af27-91e585b31218
14118.381564 | sdf: 10240 cyl, 64 heads, 32 sectors/track, start at 0
14118.381579 | sdf: vpd_vendor_id = 0 "undef" (setting: multipath internal)
14118.381601 | sdf: serial = beaf21
14118.382092 | sdf: tur state = up
14118.382174 | sde: size = 20971520
14118.382202 | sde: vendor = IET
14118.382228 | sde: product = VIRTUAL-DISK
14118.382241 | sde: rev = 0001
14118.382830 | sde: h:b:t:l = 6:0:0:1
14118.383002 | sde: tgt_node_name = iqn.2010-10.org.openstack:bd24d8da-a581-4395-af27-91e585b31218
14118.383053 | sde: 10240 cyl, 64 heads, 32 sectors/track, start at 0
14118.383108 | sde: vpd_vendor_id = 0 "undef" (setting: multipath internal)
14118.383132 | sde: serial = beaf21
14118.383577 | sde: tur state = up
14118.383625 | sda: udev property ID_WWN whitelisted
14118.383695 | sdb: udev property ID_WWN whitelisted
14118.383720 | sdc: udev property ID_WWN whitelisted
14118.383785 | sdd: udev property ID_WWN whitelisted
14118.383817 | sde: udev property ID_WWN whitelisted
14118.383840 | sdf: udev property ID_WWN whitelisted
14118.383887 | sdh: udev property ID_WWN whitelisted
14118.383932 | wwid 360000000000000000e00000000020001 is already multipathed, keeping it
14118.383959 | 360000000000000000e00000000020001: user_friendly_names = no (setting: multipath internal)
14118.383975 | 360000000000000000e00000000020001: alias = 360000000000000000e00000000020001 (setting: default to WWID)
14118.383981 | sde: size mismatch for 360000000000000000e00000000020001, not adding path
14118.383993 | sdf: size mismatch for 360000000000000000e00000000020001, not adding path
14118.384736 | sdh: tur state = up
14118.384773 | sdh: ownership set to 360000000000000000e00000000020001
14118.385206 | sdg: tur state = up
14118.385225 | sdg: ownership set to 360000000000000000e00000000020001
14118.385269 | 360000000000000000e00000000020001: failback = "immediate" (setting: storage device configuration)
14118.385285 | 360000000000000000e00000000020001: path_grouping_policy = multibus (setting: storage device configuration)
14118.385291 | 360000000000000000e00000000020001: path_selector = "queue-length 0" (setting: storage device configuration)
14118.385307 | 360000000000000000e00000000020001: no_path_retry = "fail" (setting: storage device configuration)
14118.385351 | 360000000000000000e00000000020001: retain_attached_hw_handler = yes (setting: implied in kernel >= 4.3.0)
14118.385369 | 360000000000000000e00000000020001: features = "0" (setting: multipath internal)
14118.385424 | 360000000000000000e00000000020001: hardware_handler = "0" (setting: multipath internal)
14118.385431 | 360000000000000000e00000000020001: rr_weight = "uniform" (setting: multipath internal)
14118.385435 | 360000000000000000e00000000020001: minio = 1 (setting: multipath internal)
14118.385441 | 360000000000000000e00000000020001: fast_io_fail_tmo = 5 (setting: multipath internal)
14118.385446 | 360000000000000000e00000000020001: deferred_remove = no (setting: multipath internal)
14118.385451 | 360000000000000000e00000000020001: marginal_path_err_sample_time = "no" (setting: multipath internal)
14118.385456 | 360000000000000000e00000000020001: marginal_path_err_rate_threshold = "no" (setting: multipath internal)
14118.385461 | 360000000000000000e00000000020001: marginal_path_err_recheck_gap_time = "no" (setting: multipath internal)
14118.385490 | 360000000000000000e00000000020001: marginal_path_double_failed_time = "no" (setting: multipath internal)
14118.385496 | 360000000000000000e00000000020001: san_path_err_threshold = "no" (setting: multipath internal)
14118.385500 | 360000000000000000e00000000020001: san_path_err_forget_rate = "no" (setting: multipath internal)
14118.385505 | 360000000000000000e00000000020001: san_path_err_recovery_time = "no" (setting: multipath internal)
14118.385510 | 360000000000000000e00000000020001: skip_kpartx = no (setting: multipath internal)
14118.385547 | 360000000000000000e00000000020001: ghost_delay = "no" (setting: multipath.conf defaults/devices section)
14118.385565 | 360000000000000000e00000000020001: flush_on_last_del = no (setting: multipath internal)
: 360000000000000000e00000000020001 undef IET,VIRTUAL-DISK
size=36G features='0' hwhandler='0' wp=undef
`-+- policy='queue-length 0' prio=1 status=undef
  |- 8:0:0:1 sdh 8:112 undef ready running
  `- 9:0:0:1 sdg 8:96  undef ready running
14118.386080 | sdg: udev property ID_WWN whitelisted
14118.386598 | unloading tur checker
14118.386674 | unloading const prioritizer
root@genestack-compute1:~#
timolow commented 1 month ago

We might need to add a SN field here: https://github.com/openstack/cinder/blob/master/cinder/volume/targets/iscsi.py#L195

timolow commented 1 month ago 
root@genestack-storage1:~# cat /var/lib/cinder/volumes/a19767d8-31bc-4396-a62f-3b35fd57185f 

<target iqn.2010-10.org.openstack:a19767d8-31bc-4396-a62f-3b35fd57185f>
    backing-store /dev/cinder-volumes-1/a19767d8-31bc-4396-a62f-3b35fd57185f
    driver iscsi
    incominguser Xzyajwk9N8Ej4dxxLoVa fT5EFvNWt4dYffdi

    write-cache on
</target>
timolow commented 1 month ago

Fix applied to fork of Openstack cinder: https://github.com/timolow/cinder/blob/master/cinder/volume/targets/tgt.py https://github.com/timolow/cinder/commit/f79b7fc0e4915e45c29b6bc50768b71dbf433663

TGT now outputs unique SN and ID:

Target 2: iqn.2010-10.org.openstack:4db17aa5-bd26-4ca0-bcb1-98b17b44a756
    System information:
        Driver: iscsi
        State: ready
    I_T nexus information:
        I_T nexus: 27
            Initiator: iqn.2004-10.com.ubuntu:01:ccf450ecfaa2 alias: genestack-compute1.lab.underworld.local
            Connection: 0
                IP Address: 10.1.1.212
        I_T nexus: 28
            Initiator: iqn.2004-10.com.ubuntu:01:ccf450ecfaa2 alias: genestack-compute1.lab.underworld.local
            Connection: 0
                IP Address: 10.1.2.212
    LUN information:
        LUN: 0
            Type: controller
            SCSI ID: IET     00020000
            SCSI SN: beaf20
            Size: 0 MB, Block size: 1
            Online: Yes
            Removable media: No
            Prevent removal: No
            Readonly: No
            SWP: No
            Thin-provisioning: No
            Backing store type: null
            Backing store path: None
            Backing store flags: 
        LUN: 1
            Type: disk
            SCSI ID: 4db17aa5-bd26-4ca0-bcb1-98b17b44a756
            SCSI SN: 4db17aa5-bd26-4ca0-bcb1-98b17b44a756
            Size: 10737 MB, Block size: 512
            Online: Yes
            Removable media: No
            Prevent removal: No
            Readonly: No
            SWP: No
            Thin-provisioning: No
            Backing store type: rdwr
            Backing store path: /dev/cinder-volumes-1/4db17aa5-bd26-4ca0-bcb1-98b17b44a756
            Backing store flags: 
    Account information:
        iodthiu9TmDzRZaNu2CF
    ACL information:
        ALL

Compute node with 4 attached volumes: 

root@genestack-compute1:~# multipath -v3 -d
39299.324958 | set open fds limit to 1048576/1048576
39299.324994 | loading //lib/multipath/libchecktur.so checker
39299.325114 | checker tur: message table size = 3
39299.325138 | loading //lib/multipath/libprioconst.so prioritizer
39299.325273 | _init_foreign: foreign library "nvme" is not enabled
39299.331331 | sda: size = 20971520
39299.331489 | sda: vendor = IET
39299.331532 | sda: product = VIRTUAL-DISK
39299.331565 | sda: rev = 0001
39299.332319 | sda: h:b:t:l = 2:0:0:1
39299.332821 | sda: tgt_node_name = iqn.2010-10.org.openstack:86bbf246-9676-422b-abbe-7e3264e9c953
39299.332848 | sda: uid_attribute = ID_SERIAL (setting: storage device configuration)
39299.332856 | sda: recheck_wwid = 1 (setting: multipath.conf defaults/devices section)
39299.333284 | sda: 10240 cyl, 64 heads, 32 sectors/track, start at 0
39299.333310 | sda: vpd_vendor_id = 0 "undef" (setting: multipath internal)
39299.333351 | sda: serial = 86bbf246-9676-422b-abbe-7e3264e9c953
39299.333375 | sda: detect_checker = yes (setting: multipath internal)
39299.334201 | sda: path_checker = tur (setting: multipath internal)
39299.334229 | sda: checker timeout = 30 s (setting: kernel sysfs)
39299.335425 | sda: tur state = up
39299.335455 | sda: uid = 36246096760422b0abbe07e3264e9c953 (udev)
39299.335485 | sda: detect_prio = yes (setting: multipath internal)
39299.335505 | sda: prio = const (setting: storage device configuration)
39299.335524 | sda: prio args = "" (setting: storage device configuration)
39299.335550 | sda: const prio = 1
39299.335832 | sdb: size = 20971520
39299.336114 | sdb: vendor = IET
39299.336165 | sdb: product = VIRTUAL-DISK
39299.336219 | sdb: rev = 0001
39299.337050 | sdb: h:b:t:l = 3:0:0:1
39299.337427 | sdb: tgt_node_name = iqn.2010-10.org.openstack:86bbf246-9676-422b-abbe-7e3264e9c953
39299.337448 | sdb: uid_attribute = ID_SERIAL (setting: storage device configuration)
39299.337455 | sdb: recheck_wwid = 1 (setting: multipath.conf defaults/devices section)
39299.337668 | sdb: 10240 cyl, 64 heads, 32 sectors/track, start at 0
39299.337688 | sdb: vpd_vendor_id = 0 "undef" (setting: multipath internal)
39299.337723 | sdb: serial = 86bbf246-9676-422b-abbe-7e3264e9c953
39299.337742 | sdb: detect_checker = yes (setting: multipath internal)
39299.338361 | sdb: path_checker = tur (setting: multipath internal)
39299.338382 | sdb: checker timeout = 30 s (setting: kernel sysfs)
39299.338815 | sdb: tur state = up
39299.338834 | sdb: uid = 36246096760422b0abbe07e3264e9c953 (udev)
39299.338840 | sdb: detect_prio = yes (setting: multipath internal)
39299.338845 | sdb: prio = const (setting: storage device configuration)
39299.338859 | sdb: prio args = "" (setting: storage device configuration)
39299.338877 | sdb: const prio = 1
39299.339073 | sdc: size = 20971520
39299.339270 | sdc: vendor = IET
39299.339322 | sdc: product = VIRTUAL-DISK
39299.339375 | sdc: rev = 0001
39299.340130 | sdc: h:b:t:l = 4:0:0:1
39299.340457 | sdc: tgt_node_name = iqn.2010-10.org.openstack:4db17aa5-bd26-4ca0-bcb1-98b17b44a756
39299.340475 | sdc: uid_attribute = ID_SERIAL (setting: storage device configuration)
39299.340479 | sdc: recheck_wwid = 1 (setting: multipath.conf defaults/devices section)
39299.340649 | sdc: 10240 cyl, 64 heads, 32 sectors/track, start at 0
39299.340666 | sdc: vpd_vendor_id = 0 "undef" (setting: multipath internal)
39299.340687 | sdc: serial = 4db17aa5-bd26-4ca0-bcb1-98b17b44a756
39299.340700 | sdc: detect_checker = yes (setting: multipath internal)
39299.341195 | sdc: path_checker = tur (setting: multipath internal)
39299.341215 | sdc: checker timeout = 30 s (setting: kernel sysfs)
39299.341565 | sdc: tur state = up
39299.341585 | sdc: uid = 36aa50bd2604ca00bcb1098b17b44a756 (udev)
39299.341590 | sdc: detect_prio = yes (setting: multipath internal)
39299.341614 | sdc: prio = const (setting: storage device configuration)
39299.341623 | sdc: prio args = "" (setting: storage device configuration)
39299.341651 | sdc: const prio = 1
39299.341801 | sdd: size = 20971520
39299.342073 | sdd: vendor = IET
39299.342116 | sdd: product = VIRTUAL-DISK
39299.342140 | sdd: rev = 0001
39299.342811 | sdd: h:b:t:l = 5:0:0:1
39299.343154 | sdd: tgt_node_name = iqn.2010-10.org.openstack:4db17aa5-bd26-4ca0-bcb1-98b17b44a756
39299.343172 | sdd: uid_attribute = ID_SERIAL (setting: storage device configuration)
39299.343176 | sdd: recheck_wwid = 1 (setting: multipath.conf defaults/devices section)
39299.343354 | sdd: 10240 cyl, 64 heads, 32 sectors/track, start at 0
39299.343370 | sdd: vpd_vendor_id = 0 "undef" (setting: multipath internal)
39299.343444 | sdd: serial = 4db17aa5-bd26-4ca0-bcb1-98b17b44a756
39299.343469 | sdd: detect_checker = yes (setting: multipath internal)
39299.343988 | sdd: path_checker = tur (setting: multipath internal)
39299.344380 | sdd: checker timeout = 30 s (setting: kernel sysfs)
39299.344761 | sdd: tur state = up
39299.344780 | sdd: uid = 36aa50bd2604ca00bcb1098b17b44a756 (udev)
39299.344785 | sdd: detect_prio = yes (setting: multipath internal)
39299.344797 | sdd: prio = const (setting: storage device configuration)
39299.344808 | sdd: prio args = "" (setting: storage device configuration)
39299.344813 | sdd: const prio = 1
39299.344976 | sde: size = 20971520
39299.345133 | sde: vendor = IET
39299.345174 | sde: product = VIRTUAL-DISK
39299.345205 | sde: rev = 0001
39299.345851 | sde: h:b:t:l = 6:0:0:1
39299.346193 | sde: tgt_node_name = iqn.2010-10.org.openstack:0d286758-8b21-4bca-bcd4-8defefc8e060
39299.346210 | sde: uid_attribute = ID_SERIAL (setting: storage device configuration)
39299.346214 | sde: recheck_wwid = 1 (setting: multipath.conf defaults/devices section)
39299.346404 | sde: 10240 cyl, 64 heads, 32 sectors/track, start at 0
39299.346420 | sde: vpd_vendor_id = 0 "undef" (setting: multipath internal)
39299.346441 | sde: serial = 0d286758-8b21-4bca-bcd4-8defefc8e060
39299.346454 | sde: detect_checker = yes (setting: multipath internal)
39299.347074 | sde: path_checker = tur (setting: multipath internal)
39299.347090 | sde: checker timeout = 30 s (setting: kernel sysfs)
39299.347363 | sde: tur state = up
39299.347382 | sde: uid = 3675808b2104bca0bcd408defefc8e060 (udev)
39299.347387 | sde: detect_prio = yes (setting: multipath internal)
39299.347405 | sde: prio = const (setting: storage device configuration)
39299.347410 | sde: prio args = "" (setting: storage device configuration)
39299.347424 | sde: const prio = 1
39299.347555 | sdf: size = 20971520
39299.347690 | sdf: vendor = IET
39299.347723 | sdf: product = VIRTUAL-DISK
39299.347746 | sdf: rev = 0001
39299.348486 | sdf: h:b:t:l = 7:0:0:1
39299.348814 | sdf: tgt_node_name = iqn.2010-10.org.openstack:0d286758-8b21-4bca-bcd4-8defefc8e060
39299.348832 | sdf: uid_attribute = ID_SERIAL (setting: storage device configuration)
39299.348836 | sdf: recheck_wwid = 1 (setting: multipath.conf defaults/devices section)
39299.349006 | sdf: 10240 cyl, 64 heads, 32 sectors/track, start at 0
39299.349023 | sdf: vpd_vendor_id = 0 "undef" (setting: multipath internal)
39299.349044 | sdf: serial = 0d286758-8b21-4bca-bcd4-8defefc8e060
39299.349057 | sdf: detect_checker = yes (setting: multipath internal)
39299.349521 | sdf: path_checker = tur (setting: multipath internal)
39299.349538 | sdf: checker timeout = 30 s (setting: kernel sysfs)
39299.349803 | sdf: tur state = up
39299.349822 | sdf: uid = 3675808b2104bca0bcd408defefc8e060 (udev)
39299.349827 | sdf: detect_prio = yes (setting: multipath internal)
39299.349839 | sdf: prio = const (setting: storage device configuration)
39299.349852 | sdf: prio args = "" (setting: storage device configuration)
39299.349869 | sdf: const prio = 1
39299.349999 | sdg: size = 20971520
39299.350146 | sdg: vendor = IET
39299.350181 | sdg: product = VIRTUAL-DISK
39299.350212 | sdg: rev = 0001
39299.350874 | sdg: h:b:t:l = 8:0:0:1
39299.351190 | sdg: tgt_node_name = iqn.2010-10.org.openstack:a38f7efb-8650-454d-a533-c8b5857c6451
39299.351208 | sdg: uid_attribute = ID_SERIAL (setting: storage device configuration)
39299.351212 | sdg: recheck_wwid = 1 (setting: multipath.conf defaults/devices section)
39299.351387 | sdg: 10240 cyl, 64 heads, 32 sectors/track, start at 0
39299.351403 | sdg: vpd_vendor_id = 0 "undef" (setting: multipath internal)
39299.351424 | sdg: serial = a38f7efb-8650-454d-a533-c8b5857c6451
39299.351437 | sdg: detect_checker = yes (setting: multipath internal)
39299.351840 | sdg: path_checker = tur (setting: multipath internal)
39299.351857 | sdg: checker timeout = 30 s (setting: kernel sysfs)
39299.352109 | sdg: tur state = up
39299.352129 | sdg: uid = 36efb086500454d0a5330c8b5857c6451 (udev)
39299.352134 | sdg: detect_prio = yes (setting: multipath internal)
39299.352153 | sdg: prio = const (setting: storage device configuration)
39299.352178 | sdg: prio args = "" (setting: storage device configuration)
39299.352197 | sdg: const prio = 1
39299.352346 | sdh: size = 20971520
39299.352502 | sdh: vendor = IET
39299.352543 | sdh: product = VIRTUAL-DISK
39299.352574 | sdh: rev = 0001
39299.353252 | sdh: h:b:t:l = 9:0:0:1
39299.353593 | sdh: tgt_node_name = iqn.2010-10.org.openstack:a38f7efb-8650-454d-a533-c8b5857c6451
39299.353611 | sdh: uid_attribute = ID_SERIAL (setting: storage device configuration)
39299.353616 | sdh: recheck_wwid = 1 (setting: multipath.conf defaults/devices section)
39299.353808 | sdh: 10240 cyl, 64 heads, 32 sectors/track, start at 0
39299.353825 | sdh: vpd_vendor_id = 0 "undef" (setting: multipath internal)
39299.353882 | sdh: serial = a38f7efb-8650-454d-a533-c8b5857c6451
39299.353891 | sdh: detect_checker = yes (setting: multipath internal)
39299.354357 | sdh: path_checker = tur (setting: multipath internal)
39299.354375 | sdh: checker timeout = 30 s (setting: kernel sysfs)
39299.354726 | sdh: tur state = up
39299.354745 | sdh: uid = 36efb086500454d0a5330c8b5857c6451 (udev)
39299.354750 | sdh: detect_prio = yes (setting: multipath internal)
39299.354767 | sdh: prio = const (setting: storage device configuration)
39299.354772 | sdh: prio args = "" (setting: storage device configuration)
39299.354789 | sdh: const prio = 1
39299.354905 | rbd0: device node name blacklisted
39299.355019 | xvda: device node name blacklisted
39299.355321 | xvdb: device node name blacklisted
39299.355491 | loop0: device node name blacklisted
39299.355579 | loop1: device node name blacklisted
39299.355709 | loop2: device node name blacklisted
39299.355862 | loop3: device node name blacklisted
39299.355962 | loop4: device node name blacklisted
39299.356197 | loop5: device node name blacklisted
39299.356287 | loop6: device node name blacklisted
39299.356426 | loop7: device node name blacklisted
39299.356547 | nbd0: device node name blacklisted
39299.356652 | nbd1: device node name blacklisted
39299.356800 | nbd10: device node name blacklisted
39299.356901 | nbd11: device node name blacklisted
39299.357040 | nbd12: device node name blacklisted
39299.357133 | nbd13: device node name blacklisted
39299.357231 | nbd14: device node name blacklisted
39299.357371 | nbd15: device node name blacklisted
39299.357481 | nbd2: device node name blacklisted
39299.357588 | nbd3: device node name blacklisted
39299.357694 | nbd4: device node name blacklisted
39299.357794 | nbd5: device node name blacklisted
39299.357962 | nbd6: device node name blacklisted
39299.358061 | nbd7: device node name blacklisted
39299.358167 | nbd8: device node name blacklisted
39299.358263 | nbd9: device node name blacklisted
39299.358432 | dm-0: device node name blacklisted
39299.358534 | dm-1: device node name blacklisted
39299.358634 | dm-2: device node name blacklisted
39299.358765 | dm-3: device node name blacklisted
===== paths list =====
uuid                              hcil    dev dev_t pri dm_st chk_st vend/prod/rev    dev_st 
36246096760422b0abbe07e3264e9c953 2:0:0:1 sda 8:0   1   undef undef  IET,VIRTUAL-DISK unknown
36246096760422b0abbe07e3264e9c953 3:0:0:1 sdb 8:16  1   undef undef  IET,VIRTUAL-DISK unknown
36aa50bd2604ca00bcb1098b17b44a756 4:0:0:1 sdc 8:32  1   undef undef  IET,VIRTUAL-DISK unknown
36aa50bd2604ca00bcb1098b17b44a756 5:0:0:1 sdd 8:48  1   undef undef  IET,VIRTUAL-DISK unknown
3675808b2104bca0bcd408defefc8e060 6:0:0:1 sde 8:64  1   undef undef  IET,VIRTUAL-DISK unknown
3675808b2104bca0bcd408defefc8e060 7:0:0:1 sdf 8:80  1   undef undef  IET,VIRTUAL-DISK unknown
36efb086500454d0a5330c8b5857c6451 8:0:0:1 sdg 8:96  1   undef undef  IET,VIRTUAL-DISK unknown
36efb086500454d0a5330c8b5857c6451 9:0:0:1 sdh 8:112 1   undef undef  IET,VIRTUAL-DISK unknown
39299.360765 | multipath-tools v0.8.8 (03/12, 2021)
39299.360794 | libdevmapper version 1.02.175
39299.360962 | kernel device mapper v4.45.0
39299.361006 | DM multipath kernel driver v1.14.0
39299.361260 | sda: size = 20971520
39299.361281 | sda: vendor = IET
39299.361285 | sda: product = VIRTUAL-DISK
39299.361302 | sda: rev = 0001
39299.361926 | sda: h:b:t:l = 2:0:0:1
39299.362059 | sda: tgt_node_name = iqn.2010-10.org.openstack:86bbf246-9676-422b-abbe-7e3264e9c953
39299.362106 | sda: 10240 cyl, 64 heads, 32 sectors/track, start at 0
39299.362113 | sda: vpd_vendor_id = 0 "undef" (setting: multipath internal)
39299.362150 | sda: serial = 86bbf246-9676-422b-abbe-7e3264e9c953
39299.362789 | sda: tur state = up
39299.362838 | sdb: size = 20971520
39299.362855 | sdb: vendor = IET
39299.362859 | sdb: product = VIRTUAL-DISK
39299.362863 | sdb: rev = 0001
39299.363469 | sdb: h:b:t:l = 3:0:0:1
39299.363713 | sdb: tgt_node_name = iqn.2010-10.org.openstack:86bbf246-9676-422b-abbe-7e3264e9c953
39299.363776 | sdb: 10240 cyl, 64 heads, 32 sectors/track, start at 0
39299.363799 | sdb: vpd_vendor_id = 0 "undef" (setting: multipath internal)
39299.363835 | sdb: serial = 86bbf246-9676-422b-abbe-7e3264e9c953
39299.364489 | sdb: tur state = up
39299.364614 | sde: size = 20971520
39299.364636 | sde: vendor = IET
39299.364640 | sde: product = VIRTUAL-DISK
39299.364645 | sde: rev = 0001
39299.365358 | sde: h:b:t:l = 6:0:0:1
39299.365556 | sde: tgt_node_name = iqn.2010-10.org.openstack:0d286758-8b21-4bca-bcd4-8defefc8e060
39299.365612 | sde: 10240 cyl, 64 heads, 32 sectors/track, start at 0
39299.365636 | sde: vpd_vendor_id = 0 "undef" (setting: multipath internal)
39299.365694 | sde: serial = 0d286758-8b21-4bca-bcd4-8defefc8e060
39299.366277 | sde: tur state = up
39299.366335 | sdf: size = 20971520
39299.366367 | sdf: vendor = IET
39299.366387 | sdf: product = VIRTUAL-DISK
39299.366395 | sdf: rev = 0001
39299.367048 | sdf: h:b:t:l = 7:0:0:1
39299.367275 | sdf: tgt_node_name = iqn.2010-10.org.openstack:0d286758-8b21-4bca-bcd4-8defefc8e060
39299.367315 | sdf: 10240 cyl, 64 heads, 32 sectors/track, start at 0
39299.367333 | sdf: vpd_vendor_id = 0 "undef" (setting: multipath internal)
39299.367405 | sdf: serial = 0d286758-8b21-4bca-bcd4-8defefc8e060
39299.367756 | sdf: tur state = up
39299.367879 | sdc: size = 20971520
39299.367904 | sdc: vendor = IET
39299.367913 | sdc: product = VIRTUAL-DISK
39299.367946 | sdc: rev = 0001
39299.368671 | sdc: h:b:t:l = 4:0:0:1
39299.368856 | sdc: tgt_node_name = iqn.2010-10.org.openstack:4db17aa5-bd26-4ca0-bcb1-98b17b44a756
39299.368919 | sdc: 10240 cyl, 64 heads, 32 sectors/track, start at 0
39299.368966 | sdc: vpd_vendor_id = 0 "undef" (setting: multipath internal)
39299.369015 | sdc: serial = 4db17aa5-bd26-4ca0-bcb1-98b17b44a756
39299.369456 | sdc: tur state = up
39299.369515 | sdd: size = 20971520
39299.369545 | sdd: vendor = IET
39299.369565 | sdd: product = VIRTUAL-DISK
39299.369573 | sdd: rev = 0001
39299.370219 | sdd: h:b:t:l = 5:0:0:1
39299.370436 | sdd: tgt_node_name = iqn.2010-10.org.openstack:4db17aa5-bd26-4ca0-bcb1-98b17b44a756
39299.370483 | sdd: 10240 cyl, 64 heads, 32 sectors/track, start at 0
39299.370506 | sdd: vpd_vendor_id = 0 "undef" (setting: multipath internal)
39299.370559 | sdd: serial = 4db17aa5-bd26-4ca0-bcb1-98b17b44a756
39299.370987 | sdd: tur state = up
39299.371122 | sdh: size = 20971520
39299.371145 | sdh: vendor = IET
39299.371155 | sdh: product = VIRTUAL-DISK
39299.371182 | sdh: rev = 0001
39299.371846 | sdh: h:b:t:l = 9:0:0:1
39299.372272 | sdh: tgt_node_name = iqn.2010-10.org.openstack:a38f7efb-8650-454d-a533-c8b5857c6451
39299.372311 | sdh: 10240 cyl, 64 heads, 32 sectors/track, start at 0
39299.372325 | sdh: vpd_vendor_id = 0 "undef" (setting: multipath internal)
39299.372346 | sdh: serial = a38f7efb-8650-454d-a533-c8b5857c6451
39299.372784 | sdh: tur state = up
39299.372834 | sdg: size = 20971520
39299.372842 | sdg: vendor = IET
39299.372847 | sdg: product = VIRTUAL-DISK
39299.372851 | sdg: rev = 0001
39299.373414 | sdg: h:b:t:l = 8:0:0:1
39299.373547 | sdg: tgt_node_name = iqn.2010-10.org.openstack:a38f7efb-8650-454d-a533-c8b5857c6451
39299.373584 | sdg: 10240 cyl, 64 heads, 32 sectors/track, start at 0
39299.373598 | sdg: vpd_vendor_id = 0 "undef" (setting: multipath internal)
39299.373618 | sdg: serial = a38f7efb-8650-454d-a533-c8b5857c6451
39299.374087 | sdg: tur state = up
39299.374156 | sda: udev property ID_WWN whitelisted
39299.374186 | sdb: udev property ID_WWN whitelisted
39299.374211 | sdc: udev property ID_WWN whitelisted
39299.374242 | sdd: udev property ID_WWN whitelisted
39299.374265 | sde: udev property ID_WWN whitelisted
39299.374294 | sdf: udev property ID_WWN whitelisted
39299.374318 | sdg: udev property ID_WWN whitelisted
39299.374352 | sdh: udev property ID_WWN whitelisted
39299.374785 | unloading tur checker
39299.374847 | unloading const prioritizer
root@genestack-compute1:~#