search

libstorage / libstoragemgmt

A library for storage management
https://libstorage.github.io/libstoragemgmt-doc/
GNU Lesser General Public License v2.1
82 stars 32 forks source link

megaraid (storcli) Broadcom HBA is supported? #520

Closed k0ste closed 1 year ago

k0ste commented 1 year ago

Hi, I try to ident the fault drive with lsmcli & megaraid plugin, but

[root@host]# lsmcli plugin-info
Description: LSI MegaRAID Plugin Version: 1.9.1
[root@host]# lsmcli ls -u 'local://'
PLUGIN_BUG(2): Expected key missing from MegaRAID storcli output:'Enable JBOD'
[root@host]# lsmcli ls -u 'local://' --type DISKS
PLUGIN_BUG(2): Expected key missing from MegaRAID storcli output:'Enable JBOD'

storcli is installed and works

{
  "Controllers": [
    {
      "Command Status": {
        "CLI Version": "007.2405.0000.0000 Sep 28, 2022",
        "Operating system": "Linux 4.18.0-365.el8.x86_64",
        "Controller": 0,
        "Status": "Success",
        "Description": "None"
      },
      "Response Data": {
        "Product Name": "PERC H330 Adapter",
        "Serial Number": "01G1658",
        "SAS Address": " 5f402700db026e00",
        "PCI Address": "00:19:00:00",
        "System Time": "01/20/2023 09:49:14",
        "Mfg. Date": "01/20/20",
        "Controller Time": "01/20/2023 02:48:54",
        "FW Package Build": "25.5.9.0001",
        "BIOS Version": "6.33.01.0_4.19.08.00_0x06120304",
        "FW Version": "4.300.01-8369",
        "Driver Name": "megaraid_sas",
        "Driver Version": "07.719.03.00-rh1",
        "Current Personality": "HBA-Mode ",
        "Vendor Id": 4096,
        "Device Id": 95,
        "SubVendor Id": 4136,
        "SubDevice Id": 8004,
        "Host Interface": "PCI-E",
        "Device Interface": "SAS-12G",
        "Bus Number": 25,
        "Device Number": 0,
        "Function Number": 0,
        "Domain ID": 0,
        "Security Protocol": "None",
        "JBOD Drives": 12,
        "JBOD LIST": [
          {
            "EID:Slt": "32:0",
            "DID": 0,
            "State": "JBOD",
            "DG": "-",
            "Size": "5.458 TB",
            "Intf": "SATA",
            "Med": "HDD",
            "SED": "N",
            "PI": "N",
            "SeSz": "512B",
            "Model": "TOSHIBA HDWE160",
            "Sp": "U",
            "Type": "-"
          },
          {
            "EID:Slt": "32:1",
            "DID": 1,
            "State": "JBOD",
            "DG": "-",
            "Size": "5.458 TB",
            "Intf": "SATA",
            "Med": "HDD",
            "SED": "N",
            "PI": "N",
            "SeSz": "512B",
            "Model": "TOSHIBA HDWE160",
            "Sp": "U",
            "Type": "-"
          },
          {
            "EID:Slt": "32:2",
            "DID": 2,
            "State": "JBOD",
            "DG": "-",
            "Size": "5.458 TB",
            "Intf": "SATA",
            "Med": "HDD",
            "SED": "N",
            "PI": "N",
            "SeSz": "512B",
            "Model": "TOSHIBA HDWE160",
            "Sp": "U",
            "Type": "-"
          },
          {
            "EID:Slt": "32:3",
            "DID": 3,
            "State": "JBOD",
            "DG": "-",
            "Size": "5.458 TB",
            "Intf": "SATA",
            "Med": "HDD",
            "SED": "N",
            "PI": "N",
            "SeSz": "512B",
            "Model": "TOSHIBA HDWE160",
            "Sp": "U",
            "Type": "-"
          },
          {
            "EID:Slt": "32:4",
            "DID": 4,
            "State": "JBOD",
            "DG": "-",
            "Size": "5.458 TB",
            "Intf": "SATA",
            "Med": "HDD",
            "SED": "N",
            "PI": "N",
            "SeSz": "512B",
            "Model": "TOSHIBA HDWE160",
            "Sp": "U",
            "Type": "-"
          },
          {
            "EID:Slt": "32:5",
            "DID": 5,
            "State": "JBOD",
            "DG": "-",
            "Size": "5.458 TB",
            "Intf": "SATA",
            "Med": "HDD",
            "SED": "N",
            "PI": "N",
            "SeSz": "512B",
            "Model": "TOSHIBA HDWE160",
            "Sp": "U",
            "Type": "-"
          },
          {
            "EID:Slt": "32:6",
            "DID": 6,
            "State": "JBOD",
            "DG": "-",
            "Size": "5.458 TB",
            "Intf": "SATA",
            "Med": "HDD",
            "SED": "N",
            "PI": "N",
            "SeSz": "512B",
            "Model": "TOSHIBA HDWE160",
            "Sp": "U",
            "Type": "-"
          },
          {
            "EID:Slt": "32:7",
            "DID": 7,
            "State": "JBOD",
            "DG": "-",
            "Size": "5.458 TB",
            "Intf": "SATA",
            "Med": "HDD",
            "SED": "N",
            "PI": "N",
            "SeSz": "512B",
            "Model": "TOSHIBA HDWE160",
            "Sp": "U",
            "Type": "-"
          },
          {
            "EID:Slt": "32:8",
            "DID": 8,
            "State": "JBOD",
            "DG": "-",
            "Size": "5.458 TB",
            "Intf": "SATA",
            "Med": "HDD",
            "SED": "N",
            "PI": "N",
            "SeSz": "512B",
            "Model": "TOSHIBA HDWE160",
            "Sp": "U",
            "Type": "-"
          },
          {
            "EID:Slt": "32:9",
            "DID": 9,
            "State": "JBOD",
            "DG": "-",
            "Size": "5.458 TB",
            "Intf": "SATA",
            "Med": "HDD",
            "SED": "N",
            "PI": "N",
            "SeSz": "512B",
            "Model": "TOSHIBA HDWE160",
            "Sp": "U",
            "Type": "-"
          },
          {
            "EID:Slt": "32:10",
            "DID": 10,
            "State": "JBOD",
            "DG": "-",
            "Size": "5.458 TB",
            "Intf": "SATA",
            "Med": "HDD",
            "SED": "N",
            "PI": "N",
            "SeSz": "512B",
            "Model": "TOSHIBA HDWE160",
            "Sp": "U",
            "Type": "-"
          },
          {
            "EID:Slt": "32:11",
            "DID": 11,
            "State": "JBOD",
            "DG": "-",
            "Size": "5.458 TB",
            "Intf": "SATA",
            "Med": "HDD",
            "SED": "N",
            "PI": "N",
            "SeSz": "512B",
            "Model": "TOSHIBA HDWE160",
            "Sp": "U",
            "Type": "-"
          }
        ],
        "Physical Drives": 12,
        "PD LIST": [
          {
            "EID:Slt": "32:0",
            "DID": 0,
            "State": "JBOD",
            "DG": "-",
            "Size": "5.458 TB",
            "Intf": "SATA",
            "Med": "HDD",
            "SED": "N",
            "PI": "N",
            "SeSz": "512B",
            "Model": "TOSHIBA HDWE160",
            "Sp": "U",
            "Type": "-"
          },
          {
            "EID:Slt": "32:1",
            "DID": 1,
            "State": "JBOD",
            "DG": "-",
            "Size": "5.458 TB",
            "Intf": "SATA",
            "Med": "HDD",
            "SED": "N",
            "PI": "N",
            "SeSz": "512B",
            "Model": "TOSHIBA HDWE160",
            "Sp": "U",
            "Type": "-"
          },
          {
            "EID:Slt": "32:2",
            "DID": 2,
            "State": "JBOD",
            "DG": "-",
            "Size": "5.458 TB",
            "Intf": "SATA",
            "Med": "HDD",
            "SED": "N",
            "PI": "N",
            "SeSz": "512B",
            "Model": "TOSHIBA HDWE160",
            "Sp": "U",
            "Type": "-"
          },
          {
            "EID:Slt": "32:3",
            "DID": 3,
            "State": "JBOD",
            "DG": "-",
            "Size": "5.458 TB",
            "Intf": "SATA",
            "Med": "HDD",
            "SED": "N",
            "PI": "N",
            "SeSz": "512B",
            "Model": "TOSHIBA HDWE160",
            "Sp": "U",
            "Type": "-"
          },
          {
            "EID:Slt": "32:4",
            "DID": 4,
            "State": "JBOD",
            "DG": "-",
            "Size": "5.458 TB",
            "Intf": "SATA",
            "Med": "HDD",
            "SED": "N",
            "PI": "N",
            "SeSz": "512B",
            "Model": "TOSHIBA HDWE160",
            "Sp": "U",
            "Type": "-"
          },
          {
            "EID:Slt": "32:5",
            "DID": 5,
            "State": "JBOD",
            "DG": "-",
            "Size": "5.458 TB",
            "Intf": "SATA",
            "Med": "HDD",
            "SED": "N",
            "PI": "N",
            "SeSz": "512B",
            "Model": "TOSHIBA HDWE160",
            "Sp": "U",
            "Type": "-"
          },
          {
            "EID:Slt": "32:6",
            "DID": 6,
            "State": "JBOD",
            "DG": "-",
            "Size": "5.458 TB",
            "Intf": "SATA",
            "Med": "HDD",
            "SED": "N",
            "PI": "N",
            "SeSz": "512B",
            "Model": "TOSHIBA HDWE160",
            "Sp": "U",
            "Type": "-"
          },
          {
            "EID:Slt": "32:7",
            "DID": 7,
            "State": "JBOD",
            "DG": "-",
            "Size": "5.458 TB",
            "Intf": "SATA",
            "Med": "HDD",
            "SED": "N",
            "PI": "N",
            "SeSz": "512B",
            "Model": "TOSHIBA HDWE160",
            "Sp": "U",
            "Type": "-"
          },
          {
            "EID:Slt": "32:8",
            "DID": 8,
            "State": "JBOD",
            "DG": "-",
            "Size": "5.458 TB",
            "Intf": "SATA",
            "Med": "HDD",
            "SED": "N",
            "PI": "N",
            "SeSz": "512B",
            "Model": "TOSHIBA HDWE160",
            "Sp": "U",
            "Type": "-"
          },
          {
            "EID:Slt": "32:9",
            "DID": 9,
            "State": "JBOD",
            "DG": "-",
            "Size": "5.458 TB",
            "Intf": "SATA",
            "Med": "HDD",
            "SED": "N",
            "PI": "N",
            "SeSz": "512B",
            "Model": "TOSHIBA HDWE160",
            "Sp": "U",
            "Type": "-"
          },
          {
            "EID:Slt": "32:10",
            "DID": 10,
            "State": "JBOD",
            "DG": "-",
            "Size": "5.458 TB",
            "Intf": "SATA",
            "Med": "HDD",
            "SED": "N",
            "PI": "N",
            "SeSz": "512B",
            "Model": "TOSHIBA HDWE160",
            "Sp": "U",
            "Type": "-"
          },
          {
            "EID:Slt": "32:11",
            "DID": 11,
            "State": "JBOD",
            "DG": "-",
            "Size": "5.458 TB",
            "Intf": "SATA",
            "Med": "HDD",
            "SED": "N",
            "PI": "N",
            "SeSz": "512B",
            "Model": "TOSHIBA HDWE160",
            "Sp": "U",
            "Type": "-"
          }
        ],
        "Enclosures": 1,
        "Enclosure LIST": [
          {
            "EID": 32,
            "State": "OK",
            "Slots": 12,
            "PD": 12,
            "PS": 0,
            "Fans": 0,
            "TSs": 0,
            "Alms": 0,
            "SIM": 1,
            "Port#": "00 & 00 x8",
            "ProdID": "BP14G+EXP",
            "VendorSpecific": " "
          }
        ]
      }
    }
  ]
}

Seems in #238 support was added and something wrong with my setup/docs?

tasleson commented 1 year ago

Thanks for reporting this. I'm guessing that the output of the command has changed. I'll update my test system to latest storcli and see if I can reproduce. Would you be willing to test a correction?

k0ste commented 1 year ago

Thanks for reporting this. I'm guessing that the output of the command has changed. I'll update my test system to latest storcli and see if I can reproduce. Would you be willing to test a correction?

With pleasure. Just in case, recently I posted various outputs from another HBA variants, in https://github.com/netdata/netdata/issues/14097 Just tell me if I can help with another commands outputs from prod systems

tasleson commented 1 year ago

Same version of storcli is working on my setup. Can you please provide the full output of:

/opt/hpe/storcli/storcli64 /c0 show all J

Thanks!

k0ste commented 1 year ago

Same version of storcli is working on my setup. Can you please provide the full output of:

/opt/hpe/storcli/storcli64 /c0 show all J

Thanks!

Attached hba.txt

tasleson commented 1 year ago

The current code is checking the "Capabilities" for

"Enable JBOD" : "Yes"

Your capabilities doesn't have this. Maybe we could put in a check for "Status"

"Current Personality": "HBA-Mode "

instead.

I'm wondering what else may not work, and why the output is so different even though we are using the exact same storcli command line tool.

I think we could try something like:

diff --git a/plugin/megaraid_plugin/megaraid.py b/plugin/megaraid_plugin/megaraid.py
index e40ce5d..eee7233 100644
--- a/plugin/megaraid_plugin/megaraid.py
+++ b/plugin/megaraid_plugin/megaraid.py
@@ -532,13 +532,24 @@ def systems(self, flags=Client.FLAG_RSVD):
                 ctrl_show_all_output["Version"]["Bios Version"],
                 ctrl_show_all_output["Version"]["Firmware Version"])

-            if ctrl_show_all_output["Capabilities"]["Enable JBOD"] == "Yes":
-                mode = System.MODE_HBA
+            if "Capabilities" in ctrl_show_all_output and \
+                    "Enable JBOD" in ctrl_show_all_output["Capabilities"]:
+
+                if ctrl_show_all_output["Capabilities"]["Enable JBOD"] == "Yes":
+                    mode = System.MODE_HBA
+                else:
+                    mode = System.MODE_HARDWARE_RAID
+                # Notes for JBOD/HBA mode of MegaRAID:
+                # "storcli /c0/e9/s1 set jbod" require "storcli /c0 set jbod=on"
+                # be executed first.
+            elif "Status" in ctrl_show_all_output and \
+                    "Current Personality" in ctrl_show_all_output["Status"]:
+                if ctrl_show_all_output["Status"]["Current Personality"] == "HBA-Mode ":
+                    mode = System.MODE_HBA
+                else:
+                    mode = System.MODE_HBA
             else:
-                mode = System.MODE_HARDWARE_RAID
-            # Notes for JBOD/HBA mode of MegaRAID:
-            # "storcli /c0/e9/s1 set jbod" require "storcli /c0 set jbod=on"
-            # be excuted first.
+                mode = System.MODE_UNKNOWN

             rc_lsm_syss.append(
                 System(sys_id,

Please let me know what works best for you to test this? I could probably build some rpm packages if that is helpful.

Thanks!

tasleson commented 1 year ago

@k0ste There are some test rpms you can use if one works for you with the diff listed above.

https://copr.fedorainfracloud.org/coprs/packit/libstorage-libstoragemgmt-521/build/5289288/

The RHEL7 build didn't work because the spec file being used isn't correct for py2 support, but I believe you're using an EL8 based install.

k0ste commented 1 year ago

@k0ste There are some test rpms you can use if one works for you with the diff listed above.

https://copr.fedorainfracloud.org/coprs/packit/libstorage-libstoragemgmt-521/build/5289288/

The RHEL7 build didn't work because the spec file being used isn't correct for py2 support, but I believe you're using an EL8 based install.

Thanks for copr build!

Now the tests:

[root@host]# lsmcli ls -u 'local://'
ID      | Name                                | Status | Info | FW Ver                                                                         | Mode | Read Cache Percentage
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
01G1658 | PERC H330 Adapter PCI-E 00:19:00:00 | OK     |      | Package: 25.5.9.0001, BIOS: 6.33.01.0_4.19.08.00_0x06120304, FW: 4.300.01-8369 | HBA  |
[root@host]# lsmcli ls -u 'local://' --type DISKS
ID           | Name                            | Type | Size           | Status | System ID | SCSI VPD 0x83    | Disk Paths | Revolutions Per Minute        | Link Type
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------
49P3K0OTFB8G | Disk 0 ATA TOSHIBA HDWE160      | SATA | 6000606183424  | OK     | 01G1658   | 500003993b802016 | /dev/sda   | Rotating Medium Unknown Speed | PATA/SATA
49PPK14AFB8G | Disk 1 ATA TOSHIBA HDWE160      | SATA | 6000606183424  | OK     | 01G1658   | 500003993c301f28 | /dev/sdb   | Rotating Medium Unknown Speed | PATA/SATA
49POK0VSFB8G | Disk 2 ATA TOSHIBA HDWE160      | SATA | 6000606183424  | OK     | 01G1658   | 500003993c28207f | /dev/sdc   | Rotating Medium Unknown Speed | PATA/SATA
VCK18M8P     | Disk 3 ATA WDC  WUS721010ALE6L4 | SATA | 10000294477824 | OK     | 01G1658   | 5000cca0b0eadb3f | /dev/sdd   | Rotating Medium Unknown Speed | PATA/SATA
49PTK0SUFB8G | Disk 4 ATA TOSHIBA HDWE160      | SATA | 6000606183424  | OK     | 01G1658   | 500003993c501639 | /dev/sde   | Rotating Medium Unknown Speed | PATA/SATA
49PPK14DFB8G | Disk 5 ATA TOSHIBA HDWE160      | SATA | 6000606183424  | OK     | 01G1658   | 500003993c301f2b | /dev/sdf   | Rotating Medium Unknown Speed | PATA/SATA
49PPK15TFB8G | Disk 6 ATA TOSHIBA HDWE160      | SATA | 6000606183424  | OK     | 01G1658   | 500003993c301f7c | /dev/sdg   | Rotating Medium Unknown Speed | PATA/SATA
49P5K0UOFB8G | Disk 7 ATA TOSHIBA HDWE160      | SATA | 6000606183424  | OK     | 01G1658   | 500003993b9028d6 | /dev/sdh   | Rotating Medium Unknown Speed | PATA/SATA
49P4K0NJFB8G | Disk 8 ATA TOSHIBA HDWE160      | SATA | 6000606183424  | OK     | 01G1658   | 500003993b881887 | /dev/sdi   | Rotating Medium Unknown Speed | PATA/SATA
49PQK11FFB8G | Disk 9 ATA TOSHIBA HDWE160      | SATA | 6000606183424  | OK     | 01G1658   | 500003993c3818fc | /dev/sdj   | Rotating Medium Unknown Speed | PATA/SATA
49POK0VVFB8G | Disk 10 ATA TOSHIBA HDWE160     | SATA | 6000606183424  | OK     | 01G1658   | 500003993c282082 | /dev/sdk   | Rotating Medium Unknown Speed | PATA/SATA
49P4K0OGFB8G | Disk 11 ATA TOSHIBA HDWE160     | SATA | 6000606183424  | OK     | 01G1658   | 500003993b881928 | /dev/sdl   | Rotating Medium Unknown Speed | PATA/SATA

Seems now it works with Dell PowerEdge systems 🙂

tasleson commented 1 year ago

@k0ste Thanks for testing. I'll merge this change and get it integrated into a release. If you have the time, please also check out other parts, to see if they work too. You mentioned originally that you're interested in changing LEDs. The megaraid plugin doesn't support this currently. See https://github.com/libstorage/libstoragemgmt/issues/479 for more discussion.

Update: I see you already commented on that issue

k0ste commented 1 year ago

Thanks @tasleson, if you tag new release I will create new BZ for Red Hat

Also, if you need any test with hardware at #522 please tag me. Also @napaster was very interested in LED control

braindevices commented 6 months ago

VfBServer is an example