vmware / govmomi

Go library for the VMware vSphere API
Apache License 2.0
2.31k stars 910 forks source link

Fibre Channel World Wide Names reported in decimal #1811

Closed scdc-galvin closed 3 years ago

scdc-galvin commented 4 years ago
govc host.info -json -dc="DCNAME" -host ESXHOSTNAME

The above command will output HBA information for a particular ESXi host. The standard HBA in our environment is the Emulex LPe32000. When the above command is run, the PortWorldWideName and NodeWorldWideName are returned as decimal numbers. An abbreviated example follows:

{
  "HostSystems": [
    {
      "Config": {
        "StorageDevice": {
          "HostBusAdapter": [
            {
              "Key": "key-vim.host.FibreChannelHba-vmhba2",
              "Device": "vmhba2",
              "Bus": 59,
              "Status": "online",
              "Model": "Emulex LightPulse LPe32000 PCIe Fibre Channel Adapter",
              "Driver": "lpfc",
              "Pci": "0000:3b:00.0",
              "PortWorldWideName": 1152922127287604726,
              "NodeWorldWideName": 2305843631894451702,
              "PortType": "unknown",
              "Speed": 16
            },
            {
              "Key": "key-vim.host.FibreChannelHba-vmhba3",
              "Device": "vmhba3",
              "Bus": 95,
              "Status": "online",
              "Model": "Emulex LightPulse LPe32000 PCIe Fibre Channel Adapter",
              "Driver": "lpfc",
              "Pci": "0000:5f:00.0",
              "PortWorldWideName": 1152922127287604554,
              "NodeWorldWideName": 2305843631894451530,
              "PortType": "unknown",
              "Speed": 16
            }
          ]
        }
      }
    }
  ]
}

So two things:

  1. Shouldn't world wide names be reported in hex, not decimal? (The WWN for the PERC H330 Adapter in these servers are reported in hex)
  2. Aren't decimal values this long prone to overflow? When the above json is used with jq as shown in the below govc command, jq replaces the 2 least significant digits of each WWN with zeros...
    govc host.info -json -dc="DCNAME" -host ESXHOSTNAME | jq -r '.HostSystems[].Config.StorageDevice.HostBusAdapter[]'
dougm commented 4 years ago

Haven't look at these fields myself before, but those values are in their raw form (xsd:long) as returned by the public API: https://code.vmware.com/apis/196/vsphere/doc/vim.host.FibreChannelHba.html

Which interface are you referring to that would report them in hex?

scdc-galvin commented 4 years ago

Hi @dougm,

My experience in this area is with management tools, not APIs or data specifications.

This is actually the first time I have ever seen a world wide name not presented as a hex string. In the example I posted, I pared down the output to just the fibre channel HBAs for the sake of clarity. That same command actually returned the following HBA entry for the embedded SAS RAID controller:

{
  "Key": "key-vim.host.SerialAttachedHba-vmhba4",
  "Device": "vmhba4",
  "Bus": 24,
  "Status": "unknown",
  "Model": "PERC H330 Adapter",
  "Driver": "lsi_mr3",
  "Pci": "0000:18:00.0",
  "NodeWorldWideName": "5d0946606e78ac00"
}

Admittedly not a fibre channel HBA, but that hex string presentation for the world wide name is one of the two ways I have always seen it done before. The other is with a colon (:) after every two hex characters in the string.

A world wide name is the storage world's version of a MAC address, and like a MAC address the customary presentation is a hex string.

Both the Flash and HTML5 based vSphere Clients display fibre channel adapter information like this:

Name:  vmhba2
Model: Emulex LightPulse LPe32000 PCIe Fibre Channel Adapter
WWNN:  20:00:00:90:fa:aa:55:f6
WWPN:  10:00:00:90:fa:aa:55:f6

The VMware esxcli --server ESXHOSTNAME storage san fc listcommand displays:

   Adapter: vmhba2
   Port ID: CE1C00
   Node Name: 20:00:00:90:fa:aa:55:f6
   Port Name: 10:00:00:90:fa:aa:55:f6
   Speed: 16 Gbps
   Port Type: NPort
   Port State: ONLINE
   Model Description: Emulex LightPulse LPe31000-M6-D 1-Port 16Gb Fibre Channel Adapter
   Hardware Version: 0000000c
   OptionROM Version: 12.2.350.10
   Firmware Version: 12.2.350.10
   Driver Name: lpfc
   DriverVersion: 11.4.33.25

A cisco mds9148s SAN switch sh flogi database command shows:

--------------------------------------------------------------------------------
INTERFACE        VSAN    FCID           PORT NAME               NODE NAME       
--------------------------------------------------------------------------------
fc1/4            10    0xce1c00  10:00:00:90:fa:aa:55:f6 20:00:00:90:fa:aa:55:f6

Configuring a zone on a cisco mds9148s SAN switch looks like this:

fcalias name ESXHOSTNAME_HBA1 vsan 10
    member pwwn 10:00:00:90:fa:aa:55:f6

fcalias name SANFRAME_M1P3 vsan 10
    member pwwn 50:01:73:80:67:86:01:12

zone name ESXHOSTNAME_HBA1_SANFRAME_M1P3 vsan 10
    member fcalias ESXHOSTNAME_HBA1
    member fcalias SANFRAME_M1P3

Dell iDRAC remote server administration tool displays fibre channel HBAs like this:

Fibre Channel in Slot 1-1
  Link Connection
    Link Status: Up
    Link Speed:  16 Gbps
  World Wide Names
    World Wide Node Name: 20:00:00:90:FA:AA:55:F6
    World Wide Port Name: 10:00:00:90:FA:AA:55:F6

I would contend that a decimal world wide name presentation is inherently far less useful than a hex string presentation (with or without colons). I would also contend that the size of that decimal number (it would require an 64 bit long type to store correctly) may invite conversion errors.

Also, as I alluded to in my original post, if you attempt to use jq (jq-1.5-1-a5b5cbe on Ubuntu 19.10) to pretty print the output from govc host.info you will find that the decimal world wide presentations are corrupted, while the hex string one for something like the PERC H330 is not. Obviously this is more of a jq bug dealing with large decimal numbers than a problem with govc

dougm commented 4 years ago

@scdc-galvin understood, there are plenty of other govc commands that output a formatted version of data returned by the API. This one is just new to me. For example, we could add WWNs to the govc host.storage.info command in hex encoded format. Some of the HBA data is there already:

% govc host.storage.info -t hba -host 10.92.66.76                                                                                     
Device   PCI           Driver  Status   Model                                    
vmhba0   0000:03:00.0  pvscsi  unknown  PVSCSI SCSI Controller                   
vmhba1   0000:00:07.1  vmkata  unknown  PIIX4 for 430TX/440BX/MX IDE Controller  
vmhba64  0000:00:07.1  vmkata  unknown  PIIX4 for 430TX/440BX/MX IDE Controller  

Also note you can use govc object.collect to get a subset of the host data:

% govc object.collect -s -json $(govc find / -type h -name 10.92.66.76) config.storageDevice.hostBusAdapter                                                                                                                                                                         
[
  {
    "Name": "config.storageDevice.hostBusAdapter",
    "Op": "assign",
    "Val": {
      "HostHostBusAdapter": [
        {
          "Key": "key-vim.host.ParallelScsiHba-vmhba0",
          "Device": "vmhba0",
          "Bus": 3,
          "Status": "unknown",
          "Model": "PVSCSI SCSI Controller",
          "Driver": "pvscsi",
          "Pci": "0000:03:00.0"
        },
        {
          "Key": "key-vim.host.BlockHba-vmhba1",
          "Device": "vmhba1",
          "Bus": 0,
          "Status": "unknown",
          "Model": "PIIX4 for 430TX/440BX/MX IDE Controller",
          "Driver": "vmkata",
          "Pci": "0000:00:07.1"
        },
        {
          "Key": "key-vim.host.BlockHba-vmhba64",
          "Device": "vmhba64",
          "Bus": 0,
          "Status": "unknown",
          "Model": "PIIX4 for 430TX/440BX/MX IDE Controller",
          "Driver": "vmkata",
          "Pci": "0000:00:07.1"
        }
      ]
    }
  }
]
dougm commented 4 years ago

@scdc-galvin also you could run the command above and change -json to -dump, that will output Go code with the WWNs. If you can paste that here, we can use it vcsim to implement and test the suggested change.

scdc-galvin commented 4 years ago

Hi @dougm,

Thanks for the govc object.collect example. I had not looked at that before. It does look like a workaround for the jq world wide name corruption problem. The -dump output in my environment is as follows:

[]types.BaseHostHostBusAdapter{
    &types.HostBlockHba{
        HostHostBusAdapter: types.HostHostBusAdapter{
            Key:    "key-vim.host.BlockHba-vmhba0",
            Device: "vmhba0",
            Bus:    0,
            Status: "unknown",
            Model:  "Lewisburg SATA AHCI Controller",
            Driver: "vmw_ahci",
            Pci:    "0000:00:11.5",
        },
    },
    &types.HostBlockHba{
        HostHostBusAdapter: types.HostHostBusAdapter{
            Key:    "key-vim.host.BlockHba-vmhba1",
            Device: "vmhba1",
            Bus:    0,
            Status: "unknown",
            Model:  "Lewisburg SATA AHCI Controller",
            Driver: "vmw_ahci",
            Pci:    "0000:00:17.0",
        },
    },
    &types.HostFibreChannelHba{
        HostHostBusAdapter: types.HostHostBusAdapter{
            Key:    "key-vim.host.FibreChannelHba-vmhba2",
            Device: "vmhba2",
            Bus:    59,
            Status: "online",
            Model:  "Emulex LightPulse LPe32000 PCIe Fibre Channel Adapter",
            Driver: "lpfc",
            Pci:    "0000:3b:00.0",
        },
        PortWorldWideName: 1152922127287604726,
        NodeWorldWideName: 2305843631894451702,
        PortType:          "unknown",
        Speed:             16,
    },
    &types.HostFibreChannelHba{
        HostHostBusAdapter: types.HostHostBusAdapter{
            Key:    "key-vim.host.FibreChannelHba-vmhba3",
            Device: "vmhba3",
            Bus:    95,
            Status: "online",
            Model:  "Emulex LightPulse LPe32000 PCIe Fibre Channel Adapter",
            Driver: "lpfc",
            Pci:    "0000:5f:00.0",
        },
        PortWorldWideName: 1152922127287604554,
        NodeWorldWideName: 2305843631894451530,
        PortType:          "unknown",
        Speed:             16,
    },
    &types.HostSerialAttachedHba{
        HostHostBusAdapter: types.HostHostBusAdapter{
            Key:    "key-vim.host.SerialAttachedHba-vmhba4",
            Device: "vmhba4",
            Bus:    24,
            Status: "unknown",
            Model:  "PERC H330 Adapter",
            Driver: "lsi_mr3",
            Pci:    "0000:18:00.0",
        },
        NodeWorldWideName: "5d0946606e78ac00",
    },
}

Rather than making a change that might break other people's scripts, would it be possible to just add two additional fields to the hba information containing the hex string version of the port world wide name and the node world wide name?

I dug around a bit over the weekend and found that I could use:

govc host.esxcli -json -dc="DC" -host="ESXHOSTNAME" storage san fc list | jq -r '.Values[] | { Adapter: .Adapter[0], NodeName: .NodeName[0], PortName: .PortName[0] }

That returns:

{
  "Adapter": "vmhba2",
  "NodeName": "20:00:00:90:fa:aa:55:f6",
  "PortName": "10:00:00:90:fa:aa:55:f6"
}
{
  "Adapter": "vmhba3",
  "NodeName": "20:00:00:90:fa:aa:55:4a",
  "PortName": "10:00:00:90:fa:aa:55:4a"
}

Seems like a linux only sort of solution, and then I have to insert that data into the per host data (govc host.info) I am using in my scripts, so not ideal, but doable .

dougm commented 4 years ago

Thanks for the data, that is useful. Changing the json output of host.info is tricky, changing host.storage.info would be less so. Another option would be implementing #1163 , with that we could both avoid jq for simple cases and apply formatting (such as wwn) to certain fields in a more general way.

scdc-galvin commented 4 years ago

Either (or both) of those solutions would be fine with me. The Go templates route seems like it would be more flexible and have greater long-term benefits.

github-actions[bot] commented 3 years ago

This issue is stale because it has been open for 90 days with no activity. It will automatically close after 30 more days of inactivity. Mark as fresh by adding the comment /remove-lifecycle stale.