search

HewlettPackard / PowerShell-ProLiant-SDK

PowerShell sample scripts for managing HPE servers
https://www.hpe.com/servers/powershell
90 stars 42 forks source link

New-HPESALogicalDrive Problem #57

Closed ariael closed 3 years ago

ariael commented 3 years ago

Hi, I have some problem and i can't found solution:

PS D:\MGMTTools>Set-HP_ConnectionSA

Connecting to server  : 2.252.64.201
Connection to Smart Array server is OK

IP                               : 2.252.64.201
Hostname                         : czcws658-r.prg.com
Location                         : https://2.252.64.201/redfish/v1/SessionService/Sessions/administrator000000005ffd26803ba5e354/
Uri                              : https://2.252.64.201/redfish/v1/Sessions/
DisableCertificateAuthentication : True
IsConnected                      : True
ConnectionType                   : Redfish
Username                         : Administrator
ServerName                       : ProLiant DL360 Gen10
ServerGeneration                 : GEN10

Connection to SA work well.

There is some variables: $connectionSA - standard conector SA $locationSA - Slot 0 $Raid_type - Raid 1 $selecteddrive - 1I:1:1 1I:1:2

iLO: 2.14 Feb 11 2020 BIOS: U32 v2.34 (04/08/2020)

SmartArray status result:
IP                  : 2.252.64.201
Hostname            : czcws658-r.prg.com
Status              : Warning
ConfigurationStatus : {@{ControllerLocation=Slot 0; ControllerModel=HPE Smart Array P408i-a SR Gen10; ETag=; Severity=; MessageId=Base.1.0.Success; Message=; Description=; MessageArgs=; RelatedProperty=; Resolution=}}
StatusInfo          : @{Category=Unknown; Message=Unable to process the MessageId.}

And there is attempt to create Logical Drive. This is normally used code, but after some time we need build few hundreds servers and ALL of this have same problem!

Do you have any clue about this problem? p.s. And do you known some date for release fix for iLO 2.30+ cmdlets?

Thanks Peter


New-HPESALogicalDrive -Connection $connectionSA -ControllerLocation $locationSA -Raid $Raid_type -DataDrive $selecteddrive -Verbose

VERBOSE: Performing the operation "New-HPESALogicalDrive" on target "IP: 2.252.64.201".
VERBOSE: [New-HPESALogicalDrive] Executing the cmdlets with 1 task serially.
VERBOSE: Retrieving SmartArray controller location.
VERBOSE: SmartArray controller location details retrieved successfully.
VERBOSE: Reteriving smart array controller data through iLO RIS.
New-HPESALogicalDrive : Failed for 2.252.64.201: The remote server returned an error: (403) Forbidden.
At line:1 char:17
+ ...             New-HPESALogicalDrive -Connection $script:connectionSA -C ...
+                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ConnectionError: (:) [New-HPESALogicalDrive], Exception
    + FullyQualifiedErrorId : HPE.SA.Cmdlet.NewHPESALogicalDrive
hp-powershell commented 3 years ago

Hi ,

Thanks for writing to us .

Kindly collect the log for the SA error by enabling the enable-HPESALog cmdlet and provide us the log file.

Also let us know , whether the issue is seen on only this server or with all the servers with the p408i controller .

Regarding the iLOCmdlets 3.1 version with FW 2.30 support is scheduled to be released in 2 weeks.You should be able to use it by this month end .

Thanks & Regards, Powershell Team

ariael commented 3 years ago 
2021-01-13 09:51:17,833 HPE.Framework.Logging.Logger.LogMessage INFO - [Connect-HPESA] Parameter processing started. 
2021-01-13 09:51:17,856 HPE.Framework.Logging.Logger.LogMessage INFO - [Connect-HPESA] Processing parameter - IP.
2021-01-13 09:51:17,857 HPE.Framework.Logging.Logger.LogMessage INFO - [Connect-HPESA] Processing parameter - Username.
2021-01-13 09:51:17,858 HPE.Framework.Logging.Logger.LogMessage INFO - [Connect-HPESA] Processing parameter - Password.
2021-01-13 09:51:17,860 HPE.Framework.Logging.Logger.LogMessage INFO - [Connect-HPESA] Processing parameter - Credential.
2021-01-13 09:51:17,861 HPE.Framework.Logging.Logger.LogMessage INFO - [Connect-HPESA] Processing parameter - Timeout.
2021-01-13 09:51:17,863 HPE.Framework.Logging.Logger.LogMessage INFO - [Connect-HPESA] Processing parameter - DisableCertificateAuthentication.
2021-01-13 09:51:17,870 HPE.Framework.Logging.Logger.LogMessage INFO - [Connect-HPESA] Executing the cmdlets with 1 task serially.
2021-01-13 09:51:17,872 HPE.Framework.Logging.Logger.LogMessage DEBUG - Connect-HPESA ProcessCommand() --- Starts.
2021-01-13 09:51:22,702 HPE.Framework.Logging.Logger.LogMessage DEBUG - IP 2.252.64.201 : BuildCommand() execution starts.
2021-01-13 09:51:22,706 HPE.Framework.Logging.Logger.LogMessage DEBUG - IP 2.252.64.201 : Executing Connect-HPESA Cmdlet to  the target server
2021-01-13 09:51:22,708 HPE.Framework.Logging.Logger.LogMessage DEBUG - IP 2.252.64.201 : EstablishConnection() execution starts.
2021-01-13 09:51:23,130 HPE.Framework.Logging.Logger.LogMessage DEBUG - IP 2.252.64.201 : User session created.
2021-01-13 09:51:23,134 HPE.Framework.Logging.Logger.LogMessage DEBUG - IP 2.252.64.201 : InitializeSmartArrayDetails() execution starts.
2021-01-13 09:51:23,135 HPE.Framework.Logging.Logger.LogMessage DEBUG - IP 2.252.64.201 : GetiLORISSmartArrayData() execution starts.
2021-01-13 09:51:23,357 HPE.Framework.Logging.Logger.LogMessage DEBUG - IP 2.252.64.201 : GetiLORISSmartArrayData() execution ends.
2021-01-13 09:51:23,359 HPE.Framework.Logging.Logger.LogMessage DEBUG - IP 2.252.64.201 : HPE Smart Array P408i-a SR Gen10 added to smartArray list
2021-01-13 09:51:23,361 HPE.Framework.Logging.Logger.LogMessage DEBUG - IP 2.252.64.201 : InitializeSmartArrayDetails() execution ends.
2021-01-13 09:51:23,362 HPE.Framework.Logging.Logger.LogMessage DEBUG - IP 2.252.64.201 : EstablishConnection() execution ends.
2021-01-13 09:51:23,363 HPE.Framework.Logging.Logger.LogMessage DEBUG - Connect-HPESA ProcessCommand() --- Starts.
2021-01-13 09:51:32,365 HPE.Framework.Logging.Logger.LogMessage INFO - [New-HPESALogicalDrive] Parameter processing started. 
2021-01-13 09:51:32,367 HPE.Framework.Logging.Logger.LogMessage INFO - [New-HPESALogicalDrive] Processing parameter - Connection.
2021-01-13 09:51:32,369 HPE.Framework.Logging.Logger.LogMessage INFO - [New-HPESALogicalDrive] Processing parameter - ControllerLocation.
2021-01-13 09:51:32,374 HPE.Framework.Logging.Logger.LogMessage INFO - [New-HPESALogicalDrive] Processing parameter - LogicalDriveName.
2021-01-13 09:51:32,375 HPE.Framework.Logging.Logger.LogMessage INFO - [New-HPESALogicalDrive] Processing parameter - Raid.
2021-01-13 09:51:32,376 HPE.Framework.Logging.Logger.LogMessage INFO - [New-HPESALogicalDrive] Processing parameter - CapacityGiB.
2021-01-13 09:51:32,377 HPE.Framework.Logging.Logger.LogMessage INFO - [New-HPESALogicalDrive] Processing parameter - CapacityBlocks.
2021-01-13 09:51:32,378 HPE.Framework.Logging.Logger.LogMessage INFO - [New-HPESALogicalDrive] Processing parameter - DataDrive.
2021-01-13 09:51:32,380 HPE.Framework.Logging.Logger.LogMessage INFO - [New-HPESALogicalDrive] Processing parameter - LegacyBootPriority.
2021-01-13 09:51:32,381 HPE.Framework.Logging.Logger.LogMessage INFO - [New-HPESALogicalDrive] Processing parameter - SpareRebuildMode.
2021-01-13 09:51:32,382 HPE.Framework.Logging.Logger.LogMessage INFO - [New-HPESALogicalDrive] Processing parameter - SpareDrive.
2021-01-13 09:51:32,383 HPE.Framework.Logging.Logger.LogMessage INFO - [New-HPESALogicalDrive] Processing parameter - ParityGroupCount.
2021-01-13 09:51:32,385 HPE.Framework.Logging.Logger.LogMessage INFO - [New-HPESALogicalDrive] Processing parameter - ParityInitializationType.
2021-01-13 09:51:32,386 HPE.Framework.Logging.Logger.LogMessage INFO - [New-HPESALogicalDrive] Processing parameter - StripSizeByte.
2021-01-13 09:51:32,387 HPE.Framework.Logging.Logger.LogMessage INFO - [New-HPESALogicalDrive] Processing parameter - StripeSizeByte.
2021-01-13 09:51:32,388 HPE.Framework.Logging.Logger.LogMessage INFO - [New-HPESALogicalDrive] Processing parameter - Accelerator.
2021-01-13 09:51:32,393 HPE.Framework.Logging.Logger.LogMessage INFO - [New-HPESALogicalDrive] Executing the cmdlets with 1 task serially.
2021-01-13 09:51:32,405 HPE.Framework.Logging.Logger.LogMessage DEBUG - New-HPESALogicalDrive ProcessCommand() --- Starts.
2021-01-13 09:51:32,406 HPE.Framework.Logging.Logger.LogMessage DEBUG - IP 2.252.64.201 : BuildCommand() execution starts.
2021-01-13 09:51:32,408 HPE.Framework.Logging.Logger.LogMessage DEBUG - IP 2.252.64.201 : InitializeRedfishConnection() execution starts.
2021-01-13 09:51:32,410 HPE.Framework.Logging.Logger.LogMessage DEBUG - IP 2.252.64.201 : InitializeRedfishConnection() execution ends.
2021-01-13 09:51:32,411 HPE.Framework.Logging.Logger.LogMessage DEBUG - IP 2.252.64.201 : Executing New-HPESALogicalDrive Cmdlet to  the target server
2021-01-13 09:51:32,412 HPE.Framework.Logging.Logger.LogMessage DEBUG - IP 2.252.64.201 : GetUEFIRISSmartArrayData() execution starts.
2021-01-13 09:51:32,413 HPE.Framework.Logging.Logger.LogMessage DEBUG - IP 2.252.64.201 : GetControllerLocation() Execution starts.
2021-01-13 09:51:33,256 HPE.Framework.Logging.Logger.LogMessage DEBUG - IP 2.252.64.201 : GetControllerLocation() Execution ends.
2021-01-13 09:51:33,510 HPE.Framework.Logging.Logger.LogMessage DEBUG - IP 2.252.64.201 : ValidateLogicalDriveParameter() Execution starts.
2021-01-13 09:51:33,512 HPE.Framework.Logging.Logger.LogMessage DEBUG - IP 2.252.64.201 : GetPhysicalDrive() Execution starts.
2021-01-13 09:51:33,514 HPE.Framework.Logging.Logger.LogMessage DEBUG - IP 2.252.64.201 : GetiLORISSmartArrayData() execution starts.
2021-01-13 09:51:33,758 HPE.Framework.Logging.Logger.LogMessage DEBUG - IP 2.252.64.201 : ProcessiLORISInnerOdataId() execution starts.
2021-01-13 09:51:33,989 HPE.Framework.Logging.Logger.LogMessage DEBUG - IP 2.252.64.201 : AppendControllerDetails() Execution starts.
2021-01-13 09:51:33,991 HPE.Framework.Logging.Logger.LogMessage DEBUG - IP 2.252.64.201 : AppendControllerDetails() Execution ends.
2021-01-13 09:51:33,992 HPE.Framework.Logging.Logger.LogMessage DEBUG - IP 2.252.64.201 : AppendControllerDetails() Execution starts.
2021-01-13 09:51:33,994 HPE.Framework.Logging.Logger.LogMessage DEBUG - IP 2.252.64.201 : AppendControllerDetails() Execution ends.
2021-01-13 09:51:33,995 HPE.Framework.Logging.Logger.LogMessage DEBUG - IP 2.252.64.201 : AppendControllerDetails() Execution starts.
2021-01-13 09:51:33,996 HPE.Framework.Logging.Logger.LogMessage DEBUG - IP 2.252.64.201 : AppendControllerDetails() Execution ends.
2021-01-13 09:51:33,997 HPE.Framework.Logging.Logger.LogMessage DEBUG - IP 2.252.64.201 : AppendControllerDetails() Execution starts.
2021-01-13 09:51:33,999 HPE.Framework.Logging.Logger.LogMessage DEBUG - IP 2.252.64.201 : AppendControllerDetails() Execution ends.
2021-01-13 09:51:34,000 HPE.Framework.Logging.Logger.LogMessage DEBUG - IP 2.252.64.201 : ProcessiLORISInnerOdataId() execution ends.
2021-01-13 09:51:34,001 HPE.Framework.Logging.Logger.LogMessage DEBUG - IP 2.252.64.201 : GetiLORISSmartArrayData() execution ends.
2021-01-13 09:51:34,002 HPE.Framework.Logging.Logger.LogMessage DEBUG - IP 2.252.64.201 : GetPhysicalDrive() Execution ends.
2021-01-13 09:51:34,004 HPE.Framework.Logging.Logger.LogMessage DEBUG - IP 2.252.64.201 : ValidateLogicalDriveParameter() Execution ends.
2021-01-13 09:51:34,231 HPE.Framework.Logging.Logger.LogMessage ERROR - Failed for 2.252.64.201: The remote server returned an error: (403) Forbidden.
ariael commented 3 years ago

Hi... I try this on 9 servers from one badge - same configuration iLO/BIOS/HW, standard Gen 10 model, 4 HDD, i try create 2 standard logical drives

hp-powershell commented 3 years ago

Thanks for sharing the log with us . Would like to check with you whether the FW version in servers are 2.30 or below . Kindly let us know .

Thanks & Regards, Powershell Team

ariael commented 3 years ago

i provide this info before: iLO: 2.14

hp-powershell commented 3 years ago

Hi ,

We checked the scenario you have provided in our environment with same configurations as you have provided us ,but we are not able to reproduce in our environment . We are able to create the logical drives without any error and server is not refusing the connection .

The log you have sent us hints at server actively refusing the creation .

Kindly let us know ,Has this operation worked on same server in past ?

Thanks & Regards, Powershell Team

ariael commented 3 years ago

Hi, Same server as "really same" - no. These server is new delivered from HP. But still configuration is same as all previous delivers. Same model, type and scripts usage - yes, we use my script daily for HW deployment preparation...

And this is source of my confuse - i can't found reason of this change. I check it before few weeks and all works. I suspect some problem with iLO 2.3+, but problem continue on older iLO too (but this is same piece of information - all this serves was iLO 2.30+ and did downgrade). I check firmware of raid controllers, restart all components - and now i have few dozen servers, where i have to create logical drive manually. And rest of comandlets works!

hp-powershell commented 3 years ago

Hi ,

We tried to replicate this scenario too , where the iLO FW is 2.30 and we downgraded it to 2.19 first . Tried creation of logical drive on P408 it worked without error. Post which we downgraded it further to 2.14 , even on that we could successfully create logical drive .

we are unable to reproduce and capture "Forbidden" exception. The creation of logical drives are going smoothly in the setups we tried. This error code as i mentioned earlier is from server directly.

May we know the P408 controller FW version in your server ?

We checked with the Smart Array team and they suspect if the controller FW is latest then drive locations given may not be the right locations which allows creation . Kindly reconfirm same.

Thanks & Regards, Powershell team

ariael commented 3 years ago

Hi... Firmware inside raid controller was check - it is same version as all other server (which work).

This is detected configuration: HPE Smart Array P408i-a SR Gen10 Position of RAID controler in server is: Slot 0

List all physical drives in server:

CapacityGB Location MediaType Model DiskDriveUse EncryptedDrive

300 1I:1:1 HDD EG000300JWEBF Raw false 300 1I:1:2 HDD EG000300JWEBF Raw false 300 1I:1:3 HDD EG000300JWEBF Raw false 300 1I:1:4 HDD EG000300JWEBF Raw false

Controller Status | OK Serial Number | PWXKV0BRHE381Z Model | HPE Smart Array P408i-a SR Gen10 Firmware Version | 2.65 Controller Type | HPE Smart Array Encryption Status | Not Enabled Encryption ASIC Status | OK Encryption Critical Security Parameter NVRAM Status | OK

Embedded ALOM | HP Ethernet 1Gb 4-port 331FLR Adapter | 00 | 20.14.62 | Enabled Embedded Device | HPE Smart Storage Battery | 01 | 0.70 | Enabled Embedded Device | Embedded Video Controller |   | 2.5 | Enabled Embedded LOM | HPE Ethernet 1Gb 4-port 331i Adapter - NIC |   | 20.14.62 | Enabled Embedded RAID | HPE Smart Array P408i-a SR Gen10 | B | 2.65 | Enabled

Drive HPD4 Embedded controller Port 1I Box 1 Bay 1 Drive HPD4 Embedded controller Port 1I Box 1 Bay 2 Drive HPD4 Embedded controller Port 1I Box 1 Bay 3 Drive HPD4 Embedded controller Port 1I Box 1 Bay 4 Embedded Video Controller 2.5 Embedded Device HP Ethernet 1Gb 4-port 331FLR Adapter 20.14.62 Embedded ALOM
HPE Ethernet 1Gb 4-port 331i Adapter - NIC 20.14.62 Embedded LOM
HPE Smart Array P408i-a SR Gen10 2.65 Embedded RAID
iLO 5 2.14 Feb 11 2020 System Board
Innovation Engine (IE) Firmware 0.2.2.0 System Board
Intelligent Platform Abstraction Data 12.2.0 Build 16 System Board
Intelligent Provisioning 3.46.4 System Board
Power Management Controller Firmware 1.0.7 System Board
Power Management Controller FW Bootloader 1.1 System Board
Redundant System ROM U32 v2.36 (07/16/2020) System Board
Server Platform Services (SPS) Firmware 4.1.4.339 System Board
Smart Storage Energy Pack 0.70 Energy Pack 1
System Programmable Logic Device 0x31 System Board
System ROM U32 v2.34 (04/08/2020) System Board
TPM Firmware 73.20 System Board

hp-powershell commented 3 years ago

Hi ,

May we know which user is being used for executing the cmdlets .

is it local user or a directory user .

Also whether the issue is seen on latest FW of controller which is 3.00 for snap 3.

Thanks & Regards, Powershell team

ariael commented 3 years ago

Hi All scripts run under local Administrator account... New firmware controller - we don't have tested this, and this servers is in production already (volume was created manually). I can try found some replacement with same error and try new firmware