ateodorescu / home-assistant-ipmi

IPMI connector for Home Assistant
MIT License
36 stars 8 forks source link

Connect To ILO 4 #21

Closed Redblockmasteur closed 3 months ago

Redblockmasteur commented 8 months ago

Hi, Thanks for the works, i connected my IDRAC of my dell R530 and it work great, however when i try to connect to my PH ILO, it fails. Logs from the plugin logs from ilo says that it had a login and a logout. I tried and recommend this tool IPMI Browser and I had to select a cipher other wise it wont work but when the first one is selected the tool manage to grab the info from the IPMI wihtout any problem. image

Addon's logs 2023/11/16 10:02:57 [error] 205#205: *27 FastCGI sent in stderr: "PHP message: Error occurred when running "ipmitool -H 192.168.1.25 -p 623 -U Administrator -P insertPswdHere -I lanplus bmc info". Unable to Get Channel Cipher Suites Error: Unable to establish IPMI v2 / RMCP+ session PHP message: Error occurred when running "ipmitool -H 192.168.1.25 -p 623 -U Administrator -P insertPswdHere -I lan bmc info". Authentication type NONE not supported Error: Unable to establish LAN session Error: Unable to establish IPMI v1.5 / RMCP session PHP message: Error occurred when running "ipmitool -H 192.168.1.25 -p 623 -U Administrator -P insertPswdHere -I imb bmc info". Error loading interface imb PHP message: Error occurred when running "ipmitool -H 192.168.1.25 -p 623 -U Administrator -P insertPswdHere -I open bmc info". Could not open device at /dev/ipmi0 or /dev/ipmi/0 or /dev/ipmidev/0: No such file or directory" while reading response header from upstream, client: 172.30.32.1, server: _, request: "GET /?host=192.168.1.25&port=623&user=Administrator&password=insertPswdHere HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "localhost:9595"

18262   11/16/2023 11:10 11/16/2023 11:06 3 IPMI/RMCP logout: Administrator - 192.168.1.28.
18261   11/16/2023 11:09 11/16/2023 11:06 3 IPMI/RMCP login by Administrator - 192.168.1.28.
ateodorescu commented 8 months ago

What would be a correct ipmitool synthax to connect to your server? I'm trying to figure out what is missing.

Redblockmasteur commented 8 months ago

So i'm able to connect via this command ipmitool -I lanplus -H 192.168.1.25 -U Administrator sdr Then i'm prompted for my pswd, i think it would work with -P. Since it's not the same command from what i seen the the log it may be an option image

Redblockmasteur commented 8 months ago

Update. I have done some testing and yes it seems that not using the -I lanplus flag create an auth failure image

ateodorescu commented 8 months ago

That's strange because inside the addon I run commands with -I lanplus first and if this one fails then I try -I lan or -I imb and last one -I open. So, lanplus is added to the commands first. Basically I'm trying different values for the interface param so that the user doesn't provide it but I always start with lanplus because it's the most common one. The following commands are executed to fetch data from the server: bmc info, fru, chassis power status, dcmi power reading, sdr type temperature, sdr type voltage, sdr type fan.

What does it display when running ipmitool -H 192.168.1.25 -p 623 -U Administrator -P insertPswdHere -I lanplus bmc info?

Redblockmasteur commented 8 months ago

I ran the command and it returned something. image

jmacko commented 6 months ago

That's strange because inside the addon I run commands with -I lanplus first and if this one fails then I try -I lan or -I imb and last one -I open. So, lanplus is added to the commands first. Basically I'm trying different values for the interface param so that the user doesn't provide it but I always start with lanplus because it's the most common one. The following commands are executed to fetch data from the server: bmc info, fru, chassis power status, dcmi power reading, sdr type temperature, sdr type voltage, sdr type fan.

What does it display when running ipmitool -H 192.168.1.25 -p 623 -U Administrator -P insertPswdHere -I lanplus bmc info?

I think I figured out the missing bit. When I try with the fru command, I end up with this error:

{"success":false,"output":"Error occurred when running \u0022ipmitool -I lanplus -H 10.10.10.25 -U ipmi -P testpass fru\u0022.\nUnable to Get Channel Cipher Suites\n"}

All the other commands listed return data. The fru command when run natively from command line on my server returns:

`root@storage[/]# ipmitool -Uipmi -Ptestpass -H10.10.10.25 -Ilanplus fru FRU Device Description : Builtin FRU Device (ID 0) Chassis Type : Rack Mount Chassis Chassis Serial : MXQXX05P4 Board Mfg Date : Mon Jul 22 20:00:00 2013 Board Mfg : HP Board Product : ProLiant DL360 Gen9 Board Serial : MXQXX05P4 Board Part Number : 755258-B21 Product Manufacturer : HP Product Name : ProLiant DL360 Gen9 Product Part Number : 755258-B21 Product Serial : MXQXX05P4

FRU Device Description : BMC CONTROLLER (ID 238) Product Manufacturer : HPE Product Name : BMC CONTROLLER Product Part Number : iLO 4

FRU Device Description : MB BIOS (ID 239) Product Manufacturer : HPE Product Name : SYSTEM BIOS Product Part Number : P89 Product Version : 09/21/2023

FRU Device Description : CPU 1 (ID 16) Product Manufacturer : Intel(R) Corporation Product Name : Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz

FRU Device Description : CPU 2 (ID 17) Product Manufacturer : Intel(R) Corporation Product Name : Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz

FRU Device Description : CPU 1 DIMM 1 (ID 110) Memory Type : DDR4 SDRAM SDRAM Package Type : Monolithic DRAM Device Technology : RDIMM SDRAM Die Count : 1 SDRAM Capacity : 4096 Mb Memory Bank Group : 2 (4 Bank Groups) Memory Banks : 2 (4 Banks) Primary Bus Width : 64 bits SDRAM Device Width : 4 bits Logical Rank per DIMM : 2 Memory size : 16384 MB Memory Density : 4 Gb 1.2 V Nominal Op : Yes TBD1 V Nominal Op : No TBD2 V Nominal Op : No Error Detect/Cor : 8 bits Manufacturer : Hewlett-Packard Manufacture Date : year 2017 week 8 Serial Number : 28e14419 Part Number : 809081-081

FRU Device Description : CPU 1 DIMM 4 (ID 111) Memory Type : DDR4 SDRAM SDRAM Package Type : Monolithic DRAM Device Technology : RDIMM SDRAM Die Count : 1 SDRAM Capacity : 4096 Mb Memory Bank Group : 2 (4 Bank Groups) Memory Banks : 2 (4 Banks) Primary Bus Width : 64 bits SDRAM Device Width : 4 bits Logical Rank per DIMM : 2 Memory size : 16384 MB Memory Density : 4 Gb 1.2 V Nominal Op : Yes TBD1 V Nominal Op : No TBD2 V Nominal Op : No Error Detect/Cor : 8 bits Manufacturer : Hewlett-Packard Manufacture Date : year 2017 week 8 Serial Number : 28e29379 Part Number : 809081-081

FRU Device Description : CPU 1 DIMM 9 (ID 112) Memory Type : DDR4 SDRAM SDRAM Package Type : Monolithic DRAM Device Technology : RDIMM SDRAM Die Count : 1 SDRAM Capacity : 4096 Mb Memory Bank Group : 2 (4 Bank Groups) Memory Banks : 2 (4 Banks) Primary Bus Width : 64 bits SDRAM Device Width : 4 bits Logical Rank per DIMM : 2 Memory size : 16384 MB Memory Density : 4 Gb 1.2 V Nominal Op : Yes TBD1 V Nominal Op : No TBD2 V Nominal Op : No Error Detect/Cor : 8 bits Manufacturer : Hewlett-Packard Manufacture Date : year 2017 week 8 Serial Number : 28e0c783 Part Number : 809081-081

FRU Device Description : CPU 1 DIMM 11 (ID 113) Memory Type : DDR4 SDRAM SDRAM Package Type : Monolithic DRAM Device Technology : RDIMM SDRAM Die Count : 1 SDRAM Capacity : 4096 Mb Memory Bank Group : 2 (4 Bank Groups) Memory Banks : 2 (4 Banks) Primary Bus Width : 64 bits SDRAM Device Width : 4 bits Logical Rank per DIMM : 2 Memory size : 16384 MB Memory Density : 4 Gb 1.2 V Nominal Op : Yes TBD1 V Nominal Op : No TBD2 V Nominal Op : No Error Detect/Cor : 8 bits Manufacturer : Hewlett-Packard Manufacture Date : year 2017 week 8 Serial Number : 28e2925b Part Number : 809081-081

FRU Device Description : CPU 1 DIMM 12 (ID 114) Memory Type : DDR4 SDRAM SDRAM Package Type : Monolithic DRAM Device Technology : RDIMM SDRAM Die Count : 1 SDRAM Capacity : 4096 Mb Memory Bank Group : 2 (4 Bank Groups) Memory Banks : 2 (4 Banks) Primary Bus Width : 64 bits SDRAM Device Width : 4 bits Logical Rank per DIMM : 2 Memory size : 16384 MB Memory Density : 4 Gb 1.2 V Nominal Op : Yes TBD1 V Nominal Op : No TBD2 V Nominal Op : No Error Detect/Cor : 8 bits Manufacturer : Hewlett-Packard Manufacture Date : year 2017 week 8 Serial Number : 28e29429 Part Number : 809081-081

FRU Device Description : CPU 2 DIMM 1 (ID 115) Memory Type : DDR4 SDRAM SDRAM Package Type : Monolithic DRAM Device Technology : RDIMM SDRAM Die Count : 1 SDRAM Capacity : 4096 Mb Memory Bank Group : 2 (4 Bank Groups) Memory Banks : 2 (4 Banks) Primary Bus Width : 64 bits SDRAM Device Width : 4 bits Logical Rank per DIMM : 2 Memory size : 16384 MB Memory Density : 4 Gb 1.2 V Nominal Op : Yes TBD1 V Nominal Op : No TBD2 V Nominal Op : No Error Detect/Cor : 8 bits Manufacturer : Hewlett-Packard Manufacture Date : year 2016 week 25 Serial Number : 32bb2e2a Part Number : 809081-081

FRU Device Description : CPU 2 DIMM 4 (ID 116) Memory Type : DDR4 SDRAM SDRAM Package Type : Monolithic DRAM Device Technology : RDIMM SDRAM Die Count : 1 SDRAM Capacity : 4096 Mb Memory Bank Group : 2 (4 Bank Groups) Memory Banks : 2 (4 Banks) Primary Bus Width : 64 bits SDRAM Device Width : 4 bits Logical Rank per DIMM : 2 Memory size : 16384 MB Memory Density : 4 Gb 1.2 V Nominal Op : Yes TBD1 V Nominal Op : No TBD2 V Nominal Op : No Error Detect/Cor : 8 bits Manufacturer : Hewlett-Packard Manufacture Date : year 2016 week 27 Serial Number : 32ce6688 Part Number : 809081-081

FRU Device Description : CPU 2 DIMM 9 (ID 117) Memory Type : DDR4 SDRAM SDRAM Package Type : Monolithic DRAM Device Technology : RDIMM SDRAM Die Count : 1 SDRAM Capacity : 4096 Mb Memory Bank Group : 2 (4 Bank Groups) Memory Banks : 2 (4 Banks) Primary Bus Width : 64 bits SDRAM Device Width : 4 bits Logical Rank per DIMM : 2 Memory size : 16384 MB Memory Density : 4 Gb 1.2 V Nominal Op : Yes TBD1 V Nominal Op : No TBD2 V Nominal Op : No Error Detect/Cor : 8 bits Manufacturer : Hewlett-Packard Manufacture Date : year 2016 week 25 Serial Number : 32ba6899 Part Number : 809081-081

FRU Device Description : CPU 2 DIMM 11 (ID 118) Memory Type : DDR4 SDRAM SDRAM Package Type : Monolithic DRAM Device Technology : RDIMM SDRAM Die Count : 1 SDRAM Capacity : 4096 Mb Memory Bank Group : 2 (4 Bank Groups) Memory Banks : 2 (4 Banks) Primary Bus Width : 64 bits SDRAM Device Width : 4 bits Logical Rank per DIMM : 2 Memory size : 16384 MB Memory Density : 4 Gb 1.2 V Nominal Op : Yes TBD1 V Nominal Op : No TBD2 V Nominal Op : No Error Detect/Cor : 8 bits Manufacturer : Hewlett-Packard Manufacture Date : year 2017 week 8 Serial Number : 28e1f121 Part Number : 809081-081

FRU Device Description : CPU 2 DIMM 12 (ID 119) Memory Type : DDR4 SDRAM SDRAM Package Type : Monolithic DRAM Device Technology : RDIMM SDRAM Die Count : 1 SDRAM Capacity : 4096 Mb Memory Bank Group : 2 (4 Bank Groups) Memory Banks : 2 (4 Banks) Primary Bus Width : 64 bits SDRAM Device Width : 4 bits Logical Rank per DIMM : 2 Memory size : 16384 MB Memory Density : 4 Gb 1.2 V Nominal Op : Yes TBD1 V Nominal Op : No TBD2 V Nominal Op : No Error Detect/Cor : 8 bits Manufacturer : Hewlett-Packard Manufacture Date : year 2016 week 27 Serial Number : 32d35d11 Part Number : 809081-081

FRU Device Description : Ethernet Adptr (ID 2) Board Mfg Date : Sun Jan 19 12:15:00 2014 Board Mfg : HP Board Product : HP Ethernet 10Gb 2-port 560FLR-SFP+ Adapter Board Serial : G42955-017 Board Part Number : 669281-001 Board Extra : d23043 Board Extra : d5665241b001 Product Manufacturer : HP Product Name : HP Ethernet 10Gb 2-port 560FLR-SFP+ Adapter Product Part Number : 665243-B21 Product Version : 00 Product Serial : MYI72300H0

FRU Device Description : Embedded 0 (ID 3) Board Mfg Date : Sun May 27 20:00:00 2007 Board Part Number : 871264-001 Product Manufacturer : SMP Product Name : HPE Smart Storage Battery Product Part Number : 727258-B21 Product Version : 01 Product Serial : 6WEJD0FWY6820W

FRU Device Description : Ethernet Adptr (ID 4) Product Name : HPE Ethernet 1Gb 4-port 331i Adapter - NIC

FRU Device Description : SAS Ctlr (ID 5) Product Name : Smart Array P440ar Controller Product Part Number : 749796-001 Product Version : B Product Serial : PDNLH0BRH8A13G`

Also IPMI against HP servers is really slow vs supermicro. Most of the calls take around 7-9 seconds to return. Are we hitting some sort of timeout, and the error return is a result?

Suggestions on how I can debug this?

Redblockmasteur commented 5 months ago

I dont know but in the end i used HA build int ILO integration but it broke with 2024.2

xbeaudouin commented 3 months ago

Hello, I have an HP MicroServer Gen 8 as well and I have also the problem. I am trying to burst the issue on my side but it seems to timeout when adding the server.

All commands works... Except maybe:

$ ipmitool -H 192.0.2.1 -U user -P pass -I lanplus dcmi power reading

    DCMI request failed because: Command not supported in present state (d5)

I don't know very well PHP, maybe there is some exception on dcmi power reading ?

ateodorescu commented 3 months ago

dcmi power reading is not working on all servers. For example on my server it runs but returns this Screenshot 2024-03-22 at 10 06 04 So, the addon will run the command and if it returns anything then the values will be extracted but if it fails nothing is extracted. How long does it take for the command sdr list full to run?

xbeaudouin commented 3 months ago

Well it is pretty fast:

$ time ipmitool -H 192.0.21 -U user -P pass -I lanplus sdr list full
UID Light        | 0x00              | ok
Health LED       | no reading        | ns
01-Inlet Ambient | 21 degrees C      | ok
02-CPU           | 40 degrees C      | ok
03-P1 DIMM 1-2   | 33 degrees C      | ok
04-HD Max        | disabled          | ns
05-Chipset       | 56 degrees C      | ok
06-Chipset Zone  | 43 degrees C      | ok
07-VR P1 Zone    | 47 degrees C      | ok
08-Supercap Max  | disabled          | ns
09-iLO Zone      | 47 degrees C      | ok
10-PCI 1         | disabled          | ns
11-PCI 1 Zone    | 42 degrees C      | ok
12-Sys Exhaust   | 43 degrees C      | ok
13-LOM           | disabled          | ns
Fan 1            | 19.99 percent     | ok
Power Supply 1   | 0x00              | ok
Memory           | 0x00              | ok

real    0m0.378s
user    0m0.015s
sys 0m0.000s

I think the command timeout somewhere... or the process dies... for some reasons.

ateodorescu commented 3 months ago

Well it is pretty fast:

$ time ipmitool -H 192.0.21 -U user -P pass -I lanplus sdr list full
UID Light        | 0x00              | ok
Health LED       | no reading        | ns
01-Inlet Ambient | 21 degrees C      | ok
02-CPU           | 40 degrees C      | ok
03-P1 DIMM 1-2   | 33 degrees C      | ok
04-HD Max        | disabled          | ns
05-Chipset       | 56 degrees C      | ok
06-Chipset Zone  | 43 degrees C      | ok
07-VR P1 Zone    | 47 degrees C      | ok
08-Supercap Max  | disabled          | ns
09-iLO Zone      | 47 degrees C      | ok
10-PCI 1         | disabled          | ns
11-PCI 1 Zone    | 42 degrees C      | ok
12-Sys Exhaust   | 43 degrees C      | ok
13-LOM           | disabled          | ns
Fan 1            | 19.99 percent     | ok
Power Supply 1   | 0x00              | ok
Memory           | 0x00              | ok

real  0m0.378s
user  0m0.015s
sys   0m0.000s

I think the command timeout somewhere... or the process dies... for some reasons.

Do you use last versions of both ipmi integration and addon?

xbeaudouin commented 3 months ago

Yeah. Ipmitool addon is 1.2.6... I just saw 1.2.9 of https://github.com/ateodorescu/home-assistant-addons/blob/main/ipmi-server/config.yaml, let's try to upgrade... Brb

ateodorescu commented 3 months ago

Yeah. Ipmitool addon is 1.2.6... I just saw 1.2.9 of https://github.com/ateodorescu/home-assistant-addons/blob/main/ipmi-server/config.yaml, let's try to upgrade... Brb

Hope that the last one will fix your problems.

xbeaudouin commented 3 months ago

Okay... On the begining it doesn't works. Then after tinkering I say http://hassserver:9695/command?params= works. So I remove http://localhost from the control panel whan adding host and it works !. Very strange.... only on this HP I have such issue...

ateodorescu commented 3 months ago

I assume we can close the issue now.