missing fan status

[i0x71]

Hi, It seems that ipmitool with the following syntax is able to extract more information, in my case querying an HP DL360 G10 server ilo remotely, its able to get fan status. However when querying with ipmi_export with the following settings, other info comes up but nothing regarding the fans

            driver: "LAN_2_0"
            privilege: "admin"
            collectors:
            - bmc
            - ipmi
            - chassis
            - sel
            - sm-lan-mode
            - dcmi

Any way to fix this ?

Thanks

ipmitool -H xxxxx -U xxxxxxx -P xxxxxx -I lanplus sdr elist UID | AEh | ok | 23.1 | SysHealth_Stat | ACh | ok | 23.1 | Transition to OK 01-Inlet Ambient | A2h | ok | 55.1 | 19 degrees C 02-CPU 1 | 02h | ok | 3.1 | 40 degrees C 03-CPU 2 | 03h | ok | 3.2 | 40 degrees C 04-P1 DIMM 1-6 | 24h | ok | 32.11 | 29 degrees C 05-PMM 1-6 | FEh | ns | 7.1 | Disabled 06-P1 DIMM 7-12 | 25h | ok | 32.12 | 29 degrees C 07-PMM 7-12 | FDh | ns | 7.1 | Disabled 08-P2 DIMM 1-6 | 26h | ok | 32.13 | 32 degrees C 09-PMM 1-6 | FCh | ns | 7.1 | Disabled 10-P2 DIMM 7-12 | 27h | ok | 32.14 | 31 degrees C 11-PMM 7-12 | FBh | ns | 7.1 | Disabled 12-HD Max | 98h | ns | 4.3 | Disabled 13-Exp Bay Drive | 96h | ns | 4.1 | Disabled 14-Stor Batt 1 | BFh | ok | 40.1 | 19 degrees C 15-Front Ambient | A3h | ok | 55.2 | 21 degrees C 16-VR P1 | 0Ah | ok | 20.1 | 34 degrees C 17-VR P2 | 0Bh | ok | 20.2 | 36 degrees C 18-VR P1 Mem 1 | 61h | ok | 20.21 | 25 degrees C 19-VR P1 Mem 2 | 62h | ok | 20.22 | 25 degrees C 20-VR P2 Mem 1 | 63h | ok | 20.23 | 27 degrees C 21-VR P2 Mem 2 | 64h | ok | 20.24 | 29 degrees C 22-Chipset | A4h | ok | 7.1 | 37 degrees C 23-BMC | FAh | ok | 7.1 | 66 degrees C 24-BMC Zone | F9h | ok | 7.1 | 37 degrees C 25-HD Controller | B3h | ok | 11.1 | 52 degrees C 26-HD Cntlr Zone | F8h | ok | 7.1 | 28 degrees C 27-LOM | B4h | ok | 11.3 | 44 degrees C 28-LOM Card | B5h | ns | 11.4 | Disabled 29-I/O Zone | F7h | ok | 7.1 | 29 degrees C 30-PCI 1 | 84h | ns | 49.1 | Disabled 31-PCI 1 Zone | F6h | ok | 7.1 | 30 degrees C 32-PCI 2 | 85h | ns | 49.2 | Disabled 33-PCI 2 Zone | F5h | ok | 7.1 | 30 degrees C 34-PCI 3 | 86h | ns | 49.3 | Disabled 35-PCI 3 Zone | F4h | ns | 7.1 | Disabled 37-Rear HD Max | 99h | ns | 4.4 | Disabled 38-Battery Zone | F3h | ok | 7.1 | 32 degrees C 39-P/S 1 Inlet | 4Bh | ok | 10.1 | 30 degrees C 40-P/S 2 Inlet | 4Ch | ok | 10.2 | 32 degrees C 41-P/S 1 | 43h | ok | 10.1 | 40 degrees C 42-P/S 2 | 44h | ok | 10.2 | 40 degrees C 43-E-Fuse | A5h | ok | 20.0 | 30 degrees C 44-P/S 2 Zone | F2h | ok | 7.1 | 34 degrees C 61-AHCI HD Max | F1h | ns | 7.1 | Disabled 69-PCI 1 M2 | F0h | ns | 7.1 | Disabled 70-PCI 1 M2 Zn | EFh | ns | 7.1 | Disabled 71-PCI 2 M2 | EEh | ns | 7.1 | Disabled 72-PCI 2 M2 Zn | EDh | ns | 7.1 | Disabled 73-PCI 3 M2 | ECh | ns | 7.1 | Disabled 74-PCI 3 M2 Zn | EBh | ns | 7.1 | Disabled Fan 1 | 03h | ok | 29.1 | Transition to OK Fan 1 DutyCycle | 01h | ok | 29.1 | 39.98 percent Fan 1 Presence | 02h | ok | 29.1 | Device Present Fan 2 | 06h | ok | 29.2 | Transition to OK Fan 2 DutyCycle | 04h | ok | 29.2 | 39.98 percent Fan 2 Presence | 05h | ok | 29.2 | Device Present Fan 3 | 09h | ok | 29.3 | Transition to OK Fan 3 DutyCycle | 07h | ok | 29.3 | 39.98 percent Fan 3 Presence | 08h | ok | 29.3 | Device Present Fan 4 | 0Ch | ok | 29.4 | Transition to OK Fan 4 DutyCycle | 0Ah | ok | 29.4 | 39.98 percent Fan 4 Presence | 0Bh | ok | 29.4 | Device Present Fan 5 | 0Fh | ok | 29.5 | Transition to OK Fan 5 DutyCycle | 0Dh | ok | 29.5 | 39.98 percent Fan 5 Presence | 0Eh | ok | 29.5 | Device Present Fan 6 | 12h | ok | 29.6 | Transition to OK Fan 6 DutyCycle | 10h | ok | 29.6 | 39.98 percent Fan 6 Presence | 11h | ok | 29.6 | Device Present Fan 7 | 15h | ok | 29.7 | Transition to OK Fan 7 DutyCycle | 13h | ok | 29.7 | 39.98 percent Fan 7 Presence | 14h | ok | 29.7 | Device Present Power Supply 1 | 32h | ok | 10.1 | Presence detected PS 1 Output | 3Ah | ok | 10.1 | 105 Watts Power Supply 2 | 33h | ok | 10.2 | Presence detected PS 2 Output | 3Bh | ok | 10.2 | 130 Watts Power Meter | B6h | ok | 7.1 | 250 Watts Fans | 31h | ok | 30.1 | Fully Redundant Power Supplies | 42h | ok | 19.1 | Fully Redundant Memory Status | 48h | ok | 32.0 | Presence Detected Megacell Status | BBh | ok | 40.1 | Presence Detected Intrusion | ABh | ns | 23.1 | Disabled CPU Utilization | 23h | ok | 3.1 | 48 unspecified LOM_Link_P1 | 01h | ns | 44.97 | Disabled LOM_Link_P2 | 02h | ns | 44.98 | Disabled LOM_Link_P3 | 03h | ns | 44.99 | Disabled LOM_Link_P4 | 04h | ns | 44.100 | Disabled ALOM_Link_P1 | 01h | ok | 44.97 | Heartbeat ALOM_Link_P2 | 02h | ns | 44.98 | Disabled ALOM_Link_P3 | 03h | ns | 44.99 | Disabled ALOM_Link_P4 | 04h | ns | 44.100 | Disabled Dr_Stat_1I1_B001 | 02h | ns | 0.0 | No Reading Dr_Stat_1I1_B002 | 03h | ns | 0.0 | No Reading Dr_Stat_1I1_B003 | 04h | ns | 0.0 | No Reading Dr_Stat_1I1_B004 | 05h | ns | 0.0 | No Reading CPU_Stat_C1 | 12h | ok | 3.1 | Presence detected CPU_Stat_C2 | 13h | ok | 3.2 | Presence detected

[bitfehler]

Can you try to run this command against that server and let me know if anything about the fans shows up in the output?

ipmimonitoring -h XXX -u XXX -p XXX -D LAN_2_0 --comma-separated-output --no-header-output --sdr-cache-recreate --interpret-oem-data -v

It might be that the sensors are using some OEM-specific format, that in the worst case FreeIPMI might not understand...

[i0x71]

Hi, thanks for a prompt reply, there is indeed mention of the fans, however it appears that the readings are blank.

0,UID,OEM Reserved,N/A,N/A,N/A,'OEM Event = 0001h'
1,SysHealth_Stat,Chassis,Nominal,N/A,N/A,'transition to OK'
2,01-Inlet Ambient,Temperature,Nominal,20.00,C,'OK'
3,02-CPU 1,Temperature,Nominal,40.00,C,'OK'
4,03-CPU 2,Temperature,Nominal,40.00,C,'OK'
5,04-P1 DIMM 1-6,Temperature,Nominal,30.00,C,'OK'
6,05-PMM 1-6,Temperature,N/A,N/A,C,N/A
7,06-P1 DIMM 7-12,Temperature,Nominal,30.00,C,'OK'
8,07-PMM 7-12,Temperature,N/A,N/A,C,N/A
9,08-P2 DIMM 1-6,Temperature,Nominal,35.00,C,'OK'
10,09-PMM 1-6,Temperature,N/A,N/A,C,N/A
11,10-P2 DIMM 7-12,Temperature,Nominal,34.00,C,'OK'
12,11-PMM 7-12,Temperature,N/A,N/A,C,N/A
13,12-HD Max,Temperature,N/A,N/A,C,N/A
14,13-Exp Bay Drive,Temperature,N/A,N/A,C,N/A
15,14-Stor Batt 1,Temperature,Nominal,19.00,C,'OK'
16,15-Front Ambient,Temperature,Nominal,21.00,C,'OK'
17,16-VR P1,Temperature,Nominal,36.00,C,'OK'
18,17-VR P2,Temperature,Nominal,40.00,C,'OK'
19,18-VR P1 Mem 1,Temperature,Nominal,25.00,C,'OK'
20,19-VR P1 Mem 2,Temperature,Nominal,26.00,C,'OK'
21,20-VR P2 Mem 1,Temperature,Nominal,28.00,C,'OK'
22,21-VR P2 Mem 2,Temperature,Nominal,30.00,C,'OK'
23,22-Chipset,Temperature,Nominal,39.00,C,'OK'
24,23-BMC,Temperature,Nominal,68.00,C,'OK'
25,24-BMC Zone,Temperature,Nominal,38.00,C,'OK'
26,25-HD Controller,Temperature,Nominal,53.00,C,'OK'
27,26-HD Cntlr Zone,Temperature,Nominal,29.00,C,'OK'
28,27-LOM,Temperature,Nominal,45.00,C,'OK'
29,28-LOM Card,Temperature,N/A,N/A,C,N/A
30,29-I/O Zone,Temperature,Nominal,30.00,C,'OK'
31,30-PCI 1,Temperature,N/A,N/A,C,N/A
32,31-PCI 1 Zone,Temperature,Nominal,31.00,C,'OK'
33,32-PCI 2,Temperature,N/A,N/A,C,N/A
34,33-PCI 2 Zone,Temperature,Nominal,31.00,C,'OK'
35,34-PCI 3,Temperature,N/A,N/A,C,N/A
36,35-PCI 3 Zone,Temperature,N/A,N/A,C,N/A
37,37-Rear HD Max,Temperature,N/A,N/A,C,N/A
38,38-Battery Zone,Temperature,Nominal,34.00,C,'OK'
39,39-P/S 1 Inlet,Temperature,N/A,N/A,C,N/A
40,40-P/S 2 Inlet,Temperature,N/A,N/A,C,N/A
41,41-P/S 1,Temperature,N/A,N/A,C,N/A
42,42-P/S 2,Temperature,N/A,N/A,C,N/A
43,43-E-Fuse,Temperature,Nominal,33.00,C,'OK'
44,44-P/S 2 Zone,Temperature,Nominal,38.00,C,'OK'
45,61-AHCI HD Max,Temperature,N/A,N/A,C,N/A
46,69-PCI 1 M2,Temperature,N/A,N/A,C,N/A
47,70-PCI 1 M2 Zn,Temperature,N/A,N/A,C,N/A
48,71-PCI 2 M2,Temperature,N/A,N/A,C,N/A
49,72-PCI 2 M2 Zn,Temperature,N/A,N/A,C,N/A
50,73-PCI 3 M2,Temperature,N/A,N/A,C,N/A
51,74-PCI 3 M2 Zn,Temperature,N/A,N/A,C,N/A
52,Fan 1,Fan,N/A,N/A,N/A,N/A
53,Fan 1 DutyCycle,Fan,N/A,N/A,%,N/A
54,Fan 1 Presence,Fan,N/A,N/A,N/A,N/A
55,Fan 2,Fan,N/A,N/A,N/A,N/A
56,Fan 2 DutyCycle,Fan,N/A,N/A,%,N/A
57,Fan 2 Presence,Fan,N/A,N/A,N/A,N/A
58,Fan 3,Fan,N/A,N/A,N/A,N/A
59,Fan 3 DutyCycle,Fan,N/A,N/A,%,N/A 60,Fan 3 Presence,Fan,N/A,N/A,N/A,N/A 61,Fan 4,Fan,N/A,N/A,N/A,N/A 62,Fan 4 DutyCycle,Fan,N/A,N/A,%,N/A 63,Fan 4 Presence,Fan,N/A,N/A,N/A,N/A 64,Fan 5,Fan,N/A,N/A,N/A,N/A 65,Fan 5 DutyCycle,Fan,N/A,N/A,%,N/A 66,Fan 5 Presence,Fan,N/A,N/A,N/A,N/A 67,Fan 6,Fan,N/A,N/A,N/A,N/A 68,Fan 6 DutyCycle,Fan,N/A,N/A,%,N/A 69,Fan 6 Presence,Fan,N/A,N/A,N/A,N/A 70,Fan 7,Fan,N/A,N/A,N/A,N/A 71,Fan 7 DutyCycle,Fan,N/A,N/A,%,N/A 72,Fan 7 Presence,Fan,N/A,N/A,N/A,N/A 73,Power Supply 1,Power Supply,N/A,N/A,N/A,N/A 74,PS 1 Output,Power Supply,N/A,N/A,W,N/A 75,Power Supply 2,Power Supply,N/A,N/A,N/A,N/A 76,PS 2 Output,Power Supply,N/A,N/A,W,N/A 77,Power Meter,Other Units Based Sensor,Nominal,285.00,W,'OK' 78,Fans,Fan,N/A,N/A,N/A,N/A 80,Power Supplies,Power Supply,N/A,N/A,N/A,N/A 82,Memory Status,Memory,Nominal,N/A,N/A,'Presence detected' 83,Megacell Status,Battery,Nominal,N/A,N/A,'battery presence detected' 84,Intrusion,Physical Security,N/A,N/A,N/A,N/A 85,CPU Utilization,Processor,Nominal,81.00,unspecified,'OK' 86,LOM_Link_P1,LAN,N/A,N/A,N/A,N/A 87,LOM_Link_P2,LAN,N/A,N/A,N/A,N/A 88,LOM_Link_P3,LAN,N/A,N/A,N/A,N/A 89,LOM_Link_P4,LAN,N/A,N/A,N/A,N/A 90,ALOM_Link_P1,LAN,N/A,N/A,N/A,N/A 91,ALOM_Link_P2,LAN,N/A,N/A,N/A,N/A 92,ALOM_Link_P3,LAN,N/A,N/A,N/A,N/A 93,ALOM_Link_P4,LAN,N/A,N/A,N/A,N/A 94,Dr_Stat_1I1_B001,Drive Slot,N/A,N/A,N/A,N/A 95,Dr_Stat_1I1_B002,Drive Slot,N/A,N/A,N/A,N/A 96,Dr_Stat_1I1_B003,Drive Slot,N/A,N/A,N/A,N/A 97,Dr_Stat_1I1_B004,Drive Slot,N/A,N/A,N/A,N/A 98,CPU_Stat_C1,Processor,Nominal,N/A,N/A,'Processor Presence detected' 99,CPU_Stat_C2,Processor,Nominal,N/A,N/A,'Processor Presence detected'

[bitfehler]

I am afraid then it is as I mentioned, FreeIPMI might not support reading these sensors. Your best bet might be to get in touch with them, if you find any way to display the values with ipmimonitoring (some combination of flags or whatnot) please let me know and we can try to incorporate that in the exporter.

[Tommy-Dimitrio]

In my case with HP iLO 5 I had to add -b to let it read some sensors such as fan speed:

ipmimonitoring -h $HOST -u $USER -p $PASSWORD -D LAN_2_0 --comma-separated-output --no-header-output --sdr-cache-recreate --interpret-oem-data -b.

[Tommy-Dimitrio]

@bitfehler is it possible to include this param using the workaround_flags ?

[fabricat]

@bitfehler is it possible to include this param using the workaround_flags ?

@Xunil32 unfortunately, it seems not possible 😞 If have barely tried to add the following lines to my config:

workaround_flags:
- none
- --bridge-sensors

but I get the following error:

level=warning msg="Error while calling ipmimonitoring for 192.168.1.1: Config File Error: invalid value for workaround-flags\n" source="collector.go:278"

Anyway, thanks for the proposed solution 👍

[Tommy-Dimitrio]

@fabricat Thanks! I tried the same but seems like the workaround flags are meant just for specific flags available here: https://www.gnu.org/software/freeipmi/manpages/man5/freeipmi.conf.5.html

I could also write another solution where we can specify additional parameters from the config file. Would you consider/prefer that pull request?

[fabricat]

@Xunil32 I have read the straight way to use workaround_flags... I've just tried a simple hack 😝 but I got no luck with that.

About the possibility to add more params in config, I agree... but I am just a regular user (even not skilled with Go). I can only give you my 👍 here 😄

[bitfehler]

Hey folks,

thanks for looking into this and digging up the details here, much appreciated! The man page says this about --bridge-sensors:

Bridging may not work on some interfaces/driver types.

So I am a bit reluctant to simply make it a default. However, I do have configuration refactoring in the works that this could be incorporated into. Give me a moment :slightly_smiling_face:

[Tommy-Dimitrio]

Thanks a lot @bitfehler, if you are already working on a solution I will stop here. I tested with many type of iLO and IDRAC interfaces but cover every vendor is quite hard. Your decision to make it configurable is a huge step forward for the product in my opinion, thanks for the effort.

[bitfehler]

Hey again. Sorry it took so long, but I finally did a huge refactoring that I've been wanting to do for a long time. If you are comfortable with building from source you can check out the branch bitfehler/refactoring. It adds support for specifying custom arguments for the collectors.

I even added your exact use case as an example to the sample config file: :slightly_smiling_face: https://github.com/soundcloud/ipmi_exporter/blob/bitfehler/refactoring/ipmi_remote.yml#L62-L66

It works well on the hosts that I tested, but I need to sleep over it once or twice before making a new release. If you feel like trying it I would of course much appreciate any feedback.

[Tommy-Dimitrio]

Hey again. Sorry it took so long, but I finally did a huge refactoring that I've been wanting to do for a long time. If you are comfortable with building from source you can check out the branch bitfehler/refactoring. It adds support for specifying custom arguments for the collectors.

I even added your exact use case as an example to the sample config file: 🙂 https://github.com/soundcloud/ipmi_exporter/blob/bitfehler/refactoring/ipmi_remote.yml#L62-L66

It works well on the hosts that I tested, but I need to sleep over it once or twice before making a new release. If you feel like trying it I would of course much appreciate any feedback.

wooow, thanks!

[bitfehler]

So, closing this as this should be easy with the new release. Feel free to open a new issue if something is not working.