search

lausser / check_nwc_health

nwc = network component. This plugin checks lots of aspects of routers, switches, wlan controllers, firewalls,.....
http://labs.consol.de/nagios/check_nwc_health
GNU General Public License v2.0
151 stars 87 forks source link

hardware-health timeout on newer nexus #193

Closed Noname2244 closed 5 years ago

Noname2244 commented 5 years ago

Hardware cisco Nexus9000 93180YC-EX chassis Intel(R) Xeon(R) CPU @ 1.80GHz with 24633488 kB of memory.

Software BIOS: version 07.61 NXOS: version 7.0(3)I7(3) BIOS compile time: 04/06/2017 NXOS image file is: bootflash:///nxos.7.0.3.I7.3.bin NXOS compile time: 2/12/2018 13:00:00 [02/12/2018 19:13:48]

with this inventory: NAME: "Chassis", DESCR: "Nexus9000 93180YC-EX chassis" NAME: "Slot 1", DESCR: "48x10/25G + 6x40/100G Ethernet Module" NAME: "Power Supply 1", DESCR: "Nexus9000 93180YC-EX chassis Power Supply" NAME: "Power Supply 2", DESCR: "Nexus9000 93180YC-EX chassis Power Supply" NAME: "Fan 1", DESCR: "Nexus9000 93180YC-EX chassis Fan Module" NAME: "Fan 2", DESCR: "Nexus9000 93180YC-EX chassis Fan Module" NAME: "Fan 3", DESCR: "Nexus9000 93180YC-EX chassis Fan Module" NAME: "Fan 4", DESCR: "Nexus9000 93180YC-EX chassis Fan Module"

all other checks , config-check, cpu-load etc. works fine.

show environment power det Power Supply: Voltage: 12 Volts Power Actual Actual Total Supply Model Output Input Capacity Status (Watts ) (Watts ) (Watts )

1 NXA-PAC-650W-PI 76 W 102 W 650 W Ok 2 NXA-PAC-650W-PI 80 W 102 W 650 W Ok

Power Usage Summary:

Power Supply redundancy mode (configured) PS-Redundant Power Supply redundancy mode (operational) PS-Redundant

Total Power Capacity (based on configured mode) 650.00 W Total Grid-A (first half of PS slots) Power Capacity 650.00 W Total Grid-B (second half of PS slots) Power Capacity 650.00 W Total Power of all Inputs (cumulative) 1300.00 W Total Power Output (actual draw) 156.00 W Total Power Input (actual draw) 204.00 W Total Power Allocated (budget) N/A Total Power Available for additional modules N/A

Power Usage details:

Power reserved for Supervisor(s): N/A Power reserved for Fabric, SC Module(s): N/A Power reserved for Fan Module(s): N/A Total power reserved for Sups,SCs,Fabrics,Fans: N/A

Are all inlet cords connected: Yes

Power supply details:

PS_1 total capacity: 650 W Voltage:12V Pin:102.75W Vin:238.50V Iin:0.42A Pout:76.38W Vout:12.04V Iout:6.36A Cord connected to 220V AC Software-Alarm: No Hardware alarm_bits

PS_2 total capacity: 650 W Voltage:12V Pin:102.12W Vin:236.50V Iin:0.42A Pout:80.38W Vout:12.05V Iout:6.62A Cord connected to 220V AC Software-Alarm: No Hardware alarm_bits

from http://www.oidview.com/mibs/9/CISCO-ENTITY-FRU-CONTROL-MIB.html ( is this relevant ?)

cefcFRUPowerStatusEntry 1.3.6.1.4.1.9.9.117.1.1.2.1
cefcFRUPowerAdminStatus 1.3.6.1.4.1.9.9.117.1.1.2.1.1
cefcFRUPowerOperStatus 1.3.6.1.4.1.9.9.117.1.1.2.1.2
cefcFRUCurrent 1.3.6.1.4.1.9.9.117.1.1.2.1.3
cefcFRUPowerCapability 1.3.6.1.4.1.9.9.117.1.1.2.1.4
cefcFRURealTimeCurrent 1.3.6.1.4.1.9.9.117.1.1.2.1.5
cefcPowerRedundancyMode 1.3.6.1.4.1.9.9.117.1.1.1.1.1

more oids: ftp://ftp.cisco.com/pub/mibs/supportlists/nexus9000/Nexus9000MIBSupportList.html

Noname2244 commented 5 years ago

Can one use debug on the script?

lausser commented 5 years ago

You can add "-vvvvvvvvvvvv"to see the internals.

Noname2244 commented 5 years ago 
./check_nwc_health -vvvvvvvvvvvv --mode hardware-health --hostname 1.2.3.4 --community xxxxxxx -t 30
Mon Dec 10 10:11:54 2018: AUTOLOAD Classes::Device::check_messages

Mon Dec 10 10:11:54 2018: i know package Monitoring::GLPlugin::SNMP::MibsAndOids::MIB2MIB
Mon Dec 10 10:11:54 2018: cache: 1.3.6.1.2.1.1.3.0
Mon Dec 10 10:11:55 2018: i know package Monitoring::GLPlugin::SNMP::MibsAndOids::MIB2MIB
Mon Dec 10 10:11:55 2018: GET: MIB-2-MIB::sysUpTime (1.3.6.1.2.1.1.3) : 641651646
Mon Dec 10 10:11:55 2018: i know package Monitoring::GLPlugin::SNMP::MibsAndOids::SNMPFRAMEWORKMIB
Mon Dec 10 10:11:55 2018: cache: 1.3.6.1.6.3.10.2.1.3.0
Mon Dec 10 10:11:55 2018: i know package Monitoring::GLPlugin::SNMP::MibsAndOids::SNMPFRAMEWORKMIB
Mon Dec 10 10:11:55 2018: GET: SNMP-FRAMEWORK-MIB::snmpEngineTime (1.3.6.1.6.3.10.2.1.3.0) : 6416516
Mon Dec 10 10:11:55 2018: i know package Monitoring::GLPlugin::SNMP::MibsAndOids::HOSTRESOURCESMIB
Mon Dec 10 10:11:55 2018: cache: 1.3.6.1.2.1.25.1.1
Mon Dec 10 10:11:55 2018: GET: HOST-RESOURCES-MIB::hrSystemUptime (1.3.6.1.2.1.25.1.1) : <undef>
Mon Dec 10 10:11:55 2018: i know package Monitoring::GLPlugin::SNMP::MibsAndOids::HOSTRESOURCESMIB
Mon Dec 10 10:11:55 2018: cache: 1.3.6.1.2.1.25.1.1.0
Mon Dec 10 10:11:55 2018: GET: HOST-RESOURCES-MIB::hrSystemUptime (1.3.6.1.2.1.25.1.1) : <undef>
Mon Dec 10 10:11:55 2018: i know package Monitoring::GLPlugin::SNMP::MibsAndOids::MIB2MIB
Mon Dec 10 10:11:55 2018: cache: 1.3.6.1.2.1.1.1.0
Mon Dec 10 10:11:55 2018: i know package Monitoring::GLPlugin::SNMP::MibsAndOids::MIB2MIB
Mon Dec 10 10:11:55 2018: GET: MIB-2-MIB::sysDescr (1.3.6.1.2.1.1.1) : Cisco NX-OS(tm) nxos.7.0.3.I7.3.bin, Software (nxos), Version 7.0(3)I7(3), RELEASE SOFTWARE Copyright (c) 2002-2016 by Cisco Systems, Inc. Compiled 2/12/2018 13:00:00
Mon Dec 10 10:11:55 2018: snmpEngineTime says: up since: Thu Sep 27 04:49:59 2018 / 74d 6h 21m 56s
Mon Dec 10 10:11:55 2018: sysUptime says:      up since: Thu Sep 27 04:49:58 2018 / 74d 6h 21m 56s
Mon Dec 10 10:11:55 2018: i know package Monitoring::GLPlugin::SNMP::MibsAndOids::MIB2MIB
Mon Dec 10 10:11:55 2018: cache: 1.3.6.1.2.1.1.2.0
Mon Dec 10 10:11:55 2018: i know package Monitoring::GLPlugin::SNMP::MibsAndOids::MIB2MIB
Mon Dec 10 10:11:55 2018: GET: MIB-2-MIB::sysObjectID (1.3.6.1.2.1.1.2) : 1.3.6.1.4.1.9.12.3.1.3.1812
Mon Dec 10 10:11:55 2018: uptime: 6416516
Mon Dec 10 10:11:55 2018: up since: Thu Sep 27 04:49:59 2018
Mon Dec 10 10:11:55 2018: whoami: Cisco NX-OS(tm) nxos.7.0.3.I7.3.bin, Software (nxos), Version 7.0(3)I7(3), RELEASE SOFTWARE Copyright (c) 2002-2016 by Cisco Systems, Inc. Compiled 2/12/2018 13:00:00
Mon Dec 10 10:11:55 2018: AUTOLOAD Classes::Device::check_messages

Mon Dec 10 10:11:55 2018: I am a Cisco NX-OS(tm) nxos.7.0.3.I7.3.bin, Software (nxos), Version 7.0(3)I7(3), RELEASE SOFTWARE Copyright (c) 2002-2016 by Cisco Systems, Inc. Compiled 2/12/2018 13:00:00

Mon Dec 10 10:11:55 2018: using Classes::Cisco
Mon Dec 10 10:11:55 2018: AUTOLOAD Classes::Cisco::override_opt

Mon Dec 10 10:11:55 2018: AUTOLOAD Monitoring::GLPlugin::Commandline::override_opt

Mon Dec 10 10:11:55 2018: AUTOLOAD Classes::Cisco::check_messages

Mon Dec 10 10:11:55 2018: using Classes::Cisco::NXOS
Mon Dec 10 10:11:55 2018: AUTOLOAD Classes::Cisco::NXOS::analyze_and_check_environmental_subsystem

Mon Dec 10 10:11:55 2018: i know package Monitoring::GLPlugin::SNMP::MibsAndOids::CISCOENTITYSENSORMIB
Mon Dec 10 10:11:55 2018: get_snmp_table_objects CISCO-ENTITY-SENSOR-MIB entSensorValueTable
Mon Dec 10 10:11:55 2018: i know package Monitoring::GLPlugin::SNMP::MibsAndOids::CISCOENTITYSENSORMIB
Mon Dec 10 10:11:55 2018: cache: 1.3.6.1.4.1.9.9.91.1.1.1
Mon Dec 10 10:11:55 2018: get_table $VAR1 = {
  '-baseoid' => '1.3.6.1.4.1.9.9.91.1.1.1',
  '-maxrepetitions' => 10
};

Mon Dec 10 10:12:02 2018: get_table returned 343 oids
Mon Dec 10 10:12:02 2018: get_matching_oids $VAR1 = {
  '-columns' => [
    '1.3.6.1.4.1.9.9.91.1.1.1'
  ]
};

Mon Dec 10 10:12:02 2018: get_matching_oids returns 343 from 349 oids
Mon Dec 10 10:12:02 2018: i know package Monitoring::GLPlugin::SNMP::MibsAndOids::CISCOENTITYSENSORMIB
Mon Dec 10 10:12:02 2018: get_snmp_table_objects default returns 49 entries
Mon Dec 10 10:12:02 2018: i know package Monitoring::GLPlugin::SNMP::MibsAndOids::CISCOENTITYSENSORMIB
Mon Dec 10 10:12:02 2018: get_snmp_table_objects CISCO-ENTITY-SENSOR-MIB entSensorThresholdTable
Mon Dec 10 10:12:02 2018: i know package Monitoring::GLPlugin::SNMP::MibsAndOids::CISCOENTITYSENSORMIB
Mon Dec 10 10:12:02 2018: cache: 1.3.6.1.4.1.9.9.91.1.2.1
Mon Dec 10 10:12:02 2018: get_table $VAR1 = {
  '-baseoid' => '1.3.6.1.4.1.9.9.91.1.2.1',
  '-maxrepetitions' => 10
};

Mon Dec 10 10:12:21 2018: get_table returned 940 oids
Mon Dec 10 10:12:21 2018: get_matching_oids $VAR1 = {
  '-columns' => [
    '1.3.6.1.4.1.9.9.91.1.2.1'
  ]
};

Mon Dec 10 10:12:21 2018: get_matching_oids returns 940 from 1289 oids
Mon Dec 10 10:12:21 2018: i know package Monitoring::GLPlugin::SNMP::MibsAndOids::CISCOENTITYSENSORMIB
Mon Dec 10 10:12:21 2018: get_snmp_table_objects default returns 188 entries
Mon Dec 10 10:12:21 2018: i know package Monitoring::GLPlugin::SNMP::MibsAndOids::ENTITYMIB
Mon Dec 10 10:12:21 2018: get_snmp_table_objects ENTITY-MIB entPhysicalTable
Mon Dec 10 10:12:21 2018: i know package Monitoring::GLPlugin::SNMP::MibsAndOids::ENTITYMIB
Mon Dec 10 10:12:21 2018: cache: 1.3.6.1.2.1.47.1.1.1
Mon Dec 10 10:12:21 2018: get_table $VAR1 = {
  '-baseoid' => '1.3.6.1.2.1.47.1.1.1',
  '-maxrepetitions' => 10
};

Mon Dec 10 10:12:24 2018: AUTOLOAD Classes::Cisco::NXOS::nagios_exit

UNKNOWN - check_nwc_health timed out after 30 seconds
Noname2244 commented 5 years ago

wow, the replay was just really really slow, I increased timeout to 80s and now it gets the data.

I just assumed that 30s would be enough when increasing the timer :)

solution: don't assume anything.