search

ImTheLucKyOne / check_mk_emcunity

Check_MK Extensions for Dell EMC Unity Storages
GNU General Public License v3.0
5 stars 8 forks source link

monitors for one of the SP's goes "UNKNOWN" frequently #25

Open lravelo opened 3 years ago

lravelo commented 3 years ago

I'm running this against a Unity 450F on Check_MK Enterprise v1.6.0p22 and frequently the service for CPU utilization and Temperature goes unknown for about 15 minutes or so. This happens basically every hour from what I can see. How can I troubleshoot this to identify what the cause is?

ImTheLucKyOne commented 3 years ago

The temp and cpu checks get their values from the uemcli metrics output. So what you can try first is to run the uemcli manually when the issue occurs:

# uemcli -d HOST -u USER -p PASSWORD /metrics/value/rt -path sp.*.platform.storageProcessorTemperature show -interval 5 -count 1 -output csv # uemcli -d HOST -u USER -p PASSWORD /metrics/value/rt -path sp.*.cpu.summary.utilization show -interval 5 -count 1 -output csv

Depending on that output while the cmk services are unknown you can investigate further.

lravelo commented 3 years ago

Thanks for your response. This is the output I get when I run these commands:

./uemcli: error while loading shared libraries: libboost_thread-gcc41-mt-1_43.so.1.43.0: cannot open shared object file: No such file or directory

EDIT:

I used the uemcli.sh script instead but it's giving me a permissions issue even though this is the same account that I use to monitor and is working fine:

Operation failed. Error code: 0x6000001 You do not have access to perform the requested operation. (Error Code:0x6000001)

EDIT 2:

For some reason the commands started working again after resetting the password. Next time the alarms come up in checkmk I will test the commands you've mentioned.

lravelo commented 3 years ago

after resetting the password, everything went from bad to worse. Now I can no longer monitor anything. I've deleted everything in the ./omd/sites/<monitoring_site>/.emc and reran uemcli -d HOST -u user -securePassword -sslPolicy store /sys/general show to download the certs, deleted the host from check_mk, deleted the rule for the special agent and nothing. The only thing I get back is results from the emcunity_hostcons check and it comes back as unknown.

EDIT: Turns out this was related to some character in the new password. It's reporting again. Will check again once the SP's cpu and temp services come back unknown.

lravelo commented 3 years ago

This is the only thing being returned when it happens: image

ImTheLucKyOne commented 3 years ago

Well, that looks like it should.

Next step I would do, just to double check, run the agent manually while the UNKNOWN issue occurs and see what it reports back.