UDMB sporadic data after update to 1.3

[pax0707]

Anything I can check here? AP and switch are reporting fine but UDN (base, not pro one) does not play nice.

[patricegautier]

I just updated one to 1.3 as well.. it's playing nice here so far.

Anything in the zabbix server logs? when mca-dump-short goes not supported on the controller host like above, what's the error message?

[edit] mine is a pro, not a base so that could be a difference

[pax0707]

838333:20221220:202552.874 Failed to execute command "/usr/lib/zabbix/externalscripts/mca-dump-short.sh '-d' '172.16.1.1' '-u' 'root' '-i' '/usr/lib/zabbix/externalscripts/zb_id_rsa' '-t' 'UDMP' '-p' '{$UNIFI_SSHPASS_PASSWORD_PATH}' '-o' '30'": Timeout while executing a shell script.
838327:20221220:202652.700 Failed to execute command "/usr/lib/zabbix/externalscripts/mca-dump-short.sh '-d' '172.16.1.1' '-u' 'root' '-i' '/usr/lib/zabbix/externalscripts/zb_id_rsa' '-t' 'UDMP' '-p' '{$UNIFI_SSHPASS_PASSWORD_PATH}' '-o' '30'": Timeout while executing a shell script.
838334:20221220:202752.695 Failed to execute command "/usr/lib/zabbix/externalscripts/mca-dump-short.sh '-d' '172.16.1.1' '-u' 'root' '-i' '/usr/lib/zabbix/externalscripts/zb_id_rsa' '-t' 'UDMP' '-p' '{$UNIFI_SSHPASS_PASSWORD_PATH}' '-o' '30'": Timeout while executing a shell script.
838323:20221220:202852.812 Failed to execute command "/usr/lib/zabbix/externalscripts/mca-dump-short.sh '-d' '172.16.1.1' '-u' 'root' '-i' '/usr/lib/zabbix/externalscripts/zb_id_rsa' '-t' 'UDMP' '-p' '{$UNIFI_SSHPASS_PASSWORD_PATH}' '-o' '30'": Timeout while executing a shell script.
838324:20221220:202952.658 Failed to execute command "/usr/lib/zabbix/externalscripts/mca-dump-short.sh '-d' '172.16.1.1' '-u' 'root' '-i' '/usr/lib/zabbix/externalscripts/zb_id_rsa' '-t' 'UDMP' '-p' '{$UNIFI_SSHPASS_PASSWORD_PATH}' '-o' '30'": Timeout while executing a shell script.
838334:20221220:203052.505 Failed to execute command "/usr/lib/zabbix/externalscripts/mca-dump-short.sh '-d' '172.16.1.1' '-u' 'root' '-i' '/usr/lib/zabbix/externalscripts/zb_id_rsa' '-t' 'UDMP' '-p' '{$UNIFI_SSHPASS_PASSWORD_PATH}' '-o' '30'": Timeout while executing a shell script.
838323:20221220:203152.607 Failed to execute command "/usr/lib/zabbix/externalscripts/mca-dump-short.sh '-d' '172.16.1.1' '-u' 'root' '-i' '/usr/lib/zabbix/externalscripts/zb_id_rsa' '-t' 'UDMP' '-p' '{$UNIFI_SSHPASS_PASSWORD_PATH}' '-o' '30'": Timeout while executing a shell script.
838311:20221220:203253.214 item "UniFi Controller:mca-dump-short.sh["-d","{HOST.CONN}", "-u", "root", "-i", "{$UNIFI_SSH_PRIV_KEY_PATH}", "-t", "UDMP", "-p", "{$UNIFI_SSHPASS_PASSWORD_PATH}", "-o", "{$UNIFI_CHECK_TIMEOUT}" ]" became supported
838323:20221220:203352.930 Failed to execute command "/usr/lib/zabbix/externalscripts/mca-dump-short.sh '-d' '172.16.1.1' '-u' 'root' '-i' '/usr/lib/zabbix/externalscripts/zb_id_rsa' '-t' 'UDMP' '-p' '{$UNIFI_SSHPASS_PASSWORD_PATH}' '-o' '30'": Timeout while executing a shell script.
838311:20221220:203353.103 item "UniFi Controller:mca-dump-short.sh["-d","{HOST.CONN}", "-u", "root", "-i", "{$UNIFI_SSH_PRIV_KEY_PATH}", "-t", "UDMP", "-p", "{$UNIFI_SSHPASS_PASSWORD_PATH}", "-o", "{$UNIFI_CHECK_TIMEOUT}" ]" became not supported: Timeout while executing a shell script.
838327:20221220:203452.873 Failed to execute command "/usr/lib/zabbix/externalscripts/mca-dump-short.sh '-d' '172.16.1.1' '-u' 'root' '-i' '/usr/lib/zabbix/externalscripts/zb_id_rsa' '-t' 'UDMP' '-p' '{$UNIFI_SSHPASS_PASSWORD_PATH}' '-o' '30'": Timeout while executing a shell script.
838327:20221220:203552.767 Failed to execute command "/usr/lib/zabbix/externalscripts/mca-dump-short.sh '-d' '172.16.1.1' '-u' 'root' '-i' '/usr/lib/zabbix/externalscripts/zb_id_rsa' '-t' 'UDMP' '-p' '{$UNIFI_SSHPASS_PASSWORD_PATH}' '-o' '30'": Timeout while executing a shell script.
838323:20221220:203652.702 Failed to execute command "/usr/lib/zabbix/externalscripts/mca-dump-short.sh '-d' '172.16.1.1' '-u' 'root' '-i' '/usr/lib/zabbix/externalscripts/zb_id_rsa' '-t' 'UDMP' '-p' '{$UNIFI_SSHPASS_PASSWORD_PATH}' '-o' '30'": Timeout while executing a shell script.
838333:20221220:203752.631 Failed to execute command "/usr/lib/zabbix/externalscripts/mca-dump-short.sh '-d' '172.16.1.1' '-u' 'root' '-i' '/usr/lib/zabbix/externalscripts/zb_id_rsa' '-t' 'UDMP' '-p' '{$UNIFI_SSHPASS_PASSWORD_PATH}' '-o' '30'": Timeout while executing a shell script.
838324:20221220:203852.546 Failed to execute command "/usr/lib/zabbix/externalscripts/mca-dump-short.sh '-d' '172.16.1.1' '-u' 'root' '-i' '/usr/lib/zabbix/externalscripts/zb_id_rsa' '-t' 'UDMP' '-p' '{$UNIFI_SSHPASS_PASSWORD_PATH}' '-o' '30'": Timeout while executing a shell script.
838327:20221220:203952.481 Failed to execute command "/usr/lib/zabbix/externalscripts/mca-dump-short.sh '-d' '172.16.1.1' '-u' 'root' '-i' '/usr/lib/zabbix/externalscripts/zb_id_rsa' '-t' 'UDMP' '-p' '{$UNIFI_SSHPASS_PASSWORD_PATH}' '-o' '30'": Timeout while executing a shell script.
838323:20221220:204052.424 Failed to execute command "/usr/lib/zabbix/externalscripts/mca-dump-short.sh '-d' '172.16.1.1' '-u' 'root' '-i' '/usr/lib/zabbix/externalscripts/zb_id_rsa' '-t' 'UDMP' '-p' '{$UNIFI_SSHPASS_PASSWORD_PATH}' '-o' '30'": Timeout while executing a shell script.
838327:20221220:204152.339 Failed to execute command "/usr/lib/zabbix/externalscripts/mca-dump-short.sh '-d' '172.16.1.1' '-u' 'root' '-i' '/usr/lib/zabbix/externalscripts/zb_id_rsa' '-t' 'UDMP' '-p' '{$UNIFI_SSHPASS_PASSWORD_PATH}' '-o' '30'": Timeout while executing a shell script.
838334:20221220:204252.363 Failed to execute command "/usr/lib/zabbix/externalscripts/mca-dump-short.sh '-d' '172.16.1.1' '-u' 'root' '-i' '/usr/lib/zabbix/externalscripts/zb_id_rsa' '-t' 'UDMP' '-p' '{$UNIFI_SSHPASS_PASSWORD_PATH}' '-o' '30'": Timeout while executing a shell script.
838324:20221220:204352.167 Failed to execute command "/usr/lib/zabbix/externalscripts/mca-dump-short.sh '-d' '172.16.1.1' '-u' 'root' '-i' '/usr/lib/zabbix/externalscripts/zb_id_rsa' '-t' 'UDMP' '-p' '{$UNIFI_SSHPASS_PASSWORD_PATH}' '-o' '30'": Timeout while executing a shell script.

When I run "sudo -u zabbix ./mca-dump-short.sh -i ./zb_id_rsa -u root -v -d 172.16.1.1 -t UDMP" it runs., About 2-3 seconds to execute. Tried raising timeout to 30, still timeouts.

[patricegautier]

mm.. have you tried running exactly the same command:

/usr/lib/zabbix/externalscripts/mca-dump-short.sh '-d' '172.16.1.1' '-u' 'root' '-i' '/usr/lib/zabbix/externalscripts/zb_id_rsa' '-t' 'UDMP' '-p' '{$UNIFI_SSHPASS_PASSWORD_PATH}' '-o' '30'

?

anything in /tmp/mcaDumpShort.err on the zabbix server?

[pax0707]

Output: https://gist.github.com/pax0707/701ae085daf2b61a99f23d6fd60b2cfb

tmp dir is full of empty mca*.err files.

-rw-rw-r--  1 zabbix zabbix    0 Dec 22 00:18 mca-14284.err
-rw-rw-r--  1 zabbix zabbix    0 Dec 22 00:17 mca-18696.err
-rw-rw-r--  1 zabbix zabbix    0 Dec 22 00:20 mca-19170.err
-rw-rw-r--  1 zabbix zabbix    0 Dec 22 00:25 mca-22353.err
-rw-rw-r--  1 zabbix zabbix    0 Dec 22 00:28 mca-22716.err
-rw-rw-r--  1 zabbix zabbix    0 Dec 22 00:23 mca-23358.err
-rw-rw-r--  1 zabbix zabbix    0 Dec 22 00:26 mca-27412.err
-rw-rw-r--  1 zabbix zabbix    0 Dec 22 00:22 mca-30038.err
-rw-rw-r--  1 zabbix zabbix    0 Dec 22 00:15 mca-30896.err
-rw-rw-r--  1 zabbix zabbix    0 Dec 22 00:16 mca-31093.err
-rw-rw-r--  1 zabbix zabbix    0 Dec 22 00:21 mca-4403.err
-rw-rw-r--  1 zabbix zabbix    0 Dec 22 00:24 mca-4862.err
-rw-rw-r--  1 zabbix zabbix    0 Dec 22 00:19 mca-5992.err
-rw-rw-r--  1 zabbix zabbix    0 Dec 22 00:27 mca-8796.err

[pax0707]

pi@rpi3:/tmp $ cat mcaDumpShort.err Thu 22 Dec 2022 12:35:49 AM CET 172.16.1.1

Thu 22 Dec 2022 12:35:49 AM CET 172.16.1.89

Thu 22 Dec 2022 12:36:13 AM CET 172.16.1.97

Thu 22 Dec 2022 12:36:17 AM CET 172.16.1.1

Thu 22 Dec 2022 12:36:19 AM CET 172.16.1.1

Thu 22 Dec 2022 12:36:35 AM CET 172.16.1.1

Thu 22 Dec 2022 12:36:41 AM CET 172.16.1.1

Thu 22 Dec 2022 12:36:49 AM CET 172.16.1.1

Thu 22 Dec 2022 12:36:50 AM CET 172.16.1.89

Thu 22 Dec 2022 12:37:13 AM CET 172.16.1.97

Thu 22 Dec 2022 12:37:48 AM CET 172.16.1.1

Thu 22 Dec 2022 12:37:50 AM CET 172.16.1.89

Thu 22 Dec 2022 12:37:57 AM CET 172.16.1.1

Thu 22 Dec 2022 12:37:58 AM CET 172.16.1.1

Thu 22 Dec 2022 12:38:13 AM CET 172.16.1.97

Thu 22 Dec 2022 12:38:31 AM CET 172.16.1.1

Thu 22 Dec 2022 12:38:49 AM CET 172.16.1.1

Thu 22 Dec 2022 12:38:49 AM CET 172.16.1.89

Thu 22 Dec 2022 12:39:07 AM CET 172.16.1.1

Thu 22 Dec 2022 12:39:13 AM CET 172.16.1.97

pi@rpi3:/tmp $

[pax0707]

This is Fed up... Restarted UDM and everything is working....

[pax0707]

Didn't last....

"Timeout while executing a shell script."

It's very annoying to troubleshoot since it works for a bit, starts to die, and then mostly doesn't work.

[patricegautier]

What is your overall TIMEOUT value set in the Zabbix conf?Sent from my iPadOn Dec 22, 2022, at 8:44 AM, pax0707 @.***> wrote: Didn't last.... "Timeout while executing a shell script."

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: @.***>

[patricegautier]

hi @pax0707  - could I get you to try with an updated mcaDumpShort.sh and templates? the situation around timeouts has been improved and there also should be more info in /tmp/mcaDumpShort.err on the zabbixXerver

[pax0707]

pi@rpi3:~ $ cat /tmp/mcaDumpShort.err 
----------------------------------
Thu 12 Jan 2023 09:17:15 PM CET 172.16.1.97
   { "at":"21:17:15", "r":"please install expect to run SWITCH_DISCOVERY", "device":"172.16.1.97", "mcaDumpError":"Error" }\n  

Gonna deal with this while at it.

[pax0707]

Working so far, monitoring...

6 hours later and still ok. Gonna close this tomorrow if it continues to be.

[pax0707]

Working fine.