search

jumation / bwutil

Event script which triggers a syslog message if interface(s) bandwidth utilization threshold is exceeded
GNU General Public License v3.0
0 stars 0 forks source link

Where find logs for alert #1

Open valerarar opened 3 years ago

valerarar commented 3 years ago 
Jan 19 16:51:53 test-switch: mgd[23334]: UI_AUTH_EVENT: Authenticated user '(null)' assigned to class 'super-user'
Jan 19 16:51:53 test-switch: mgd[23334]: UI_LOGIN_EVENT: User 'root' login, class 'super-user' [23334], ssh-connection '', client-mode 'junoscript'
Jan 19 16:51:53 test-switch: mgd[23334]: UI_JUNOSCRIPT_CMD: User 'root' used JUNOScript client to run command 'file-list path=/var/tmp/bwutil/'
Jan 19 16:51:53 test-switch: mgd[23334]: UI_CHILD_START: Starting child '/bin/sh'
Jan 19 16:51:53 test-switch: mgd[23334]: UI_CHILD_STATUS: Cleanup child '/bin/sh', PID 23335, status 0
Jan 19 16:51:53 test-switch: mgd[23334]: UI_JUNOSCRIPT_CMD: User 'root' used JUNOScript client to run command 'get-interface-information interface-name=et-0/0/8.0'
Jan 19 16:51:53 test-switch: mgd[23334]: UI_CHILD_START: Starting child '/sbin/ifinfo'
Jan 19 16:51:53 test-switch: mgd[23334]: UI_CHILD_STATUS: Cleanup child '/sbin/ifinfo', PID 23336, status 0
Jan 19 16:51:53 test-switch: mgd[23334]: UI_JUNOSCRIPT_CMD: User 'root' used JUNOScript client to run command 'file-list path=/var/tmp/bwutil/octets_IfIndex554'
Jan 19 16:51:53 test-switch: mgd[23334]: UI_CHILD_START: Starting child '/bin/sh'
Jan 19 16:51:53 test-switch: mgd[23334]: UI_CHILD_STATUS: Cleanup child '/bin/sh', PID 23337, status 0
Jan 19 16:51:53 test-switch: mgd[23334]: UI_JUNOSCRIPT_CMD: User 'root' used JUNOScript client to run command 'file-get filename=/var/tmp/bwutil/octets_IfIndex554 encoding=ascii'
Jan 19 16:51:53 test-switch: mgd[23334]: UI_JUNOSCRIPT_CMD: User 'root' used JUNOScript client to run command 'get-system-uptime-information'
Jan 19 16:51:53 test-switch: mgd[23338]: UI_CMDLINE_READ_LINE: User '(authentication in progress)', command 'rpc command .set auth environment user root logname root host test-switch agent mgd current-directory / pid 23334 ppid 23333 '
Jan 19 16:51:53 test-switch: mgd[23338]: UI_JUNOSCRIPT_CMD: User '(authentication in progress)' used JUNOScript client to run command 'request-authentication user=root logname=root host=test-switch agent=mgd current-directory=/ pid=23334 ppid=23333'
Jan 19 16:51:53 test-switch: mgd[23338]: UI_AUTH_EVENT: Authenticated user 'root' assigned to class 'super-user'
Jan 19 16:51:53 test-switch: mgd[23338]: UI_LOGIN_EVENT: User 'root' login, class 'super-user' localre[23338], ssh-connection '', client-mode 'junoscript'
Jan 19 16:51:53 test-switch: mgd[23338]: UI_CMDLINE_READ_LINE: User 'root', command 'command rpc rpc command show system uptime no-forwarding '
Jan 19 16:51:53 test-switch: mgd[23338]: UI_JUNOSCRIPT_CMD: User 'root' used JUNOScript client to run command 'get-system-uptime-information'
Jan 19 16:51:53 test-switch: mgd[23338]: UI_CHILD_START: Starting child '/usr/libexec/ui/ntpsync'
Jan 19 16:51:53 test-switch: mgd[23338]: UI_CHILD_STATUS: Cleanup child '/usr/libexec/ui/ntpsync', PID 23340, status 0
Jan 19 16:51:53 test-switch: mgd[23338]: UI_CHILD_START: Starting child '/usr/libexec/ui/uptime'
Jan 19 16:51:53 test-switch: mgd[23338]: UI_CHILD_STATUS: Cleanup child '/usr/libexec/ui/uptime', PID 23346, status 0
Jan 19 16:51:53 test-switch: mgd[23338]: UI_JUNOSCRIPT_CMD: User 'root' used JUNOScript client to run command 'request-end-session'
Jan 19 16:51:53 test-switch: mgd[23338]: UI_LOGOUT_EVENT: User 'root' logout
Jan 19 16:51:53 test-switch: mgd[23334]: UI_JUNOSCRIPT_CMD: User 'root' used JUNOScript client to run command 'get-snmp-object snmp-object-name=ifHCInOctets.554'
Jan 19 16:51:53 test-switch: mgd[23334]: UI_JUNOSCRIPT_CMD: User 'root' used JUNOScript client to run command 'get-snmp-object snmp-object-name=ifHCOutOctets.554'
Jan 19 16:51:53 test-switch: mgd[23334]: UI_JUNOSCRIPT_CMD: User 'root' used JUNOScript client to run command 'get-snmp-object snmp-object-name=ifHighSpeed.554'
Jan 19 16:51:53 test-switch: mgd[23334]: UI_JUNOSCRIPT_CMD: User 'root' used JUNOScript client to run command 'file-put filename=/var/tmp/bwutil/octets_IfIndex554 permission=0644 encoding=ascii delete-if-exist file-contents=/var/tmp/bwutil/octets_IfIndex554.kIFwb'
Jan 19 16:51:53 test-switch: mgd[23334]: UI_JUNOSCRIPT_CMD: User 'root' used JUNOScript client to run command 'get-interface-information interface-name=et-0/0/16.0'
Jan 19 16:51:53 test-switch: mgd[23334]: UI_CHILD_START: Starting child '/sbin/ifinfo'
Jan 19 16:51:53 test-switch: mgd[23334]: UI_CHILD_STATUS: Cleanup child '/sbin/ifinfo', PID 23354, status 0
Jan 19 16:51:53 test-switch: mgd[23334]: UI_JUNOSCRIPT_CMD: User 'root' used JUNOScript client to run command 'file-list path=/var/tmp/bwutil/octets_IfIndex538'
Jan 19 16:51:53 test-switch: mgd[23334]: UI_CHILD_START: Starting child '/bin/sh'
Jan 19 16:51:53 test-switch: mgd[23334]: UI_CHILD_STATUS: Cleanup child '/bin/sh', PID 23355, status 0
Jan 19 16:51:53 test-switch: mgd[23334]: UI_JUNOSCRIPT_CMD: User 'root' used JUNOScript client to run command 'file-get filename=/var/tmp/bwutil/octets_IfIndex538 encoding=ascii'
Jan 19 16:51:53 test-switch: mgd[23334]: UI_JUNOSCRIPT_CMD: User 'root' used JUNOScript client to run command 'get-system-uptime-information'
Jan 19 16:51:53 test-switch: mgd[23356]: UI_CMDLINE_READ_LINE: User '(authentication in progress)', command 'rpc command .set auth environment user root logname root host test-switch agent mgd current-directory / pid 23334 ppid 23333 '
Jan 19 16:51:53 test-switch: mgd[23356]: UI_JUNOSCRIPT_CMD: User '(authentication in progress)' used JUNOScript client to run command 'request-authentication user=root logname=root host=test-switch agent=mgd current-directory=/ pid=23334 ppid=23333'
Jan 19 16:51:53 test-switch: mgd[23356]: UI_AUTH_EVENT: Authenticated user 'root' assigned to class 'super-user'
Jan 19 16:51:53 test-switch: mgd[23356]: UI_LOGIN_EVENT: User 'root' login, class 'super-user' localre[23356], ssh-connection '', client-mode 'junoscript'
Jan 19 16:51:53 test-switch: mgd[23356]: UI_CMDLINE_READ_LINE: User 'root', command 'command rpc rpc command show system uptime no-forwarding '
Jan 19 16:51:53 test-switch: mgd[23356]: UI_JUNOSCRIPT_CMD: User 'root' used JUNOScript client to run command 'get-system-uptime-information'
Jan 19 16:51:53 test-switch: mgd[23356]: UI_CHILD_START: Starting child '/usr/libexec/ui/ntpsync'
Jan 19 16:51:53 test-switch: mgd[23356]: UI_CHILD_STATUS: Cleanup child '/usr/libexec/ui/ntpsync', PID 23358, status 0
Jan 19 16:51:53 test-switch: mgd[23356]: UI_CHILD_START: Starting child '/usr/libexec/ui/uptime'
Jan 19 16:51:53 test-switch: mgd[23356]: UI_CHILD_STATUS: Cleanup child '/usr/libexec/ui/uptime', PID 23364, status 0
Jan 19 16:51:54 test-switch: mgd[23356]: UI_JUNOSCRIPT_CMD: User 'root' used JUNOScript client to run command 'request-end-session'
Jan 19 16:51:54 test-switch: mgd[23356]: UI_LOGOUT_EVENT: User 'root' logout
Jan 19 16:51:54 test-switch: mgd[23334]: UI_JUNOSCRIPT_CMD: User 'root' used JUNOScript client to run command 'get-snmp-object snmp-object-name=ifHCInOctets.538'
Jan 19 16:51:54 test-switch: mgd[23334]: UI_JUNOSCRIPT_CMD: User 'root' used JUNOScript client to run command 'get-snmp-object snmp-object-name=ifHCOutOctets.538'
Jan 19 16:51:54 test-switch: mgd[23334]: UI_JUNOSCRIPT_CMD: User 'root' used JUNOScript client to run command 'get-snmp-object snmp-object-name=ifHighSpeed.538'
Jan 19 16:51:54 test-switch: mgd[23334]: UI_JUNOSCRIPT_CMD: User 'root' used JUNOScript client to run command 'file-put filename=/var/tmp/bwutil/octets_IfIndex538 permission=0644 encoding=ascii delete-if-exist file-contents=/var/tmp/bwutil/octets_IfIndex538.YoZqq'

And

% ls -la /var/tmp/bwutil/
total 40
drwxr-xr-x  2 root  wheel   512 Jan 19 16:37 .
drwxrwxrwx  8 root  wheel  1024 Jan 12 15:06 ..
-rw-r--r--  1 root  wheel     3 Jan 19 16:36 octets_IfIndex
-rw-r--r--  1 root  wheel    27 Jan 19 16:37 octets_IfIndex538
-rw-r--r--  1 root  wheel    26 Jan 19 16:37 octets_IfIndex554
% cat /var/tmp/bwutil/octets_IfIndex554
,257559126,860562440,25000% cat /var/tmp/bwutil/octets_IfIndex554
,257560649,860569031,25000% cat /var/tmp/bwutil/octets_IfIndex554
,257562028,860575622,25000% cat /var/tmp/bwutil/octets_IfIndex554
,257562028,860575622,25000% cat /var/tmp/bwutil/octets_IfIndex554

But I'm not sure if an alert is triggered for an exceeding event, I set 1%

                    <junos:comment> "/* Bandwidth utilization threshold in
                                        percentages of port bandwidth */";
                    <arguments> {
                        <name> "threshold";
                        <value> "1";
                    }

Model: qfx5120-48y-8c Junos: 19.4R3.11 flex

tonusoo commented 3 years ago

Where find logs for alert

This depends on your logging configuration. The script sends the alerts with info severity level under external facility. For example, if you are logging into the messages file, then the configuration below works:

root@CE-A> show configuration system syslog file messages
any notice;
authorization info;
external info;
explicit-priority;

root@CE-A>

As seen above, for the info and higher severity level messages for facility external are logged into the messages(/var/log/messages) file. Example of an alert message can be seen below:

root@CE-A> show log messages | match utilization | last 1
Jan 20 15:47:55  CE-A cscript.crypto: %EXTERNAL-6: bwutil.slax: ingress & egress bandwidth utilization exceeds 10% on interface ge-0/0/0.0

root@CE-A>

However, the content of your /var/tmp/bwutil/octets_IfIndex554 file is incomplete. The first field(epoch timestamp) is empty. Can you please post the output of show system uptime | display xml? Or did you copy a partial output of the /var/tmp/bwutil/octets_IfIndex554 file? In addition, as your et-0/0/8.0 interface is a 25Gbps one, then you need to generate at least 250Mbps of traffic in either direction in order to trigger the alert message if you set the threshold to 1%.

valerarar commented 3 years ago

Thanks for the answer Syslog:

show configuration system syslog | display set 
set system syslog archive size 10m
set system syslog archive files 5
set system syslog archive world-readable
set system syslog user * any emergency
set system syslog host 172.16.xxx any any
set system syslog host 172.16.xxx authorization any
set system syslog host 172.16.xxx external info
set system syslog host 172.16.xxx log-prefix ds-leaf-test
set system syslog file messages any notice
set system syslog file messages authorization any
set system syslog file messages external info
set system syslog file messages archive world-readable
set system syslog file messages explicit-priority

show log messages| match utilization and in syslog host - empty

show system uptime | display xml 
<rpc-reply xmlns:junos="http://xml.juniper.net/junos/19.4R0/junos">
    <multi-routing-engine-results>

        <multi-routing-engine-item>

            <re-name>localre</re-name>

            <system-uptime-information xmlns="http://xml.juniper.net/junos/19.4R0/junos">
                <current-time>
                    <date-time junos:seconds="1611236300">2021-01-21 16:38:20 MSK</date-time>
                </current-time>
                <time-source> NTP CLOCK </time-source>
                <system-booted-time>
                    <date-time junos:seconds="1603208029">2020-10-20 18:33:49 MSK</date-time>
                    <time-length junos:seconds="8028271">13w1d 22:04</time-length>
                </system-booted-time>
                <protocols-started-time>
                    <date-time junos:seconds="1603208062">2020-10-20 18:34:22 MSK</date-time>
                    <time-length junos:seconds="8028238">13w1d 22:03</time-length>
                </protocols-started-time>
                <last-configured-time>
                    <date-time junos:seconds="1611162160">2021-01-20 20:02:40 MSK</date-time>
                    <time-length junos:seconds="74140">20:35:40</time-length>
                    <user>admin</user>
                </last-configured-time>
                <uptime-information>
                    <date-time junos:seconds="1611236300">4:38PM</date-time>
                    <up-time junos:seconds="8028300">92 days, 22:05</up-time>
                    <active-user-count junos:format="2 users">2</active-user-count>
                    <load-average-1>0.28</load-average-1>
                    <load-average-5>0.32</load-average-5>
                    <load-average-15>0.27</load-average-15>
                    <user-table></user-table>
                </uptime-information>
            </system-uptime-information>
        </multi-routing-engine-item>

    </multi-routing-engine-results>
    <cli>
        <banner>{master:0}</banner>
    </cli>
</rpc-reply>
cat /var/tmp/bwutil/octets_IfIndex554
,258260945,879482159,25000%

Port is loaded with 1Gbit/s one way

tonusoo commented 3 years ago

Looks like the QFX series switches and probably some other platforms like SRX series firewalls in cluster mode return the <system-uptime-information> under the <multi-routing-engine-item> hierarchy. I updated the script. Please test with the latest revision of bwutil.slax.