Closed hakong closed 7 years ago
You have changed output format from nagios to:
service_perfdata_file_template=DATATYPE::SERVICEPERFDATA\tTIMET::$TIMET$\tHOSTNAME::$HOSTNAME$\tSERVICEDESC::$SERVICEDESC$\tSERVICEPERFDATA::$SERVICEPERFDATA$\tSERVICECHECKCOMMAND::$SERVICECHECKCOMMAND$\tHOSTSTATE::$HOSTSTATE$\tHOSTSTATETYPE::$HOSTSTATETYPE$\tSERVICESTATE::$SERVICESTATE$\tSERVICESTATETYPE::$SERVICESTATETYPE$
but the format nagios2influx expects remains unchanged as
perfformat=time tag=hostname label tag=state void void void void void data
these two don't match! Perhaps nagios2influx should be more unforgiving, and report errors if time doesn't look like an epoch time, for example.
I don't know why you have altered the output from nagios, but nagios2influx doesn't support the trim::data format you have used for your fields. I suggest you use the default service_perfdata_file_template setting and get data flowing with one of the example formats, before experimenting with custom formats.
Ah, I see now. Thanks. I changed back to the default format, restarted naemon, dropped the database "naemon" and created it again, but nagios2influx isn't importing anything now.
[root@localhost bin]# ./nagios2influx -n -d 127
[ 8] nagios2influx: --
[ 8] nagios2influx: timereplacement=Time
[ 8] nagios2influx: global_prefix=naemon
[ 8] nagios2influx: output=http://localhost:8086
[ 8] nagios2influx: timefactor=9
[ 8] nagios2influx: username=naemon
[ 8] nagios2influx: database=naemon
[ 8] nagios2influx: perfdelimiter=\t
[ 8] nagios2influx: password=naemon
[ 8] nagios2influx: separator=
[ 8] nagios2influx: debug=127
[ 8] nagios2influx: perfdata=/var/lib/naemon/service-perfdata
[ 8] nagios2influx: perfformat=time tag=hostname label tag=state void void void void void data
[ 8] nagios2influx: --
[ 16] nagios2influx: Delim:\t.
[ 16] nagios2influx: Column 0:time.
[ 16] nagios2influx: Column 1:tag=hostname.
[ 16] nagios2influx: Column 2:label.
[ 16] nagios2influx: Column 3:tag=state.
[ 16] nagios2influx: Column 4:void.
[ 16] nagios2influx: Column 5:void.
[ 16] nagios2influx: Column 6:void.
[ 16] nagios2influx: Column 7:void.
[ 16] nagios2influx: Column 8:void.
[ 16] nagios2influx: Column 9:data.
[ 4] nagios2influx: Reading /var/lib/naemon/service-perfdata ...
[ 32] nagios2influx: [SERVICEPERFDATA] 1496242724 localhost Apache Statistics 0.070 0.288 (No output on stdout) stderr: execvp(/usr/lib64/nagios/plugins/check_apachestatus.pl, ...) failed. errno is 2: No such file or directory
[ 32] nagios2influx: [SERVICEPERFDATA] 1496242724 localhost HTTP localhost 0.081 0.288 HTTP WARNING: HTTP/1.1 403 Forbidden - 5241 bytes in 0.064 second response time time=0.064434s;2.000000;10.000000;0.000000 size=5241B;;;0
[ 32] nagios2influx: [SERVICEPERFDATA] 1496242724 google.com HTTPS SSL Certificate google.com 0.167 0.287 OK - Certificate 'google.com' will expire on 2017-08-10 11:39 +0000/GMT.
[ 32] nagios2influx: [SERVICEPERFDATA] 1496242724 localhost HTTPS SSL Certificate localhost 0.194 0.288 OK - Certificate 'localhost.localdomain' will expire on 2018-05-30 14:18 +0000/GMT.
[ 32] nagios2influx: [SERVICEPERFDATA] 1496242724 localhost HTTPS localhost 0.200 0.288 HTTP WARNING: HTTP/1.1 403 Forbidden - 5241 bytes in 0.150 second response time time=0.149761s;2.000000;10.000000;0.000000 size=5241B;;;0
[ 32] nagios2influx: [SERVICEPERFDATA] 1496242724 localhost SSH Connectivity 0.217 0.288 SSH OK - OpenSSH_6.6.1 (protocol 2.0) time=0.203221s;;;0.000000;10.000000
[ 32] nagios2influx: [SERVICEPERFDATA] 1496242724 localhost Reverse Lookup 0.251 0.288 DNS CRITICAL - expected 'localhost.' but got 'name = localhost.'
[ 32] nagios2influx: [SERVICEPERFDATA] 1496242724 localhost Forward Lookup 0.258 0.288 DNS OK: 0.242 seconds response time. localhost returns 127.0.0.1 time=0.242316s;;;0.000000
[ 32] nagios2influx: [SERVICEPERFDATA] 1496242724 localhost Nagios Ghost Services 0.269 0.288 Unable to locate logfile for parsing
[ 32] nagios2influx: [SERVICEPERFDATA] 1496242724 localhost Nagios config up-to-date 0.339 0.288 Unknown - Error running script: [Errno 2] No such file or directory: '/etc/nagios/nagios.cfg'
[ 32] nagios2influx: [SERVICEPERFDATA] 1496242724 localhost Disk Usage 0.343 0.288 DISK OK ... /dev/shm=0B;460763136;486361088;0;511959040 /=3021668352B;7717729075;8146491802;0;8575254528 /dev=0B;450857779;475905434;0;500953088 /sys/fs/cgroup=0B;460763136;486361088;0;511959040 /run/user/0=0B;92152627;97272218;0;102391808 /boot=192581632B;956930458;1010093261;0;1063256064 /run=7016448B;460763136;486361088;0;511959040
[ 32] nagios2influx: [SERVICEPERFDATA] 1496242724 localhost Nagios Configuration 0.365 0.288 Could not validate Nagios configuration. warnings=0 errors=1
[ 32] nagios2influx: [SERVICEPERFDATA] 1496242724 localhost Process crond 0.421 0.288 PROCS OK: 1 process with command name 'crond' procs=1;1:;:10;0; | procs=1
[ 32] nagios2influx: [SERVICEPERFDATA] 1496242724 localhost Process nagios 0.429 0.288 PROCS WARNING: 0 processes with command name 'nagios' procs=0;1:;:10;0; | procs=0
[ 32] nagios2influx: [SERVICEPERFDATA] 1496242724 localhost Load 0.455 0.288 OK - load average: 0.10, 0.21, 0.23 load1=0.100;15.000;30.000;0; load5=0.210;15.000;30.000;0; load15=0.230;15.000;30.000;0;
[ 32] nagios2influx: [SERVICEPERFDATA] 1496242724 localhost Running Processes 0.479 0.288 PROCS OK: 165 processes procs=165;500;1000;0; | procs=165
[ 32] nagios2influx: [SERVICEPERFDATA] 1496242724 localhost Process httpd 0.503 0.288 PROCS OK: 13 processes with command name 'httpd' procs=13;1:;:50;0; | procs=13
[ 32] nagios2influx: [SERVICEPERFDATA] 1496242724 localhost Uptime 0.513 0.288 Ok, 127.0.0.1 uptime is 14:58:44 up 27 min, 1 user, load average: 0.10, 0.21, 0.23
[ 32] nagios2influx: [SERVICEPERFDATA] 1496242724 localhost Swap Usage 0.518 0.288 SWAP OK - 100% free (1023 MB out of 1023 MB) swap=1023MB;511;204;0;1023
[ 32] nagios2influx: [SERVICEPERFDATA] 1496242724 localhost Process npcd 0.537 0.288 PROCS OK: 1 process with command name 'npcd' procs=1;1:;:10;0; | procs=1
[ 32] nagios2influx: [SERVICEPERFDATA] 1496242724 localhost Time 0.541 0.288 OK - timedrift=0 seconds timedrift=0s;60;86400
[ 32] nagios2influx: [SERVICEPERFDATA] 1496242724 google.com HTTPS google.com 0.571 0.287 HTTP OK: HTTP/1.1 200 OK - 12426 bytes in 0.543 second response time time=0.543394s;2.000000;10.000000;0.000000 size=12426B;;;0
[ 32] nagios2influx: [SERVICEPERFDATA] 1496242725 localhost CPU Utilization 1.634 0.288 OK - user: 3.33, nice: 0.50, sys: 4.27, iowait: 2.38, irq: 0.50, softirq: 0.50 idle: 92.00 'user'=3.33 'nice'=0.50 'sys'=4.27 'softirq'=0.50 'iowait'=2.38 'irq'=0.50 'idle'=92.00
[ 32] nagios2influx: [SERVICEPERFDATA] 1496242726 localhost Apache Statistics 0.070 0.908 (No output on stdout) stderr: execvp(/usr/lib64/nagios/plugins/check_apachestatus.pl, ...) failed. errno is 2: No such file or directory
[ 32] nagios2influx: [SERVICEPERFDATA] 1496242726 localhost HTTP localhost 0.071 0.908 HTTP WARNING: HTTP/1.1 403 Forbidden - 5241 bytes in 0.007 second response time time=0.007263s;2.000000;10.000000;0.000000 size=5241B;;;0
[ 32] nagios2influx: [SERVICEPERFDATA] 1496242727 google.com HTTPS SSL Certificate google.com 0.167 0.908 OK - Certificate 'google.com' will expire on 2017-08-10 11:39 +0000/GMT.
[ 32] nagios2influx: [SERVICEPERFDATA] 1496242727 localhost HTTPS SSL Certificate localhost 0.202 0.908 OK - Certificate 'localhost.localdomain' will expire on 2018-05-30 14:18 +0000/GMT.
[ 32] nagios2influx: [SERVICEPERFDATA] 1496242727 localhost HTTPS localhost 0.205 0.909 HTTP WARNING: HTTP/1.1 403 Forbidden - 5241 bytes in 0.168 second response time time=0.168391s;2.000000;10.000000;0.000000 size=5241B;;;0
[ 32] nagios2influx: [SERVICEPERFDATA] 1496242727 localhost Forward Lookup 0.219 0.908 DNS OK: 0.210 seconds response time. localhost returns 127.0.0.1 time=0.209986s;;;0.000000
[ 32] nagios2influx: [SERVICEPERFDATA] 1496242727 localhost SSH Connectivity 0.235 0.909 SSH OK - OpenSSH_6.6.1 (protocol 2.0) time=0.180747s;;;0.000000;10.000000
[ 32] nagios2influx: [SERVICEPERFDATA] 1496242727 localhost Reverse Lookup 0.257 0.909 DNS CRITICAL - expected 'localhost.' but got 'name = localhost.'
[ 32] nagios2influx: [SERVICEPERFDATA] 1496242727 localhost Nagios Ghost Services 0.262 0.909 Unable to locate logfile for parsing
[ 32] nagios2influx: [SERVICEPERFDATA] 1496242727 localhost Nagios config up-to-date 0.319 0.909 Unknown - Error running script: [Errno 2] No such file or directory: '/etc/nagios/nagios.cfg'
[ 32] nagios2influx: [SERVICEPERFDATA] 1496242727 google.com HTTP google.com 0.318 0.908 HTTP OK: HTTP/1.1 200 OK - 12356 bytes in 0.253 second response time time=0.253451s;2.000000;10.000000;0.000000 size=12356B;;;0
[ 32] nagios2influx: [SERVICEPERFDATA] 1496242727 localhost Disk Usage 0.348 0.908 DISK OK ... /dev/shm=0B;460763136;486361088;0;511959040 /=3021656064B;7717729075;8146491802;0;8575254528 /dev=0B;450857779;475905434;0;500953088 /sys/fs/cgroup=0B;460763136;486361088;0;511959040 /run/user/0=0B;92152627;97272218;0;102391808 /boot=192581632B;956930458;1010093261;0;1063256064 /run=7016448B;460763136;486361088;0;511959040
[ 32] nagios2influx: [SERVICEPERFDATA] 1496242727 localhost Nagios Configuration 0.373 0.909 Could not validate Nagios configuration. warnings=0 errors=1
[ 32] nagios2influx: [SERVICEPERFDATA] 1496242727 localhost Process httpd 0.410 0.909 PROCS OK: 13 processes with command name 'httpd' procs=13;1:;:50;0; | procs=13
[ 32] nagios2influx: [SERVICEPERFDATA] 1496242727 localhost Load 0.454 0.909 OK - load average: 0.10, 0.20, 0.23 load1=0.100;15.000;30.000;0; load5=0.200;15.000;30.000;0; load15=0.230;15.000;30.000;0;
[ 32] nagios2influx: [SERVICEPERFDATA] 1496242727 localhost Running Processes 0.464 0.909 PROCS OK: 167 processes procs=167;500;1000;0; | procs=167
[ 32] nagios2influx: [SERVICEPERFDATA] 1496242727 localhost Process crond 0.466 0.909 PROCS OK: 1 process with command name 'crond' procs=1;1:;:10;0; | procs=1
[ 32] nagios2influx: [SERVICEPERFDATA] 1496242727 localhost Process nagios 0.513 0.909 PROCS WARNING: 0 processes with command name 'nagios' procs=0;1:;:10;0; | procs=0
[ 32] nagios2influx: [SERVICEPERFDATA] 1496242727 localhost Uptime 0.517 0.909 Ok, 127.0.0.1 uptime is 14:58:47 up 27 min, 1 user, load average: 0.10, 0.20, 0.23
[ 32] nagios2influx: [SERVICEPERFDATA] 1496242727 localhost Swap Usage 0.521 0.909 SWAP OK - 100% free (1023 MB out of 1023 MB) swap=1023MB;511;204;0;1023
[ 32] nagios2influx: [SERVICEPERFDATA] 1496242727 localhost Process npcd 0.528 0.909 PROCS OK: 1 process with command name 'npcd' procs=1;1:;:10;0; | procs=1
[ 32] nagios2influx: [SERVICEPERFDATA] 1496242727 localhost Time 0.532 0.909 OK - timedrift=0 seconds timedrift=0s;60;86400
[ 32] nagios2influx: [SERVICEPERFDATA] 1496242727 google.com HTTPS google.com 0.581 0.908 HTTP OK: HTTP/1.1 200 OK - 13704 bytes in 0.512 second response time time=0.512228s;2.000000;10.000000;0.000000 size=13704B;;;0
[ 32] nagios2influx: [SERVICEPERFDATA] 1496242728 localhost Ping 4.147 0.909 PING OK - Packet loss = 0%, RTA = 0.07 ms rta=0.072000ms;100.000000;500.000000;0.000000 pl=0%;20;40;0
[ 32] nagios2influx: [SERVICEPERFDATA] 1496242728 google.com Ping 4.165 0.908 PING OK - Packet loss = 0%, RTA = 8.69 ms rta=8.693000ms;100.000000;500.000000;0.000000 pl=0%;20;40;0
[ 32] nagios2influx: [SERVICEPERFDATA] 1496242728 localhost CPU Utilization 1.635 0.908 OK - user: 3.41, nice: 0.50, sys: 3.41, iowait: 0.50, irq: 0.50, softirq: 0.50 idle: 94.67 'user'=3.41 'nice'=0.50 'sys'=3.41 'softirq'=0.50 'iowait'=0.50 'irq'=0.50 'idle'=94.67
[ 32] nagios2influx: [SERVICEPERFDATA] 1496242731 google.com Ping 4.143 0.908 PING OK - Packet loss = 0%, RTA = 15.52 ms rta=15.524000ms;100.000000;500.000000;0.000000 pl=0%;20;40;0
[ 32] nagios2influx: [SERVICEPERFDATA] 1496242731 localhost Ping 4.148 0.909 PING OK - Packet loss = 0%, RTA = 0.08 ms rta=0.076000ms;100.000000;500.000000;0.000000 pl=0%;20;40;0
[root@localhost bin]#
I tailed the influxdb journal, but nothing showed up. No connection is being made.
[root@localhost bin]# journalctl -u influxdb.service -f -n 0
-- Logs begin at Wed 2017-05-31 14:31:26 GMT. --
You've not supplied the service data format, but it still doesn't match the format expected by nagios2influx, otherwise you'd have [ 64] debug line showing a successful decode. I'm guessing it's the '[SERVICEPERFDATA]' at the start of the line. Try removing this field, or add an addition void at the start of the format line.
Yeah, I figured it out, think I got the hang of this. Thanks for the help!
tl;dr: See gif on bottom of page.
I think I mostly followed the instructions and got this to work (kinda). However, data I see in Grafana seems to be in the wrong format (mislabeled, nonexistent?). I'm not too familiar with Grafana/InfluxDB, so please bear with me.
Here are all perfdata settings in naemon.cfg:
all perfdata commands:
/etc/naemon/nagios2influx.cfg:
/usr/local/bin/nagios-perf:
only changed line in /usr/local/bin/nagios2influx:
$config = '/etc/naemon/nagios2influx.cfg';
influx:
influxdb journal:
snippet from /tmp/nagios-perf:
./nagios2influx -n -d 127