CHECK_NRPE: Unclear error message during SSL mismatch

[tjyang]

This is a test case using nagios server and nrpe agent on same host.

• error message

  
  [pi@nagios ~]$ check_nrpe -H localhost -n
  CHECK_NRPE: Receive header underflow - only -1 bytes received (4 expected).
  [pi@nagios ~]$ which check_nrpe
  /usr/lib64/nagios/plugins/check_nrpe
  [pi@nagios ~]$

[pi@nagios ~]$ check_nrpe -H localhost -c check_logfiles_4A -a "/var/log/nagios/nagios.log" "nagios-log" ".] Error:." ".NOERROR." CHECK_NRPE: Receive header underflow - only 0 bytes received (4 expected). [pi@nagios ~]$

* Good check_nrpe results

[pi@nagios ~]$ check_nrpe -H localhost -c check_cpu OK - user: 0.74, nice: 0.50, sys: 1.48, iowait: 0.50, irq: 0.50, softirq: 0.50 idle: 99.27 | 'user'=0.74 ' [pi@nagios ~]$

[pi@nagios4 src]$ grep check_time /etc/nrpe.d/check_time.cfg command[check_time]=/usr/lib64/nagios/plugins/check_time.sh -H '$ARG1$' -w '$ARG2$' -c '$ARG3$'

[pi@nagios4 src]$ check_nrpe -H localhost -c check_time -a localhost 1 2 OK - timedrift=0 seconds | timedrift=0s;1;2 [pi@nagios4 src]$

* OS and Nagios server  info

[pi@nagios ~]$ rpm -qa |egrep 'nrpe-3.2.1|nagios-4.4.5';cat /etc/redhat-release;date nagios-plugins-nrpe-3.2.1-8.el7.x86_64 nagios-4.4.5-1.el7.x86_64 nrpe-3.2.1-8.el7.x86_64 CentOS Linux release 7.7.1908 (Core) Fri Nov 1 06:49:26 CDT 2019 [pi@nagios ~]$

* nrpe  info

[pi@nagios ~]$ /usr/lib64/nagios/plugins/check_nrpe -H localhost NRPE v3.2.1 [pi@nagios ~]$ [pi@nagios ~]$ egrep '^dont_blame_nrpe' /etc/nagios/nrpe.cfg dont_blame_nrpe=1 [pi@nagios ~]$ [pi@nagios ~]$ egrep 'logfile' /etc/nrpe.d/check_logfile.cfg command[check_logfiles_4A]=/usr/lib64/nagios/plugins/contrib/check_logfiles --logfile $ARG1$ --tag $ARG2$ --warningpattern $ARG3$ --criticalpattern $ARG4$ [pi@nagios ~]$ [pi@nagios ~]$ egrep 'cpu' /etc/nrpe.d/check_cpu.cfg command[check_cpu]=/usr/lib64/nagios/plugins/check_cpu.sh [pi@nagios ~]$

[tjyang]

Hi

Can you confirm following issue as bug ?

• reqular expression patter within double qoutes doesn't work.

[pi@nagios4 ~]$ check_nrpe -H localhost   -c check_logfiles_4A -a "/var/log/nagios/nagios.log" "nagios-log" ".*[^]] Error:.*" ".*NOERROR.*"
CHECK_NRPE: Receive header underflow - only 0 bytes received (4 expected).
[pi@nagios4 ~]$ 

• work arounds that works.

[pi@nagios4 ~]$ check_nrpe -H localhost   -c check_logfiles_4A -a "/var/log/nagios/nagios.log" "nagios-log" .*[^]] Error:.*  .*NOERROR.*                 
OK - no errors or warnings|nagios-log_lines=0 nagios-log_warnings=0 nagios-log_criticals=0 nagios-log_unknowns=0
[pi@nagios4 ~]$ check_nrpe -H localhost   -c check_logfiles_2A_local -a "/var/log/nagios/nagios.log" "nagios-log"
OK - no errors or warnings|nagios-log_lines=0 nagios-log_warnings=0 nagios-log_criticals=0 nagios-log_unknowns=0
[pi@nagios4 ~]$ grep local /etc/nrpe.d/check_logfile_local.cfg
command[check_logfiles_2A_local]=/usr/lib64/nagios/plugins/contrib/check_logfiles  --logfile $ARG1$ --tag $ARG2$ --warningpattern  .*[^]] Error:.*  --criticalpattern  .*NOERROR.*
[pi@nagios4 ~]$ date
Thu Nov  7 16:47:21 CST 2019
[pi@nagios4 ~]$

[sawolf]

Hi @tjyang - thanks for reporting this. I'm not too familiar with NRPE, so hopefully @hedenface will be able to tell you whether or not that's a bug (though I agree that it looks like a system-level error).

One thing I did notice, though - removing the quotes should have the same effect as running the command like

check_nrpe -H localhost   -c check_logfiles_4A -a "/var/log/nagios/nagios.log" "nagios-log" ".*[^]]" "Error:.*"  ".*NOERROR.*"

that is, the whitespace would separate different shell arguments. Also, since you're invoking these directly on the shell, you may be getting issues related to globbing expansion. I would try using single quotes in this situation, i.e.

check_nrpe -H localhost   -c check_logfiles_4A -a "/var/log/nagios/nagios.log" "nagios-log" '.*[^]] Error:.*'  '.*NOERROR.*'

[sawolf]

I just talked to Bryan - he recommends taking a look at your SSL settings, both on the check_nrpe and NRPE sides. Your other commands aren't using the -n option, so that may be the source of your problems. He also says to check the version numbers - I think it's unlikely to be an issue since you're running it against localhost, but may be worth a shot.

[hedenface]

Why are you using -n in the first command you list but no others? Let's see ./check_nrpe -H localhost (without the -n).

[tjyang]

@hedenface

"-n, --no-ssl Do no use SSL"

• Why are you using -n in the first command you list but no others?

I was hoping to see the output from with/without -n should be the same.

[pi@nagios4 ~]$ check_nrpe -H localhost ; check_nrpe -H localhost -n
NRPE v3.2.1
CHECK_NRPE: Receive header underflow - only -1 bytes received (4 expected).
[pi@nagios4 ~]$

[sawolf]

If you want to use non-SSL connections, you need to make sure the NRPE daemon is also running with -n. The check_nrpe client currently doesn't do anything to determine whether there's an SSL mismatch, hence the errors. I do agree that the error messages could be clearer, though.

[tjyang]

Thanks @sawolf to put this issue as an RFE.

• "-n" added in nrpe.service to allow no ssl connection.

  
  [pi@nagios4 ~]$ egrep "\-n"  /usr/lib/systemd/system/nrpe.service
  ExecStart=/usr/sbin/nrpe -n -c /etc/nagios/nrpe.cfg -f $NRPE_SSL_OPT
  [pi@nagios4 ~]$

[pi@nagios4 ~]$ check_nrpe -H localhost -n NRPE v3.2.1 [pi@nagios4 ~]$

[tjyang]

Now back to the real goal I want to achieve, ie. to pass over Regular Expression pattern for check_logfile command.

• remove all nasty metachars blocking between two double quotes below.

  
  [pi@nagios4 ~]$ grep nasty /etc/nagios/nrpe.cfg
  nasty_metachars=""
  [pi@nagios4 ~]$

* successful log after nrpe daemon restart. 

[pi@nagios4 ~]$ check_nrpe -H localhost -c check_logfiles_4A -a "/var/log/nagios/nagios.log" "nagios-log" ".[^]] Error:." ".NOERROR." OK - no errors or warnings|nagios-log_lines=0 nagios-log_warnings=0 nagios-log_criticals=0 nagios-log_unknowns=0 [pi@nagios4 ~]$

* suggestion: return error should related RE characters was blocked by setting in /etc/nagios/nrpe.cfg.