Icinga / icinga2

The core of our monitoring platform with a powerful configuration language and REST API.
https://icinga.com/docs/icinga2/latest
GNU General Public License v2.0
2.03k stars 579 forks source link

check nscp api failing on -a arguments #6855

Closed jgrammen-agilitypr closed 5 years ago

jgrammen-agilitypr commented 5 years ago

Expected Behavior

querying check_nscp_api.exe with -a drive=c should return results

Current Behavior

running a query with drive= almost anything causes a strange error.

C:\Program Files\ICINGA2\sbin>check_nscp_api.exe --password icinga -P 8443 -H localhost -a drive=* -q check_drivesize
check_drivesize OK All 2 drive(s) are ok | 'C:\ used'=14.595673GB;39.569528;44.515719;0;49.461910 'C:\ used %'=30%;80;90;0;100 'D:\ used'=0B;0;0;0;0

C:\Program Files\ICINGA2\sbin>check_nscp_api.exe --password icinga -P 8443 -H localhost -a drive=c -q check_drivesize
Caught exception: Error: Tried to read/write from a closed socket.

UNKNOWN: No data received.

Possible Solution

Steps to Reproduce (for bugs)

  1. Install icinga2 latest version on windows, include the nsclient++
  2. use the command line to query the nsclient
  3. note that the query with -a drive=* works but -a drive=c does not

Context

cannot use nscp queries as per the documentation examples https://icinga.com/docs/icinga2/latest/doc/06-distributed-monitoring/#nsclient-with-check_nscp_api

Your Environment

Copyright (c) 2012-2018 Icinga Development Team (https://icinga.com/) License GPLv2+: GNU GPL version 2 or later http://gnu.org/licenses/gpl2.html This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law.

System information: Platform: Windows Platform version: 8 Kernel: Windows Kernel version: 6.2 Architecture: x86_64

Build information: Compiler: MSVC 19.0.24215.1 Build host: ICINGA-BUILD-WI

Application information:

General paths: Config directory: C:\ProgramData\icinga2\etc\icinga2 Data directory: C:\ProgramData\icinga2\var\lib\icinga2 Log directory: C:\ProgramData\icinga2\var\log\icinga2 Cache directory: C:\ProgramData\icinga2\var\cache\icinga2 Spool directory: C:\ProgramData\icinga2\var\spool\icinga2 Run directory: C:\ProgramData\icinga2\var\run\icinga2

Old paths (deprecated): Installation root: C:\Program Files\ICINGA2\ Sysconf directory: C:\ProgramData\icinga2\etc Run directory (base): C:\ProgramData\icinga2\var\run Local state directory: C:\ProgramData\icinga2\var

Internal paths: Package data directory: C:\Program Files\ICINGA2\share\icinga2 State path: C:\ProgramData\icinga2\var\lib\icinga2/icinga2.state Modified attributes path: C:\ProgramData\icinga2\var\lib\icinga2/modified-attributes.conf Objects path: C:\ProgramData\icinga2\var\cache\icinga2/icinga2.debug Vars path: C:\ProgramData\icinga2\var\cache\icinga2/icinga2.vars PID path: C:\ProgramData\icinga2\var\run\icinga2/icinga2.pid


* Operating System and version:
* Enabled features (`icinga2 feature list`):

C:\Program Files\ICINGA2\sbin>icinga2.exe feature list Disabled features: command compatlog elasticsearch gelf graphite ido-mysql ido-pgsql influxdb livestatus notification opentsdb perfdata statusdata Enabled features: api checker debuglog mainlog

* Icinga Web 2 version and modules (System - About):
not relevant icinga2web is running on the master server, the error is on the client
* Config validation (`icinga2 daemon -C`):

C:\Program Files\ICINGA2\sbin>icinga2.exe daemon -C [2018-12-20 09:57:32 -0800] information/cli: Icinga application loader (version: v2.10.2) [2018-12-20 09:57:32 -0800] information/cli: Loading configuration file(s). [2018-12-20 09:57:32 -0800] information/ConfigItem: Committing config item(s). [2018-12-20 09:57:32 -0800] information/ApiListener: My API identity: DESKTOP-S8J0OKI [2018-12-20 09:57:34 -0800] information/ConfigItem: Instantiated 1 ApiListener. [2018-12-20 09:57:34 -0800] information/ConfigItem: Instantiated 1 IcingaApplication. [2018-12-20 09:57:34 -0800] information/ConfigItem: Instantiated 2 Endpoints. [2018-12-20 09:57:34 -0800] information/ConfigItem: Instantiated 4 Zones. [2018-12-20 09:57:34 -0800] information/ConfigItem: Instantiated 1 CheckerComponent. [2018-12-20 09:57:34 -0800] information/ConfigItem: Instantiated 2 FileLoggers. [2018-12-20 09:57:34 -0800] information/ConfigItem: Instantiated 215 CheckCommands. [2018-12-20 09:57:34 -0800] information/ScriptGlobal: Dumping variables to file 'C:\ProgramData\icinga2\var\cache\icinga2/icinga2.vars' [2018-12-20 09:57:34 -0800] information/cli: Finished validating the configuration file(s).

* If you run multiple Icinga 2 instances, the `zones.conf` file (or `icinga2 object list --type Endpoint` and `icinga2 object list --type Zone`) from all affected nodes.

C:\Program Files\ICINGA2\sbin>icinga2.exe object list --type Endpoint Object 'icinga.vagrant' of type 'Endpoint': % declared in 'C:\ProgramData\icinga2\etc\icinga2/zones.conf', lines 6:1-6:32

Object 'DESKTOP-S8J0OKI' of type 'Endpoint': % declared in 'C:\ProgramData\icinga2\etc\icinga2/zones.conf', lines 15:1-15:33

C:\Program Files\ICINGA2\sbin>icinga2.exe object list --type Zone
Object 'master' of type 'Zone':
  % declared in 'C:\ProgramData\icinga2\etc\icinga2/zones.conf', lines 11:1-11:20
  * __name = "master"
  * endpoints = [ "icinga.vagrant" ]
    % = modified in 'C:\ProgramData\icinga2\etc\icinga2/zones.conf', lines 12:2-12:33
  * global = false
  * name = "master"
  * package = "_etc"
  * parent = ""
  * source_location
    * first_column = 1
    * first_line = 11
    * last_column = 20
    * last_line = 11
    * path = "C:\ProgramData\icinga2\etc\icinga2/zones.conf"
  * templates = [ "master" ]
    % = modified in 'C:\ProgramData\icinga2\etc\icinga2/zones.conf', lines 11:1-11:20
  * type = "Zone"
  * zone = ""

Object 'director-global' of type 'Zone':
  % declared in 'C:\ProgramData\icinga2\etc\icinga2/zones.conf', lines 27:1-27:29
  * __name = "director-global"
  * endpoints = null
  * global = true
    % = modified in 'C:\ProgramData\icinga2\etc\icinga2/zones.conf', lines 28:2-28:14
  * name = "director-global"
  * package = "_etc"
  * parent = ""
  * source_location
    * first_column = 1
    * first_line = 27
    * last_column = 29
    * last_line = 27
    * path = "C:\ProgramData\icinga2\etc\icinga2/zones.conf"
  * templates = [ "director-global" ]
    % = modified in 'C:\ProgramData\icinga2\etc\icinga2/zones.conf', lines 27:1-27:29
  * type = "Zone"
  * zone = ""

Object 'DESKTOP-S8J0OKI' of type 'Zone':
  % declared in 'C:\ProgramData\icinga2\etc\icinga2/zones.conf', lines 18:1-18:29
  * __name = "DESKTOP-S8J0OKI"
  * endpoints = [ "DESKTOP-S8J0OKI" ]
    % = modified in 'C:\ProgramData\icinga2\etc\icinga2/zones.conf', lines 19:2-19:34
  * global = false
  * name = "DESKTOP-S8J0OKI"
  * package = "_etc"
  * parent = "master"
    % = modified in 'C:\ProgramData\icinga2\etc\icinga2/zones.conf', lines 20:2-20:18
  * source_location
    * first_column = 1
    * first_line = 18
    * last_column = 29
    * last_line = 18
    * path = "C:\ProgramData\icinga2\etc\icinga2/zones.conf"
  * templates = [ "DESKTOP-S8J0OKI" ]
    % = modified in 'C:\ProgramData\icinga2\etc\icinga2/zones.conf', lines 18:1-18:29
  * type = "Zone"
  * zone = ""

Object 'global-templates' of type 'Zone':
  % declared in 'C:\ProgramData\icinga2\etc\icinga2/zones.conf', lines 23:1-23:30
  * __name = "global-templates"
  * endpoints = null
  * global = true
    % = modified in 'C:\ProgramData\icinga2\etc\icinga2/zones.conf', lines 24:2-24:14
  * name = "global-templates"
  * package = "_etc"
  * parent = ""
  * source_location
    * first_column = 1
    * first_line = 23
    * last_column = 30
    * last_line = 23
    * path = "C:\ProgramData\icinga2\etc\icinga2/zones.conf"
  * templates = [ "global-templates" ]
    % = modified in 'C:\ProgramData\icinga2\etc\icinga2/zones.conf', lines 23:1-23:30
  * type = "Zone"
  * zone = ""
dnsmichi commented 5 years ago

That sounds more like a connection problem with the time of execution. Is that reproducible with every call that happens e.g. in a loop?

jgrammen-agilitypr commented 5 years ago

The check fails inconsistently within icinga. Here is the ouput for a number of sequential manual attempts. Interestingly the first one works.

C:\Program Files\ICINGA2\sbin>check_nscp_api.exe --password somepass -P 8443 -H localhost -a drive=c -q check_drivesize
check_drivesize OK All 1 drive(s) are ok | 'c: used'=12.580624GB;39.569528;44.515719;0;49.461910 'c: used %'=25%;80;90;0;100

C:\Program Files\ICINGA2\sbin>check_nscp_api.exe --password somepass -P 8443 -H localhost -a drive=c -q check_drivesize
Caught exception: Error: Tried to read/write from a closed socket.

UNKNOWN: No data received.

C:\Program Files\ICINGA2\sbin>check_nscp_api.exe --password somepass -P 8443 -H localhost -a drive=c -q check_drivesize
Caught exception: Error: Tried to read/write from a closed socket.

UNKNOWN: No data received.

C:\Program Files\ICINGA2\sbin>check_nscp_api.exe --password somepass -P 8443 -H localhost -a drive=c -q check_drivesize
Caught exception: Error: Tried to read/write from a closed socket.

UNKNOWN: No data received.

C:\Program Files\ICINGA2\sbin>check_nscp_api.exe --password somepass -P 8443 -H localhost -a drive=c -q check_drivesize
Caught exception: Error: Tried to read/write from a closed socket.

UNKNOWN: No data received.

C:\Program Files\ICINGA2\sbin>check_nscp_api.exe --password somepass -P 8443 -H localhost -a drive=c -q check_drivesize
Caught exception: Error: Tried to read/write from a closed socket.

UNKNOWN: No data received.

C:\Program Files\ICINGA2\sbin>check_nscp_api.exe --password somepass -P 8443 -H localhost -a drive=c -q check_drivesize
Caught exception: Error: Tried to read/write from a closed socket.

UNKNOWN: No data received.

C:\Program Files\ICINGA2\sbin>check_nscp_api.exe --password somepass -P 8443 -H localhost -a drive=c -q check_drivesize
Caught exception: Error: Tried to read/write from a closed socket.

UNKNOWN: No data received.

C:\Program Files\ICINGA2\sbin>check_nscp_api.exe --password somepass -P 8443 -H localhost -a drive=c -q check_drivesize
Caught exception: Error: Tried to read/write from a closed socket.

UNKNOWN: No data received.

C:\Program Files\ICINGA2\sbin>check_nscp_api.exe --password somepass -P 8443 -H localhost -a drive=c -q check_drivesize
Caught exception: Error: Tried to read/write from a closed socket.

UNKNOWN: No data received.
Crunsher commented 5 years ago

Scrolling over the code I found us not closing the connection. Is it possible for the socket to stay open after the program terminates :thinking:

lippserd commented 5 years ago

We're a little short on resources at the moment but we try to help you as soon as possible.

dnsmichi commented 5 years ago

Will be solved as part of the network stack rewrite.

dnsmichi commented 5 years ago

This is fixed in git master.