search

centreon / centreon-plugins

Collection of standard plugins to discover and gather cloud-to-edge metrics and status across your whole IT infrastructure.
https://www.centreon.com
Apache License 2.0
310 stars 273 forks source link

Multiple InfluxDB Queries with InfluxDB Plugin #4314

Closed wp-perc closed 1 year ago

wp-perc commented 1 year ago

Bug/Question

Hello guys, I am facing a strange behavior of InfluxDB Plugin, but since I am less than a beginner in Perl I am really unable to understand why it happens.

Scenario: I am trying to monitor the Load of a Linux-based machine using data provided with Telegraf. To be more specific, I want to implement a check reporting Load1, Load5 and Load15 from a specific server using data available in measurement system; this measurement is generated using system input from Telegraf. This is something like the example reported in the Plugin's help.

What I expect to happen: I use three queries, one for Load1, one for Load5 and one for Load15, and get three metrics. Here the command line.

'/usr/lib/centreon/plugins/centreon_influxdb.pl' \
'--plugin' 'database::influxdb::plugin' \
'--hostname' 'MYINFLUXDBSERVER' '--port' '8086' '--proto' 'https' '--ssl' 'TLSv1' '--username' 'MYUSER' '--password 'MYPASSWORD' --http-backend' 'lwp' \
'--mode' 'query' '--timeout' '30' \
'--query' 'load1,SELECT max(load1) AS load1 FROM "telegraf_master"."autogen"."system" WHERE host='\''<MYHOST>'\'' AND time > (now() - 10m) GROUP BY host;' \
'--query' 'load5,SELECT max(load5) AS load5 FROM "telegraf_master"."autogen"."system" WHERE host='\''<MYHOST>'\'' AND time > (now() - 10m) GROUP BY host;' \
'--query' 'load15,SELECT max(load15) AS load15 FROM "telegraf_master"."autogen"."system" WHERE host='\''<MYHOST>'\'' AND time > (now() - 10m) GROUP BY host;' \
'--instance' 'host' \
'--output' 'System Load (1,5,15): %{load1} %{load5} %{load15}' \
'--multiple-output' 'System Load (1,5,15): %{load1}, %{load5}, %{load15}'

What really happens: about 9 times out of 10 the plugin returs a series of strange errors and does nothig.

'/usr/lib/centreon/plugins/centreon_influxdb.pl' \
'--plugin' 'database::influxdb::plugin' \
'--hostname' 'MYINFLUXDBSERVER' '--port' '8086' '--proto' 'https' '--ssl' 'TLSv1' '--username' 'MYUSER' '--password 'MYPASSWORD' --http-backend' 'lwp' \
'--mode' 'query' '--timeout' '30' \
'--query' 'load1,SELECT max(load1) AS load1 FROM "telegraf_master"."autogen"."system" WHERE host='\''<MYHOST>'\'' AND time > (now() - 10m) GROUP BY host;' \
'--query' 'load5,SELECT max(load5) AS load5 FROM "telegraf_master"."autogen"."system" WHERE host='\''<MYHOST>'\'' AND time > (now() - 10m) GROUP BY host;' \
'--query' 'load15,SELECT max(load15) AS load15 FROM "telegraf_master"."autogen"."system" WHERE host='\''<MYHOST>'\'' AND time > (now() - 10m) GROUP BY host;' \
'--instance' 'host' \
'--output' 'System Load (1,5,15): %{load1} %{load5} %{load15}' \
'--multiple-output' 'System Load (1,5,15): %{load1}, %{load5}, %{load15}' 
Use of uninitialized value $options{"column"} in array element at /usr/lib/centreon/plugins/centreon_influxdb.pl line 8649.
Use of uninitialized value $column_index in array element at /usr/lib/centreon/plugins/centreon_influxdb.pl line 9402.
Use of uninitialized value $options{"column"} in array element at /usr/lib/centreon/plugins/centreon_influxdb.pl line 8649.
Use of uninitialized value $column_index in array element at /usr/lib/centreon/plugins/centreon_influxdb.pl line 9402.
Use of uninitialized value $options{"column"} in array element at /usr/lib/centreon/plugins/centreon_influxdb.pl line 8649.
Use of uninitialized value $column_index in array element at /usr/lib/centreon/plugins/centreon_influxdb.pl line 9402.
OK: status : skipped (no value(s))

But hey, 1 time out of ten it works perfectly...

This error happens event with two or 4 queries, regardless of their content. But if I use only one query, it works flawlessly.

Am I missing something? Can you help me in understanding what is happening?

Thanks in advance for your time.

garnier-quentin commented 1 year ago

When you have the error, could you provide the --debug ?

wp-perc commented 1 year ago

Here the debug output. I just omitted the password.

'/usr/lib/centreon/plugins/centreon_influxdb.pl' \
'--plugin' 'database::influxdb::plugin' \
'--hostname' 'influxdb.neteyelocal' '--port' '8086' '--proto' 'https' '--ssl' 'TLSv1' '--username' 'telegraf_master_reader' '--password' 'MYPASSWORD' '--http-backend' 'lwp' \
'--mode' 'query' '--timeout' '30' \
'--query' 'load1,SELECT max(load1) AS load1 FROM "telegraf_master"."autogen"."system" WHERE host='\''neteye4testn1.wp.lan'\'' AND time > (now() - 10m) GROUP BY host;' \
'--query' 'load5,SELECT max(load5) AS load5 FROM "telegraf_master"."autogen"."system" WHERE host='\''neteye4testn1.wp.lan'\'' AND time > (now() - 10m) GROUP BY host;' \
'--query' 'load15,SELECT max(load15) AS load15 FROM "telegraf_master"."autogen"."system" WHERE host='\''neteye4testn1.wp.lan'\'' AND time > (now() - 10m) GROUP BY host;' \
'--instance' 'host'   \
'--output' 'System Load (1,5,15): %{load1} %{load5} %{load15}' \
'--multiple-output' 'System Load (1,5,15): %{load1}, %{load5}, %{load15}'
Use of uninitialized value $options{"column"} in array element at /usr/lib/centreon/plugins/centreon_influxdb.pl line 8649.
Use of uninitialized value $column_index in array element at /usr/lib/centreon/plugins/centreon_influxdb.pl line 9402.
Use of uninitialized value $options{"column"} in array element at /usr/lib/centreon/plugins/centreon_influxdb.pl line 8649.
Use of uninitialized value $column_index in array element at /usr/lib/centreon/plugins/centreon_influxdb.pl line 9402.
OK: status : skipped (no value(s))
URL: 'https://influxdb.neteyelocal:8086/query?epoch=s'
Parameters: 'q=SELECT max(load15) AS load15 FROM "telegraf_master"."autogen"."system" WHERE host='neteye4testn1.wp.lan' AND time > (now() - 10m) GROUP BY host;'
======> request send
POST https://influxdb.neteyelocal:8086/query?epoch=s
Authorization: Basic dGVsZWdyYWZfbWFzdGVyX3JlYWRlcjo0bmY3MDkzbm1mODczNGpubWc1
User-Agent: centreon::plugins::backend::http::useragent
Content-Type: application/x-www-form-urlencoded

q=SELECT+max(load15)+AS+load15+FROM+%22telegraf_master%22.%22autogen%22.%22system%22+WHERE+host%3D'neteye4testn1.wp.lan'+AND+time+%3E+(now()+-+10m)+GROUP+BY+host%3B
======> response done
HTTP/1.1 200 OK
Date: Thu, 30 Mar 2023 19:55:03 GMT
Content-Type: application/json
Client-Date: Thu, 30 Mar 2023 19:55:03 GMT
Client-Peer: 192.168.232.34:8086
Client-Response-Num: 1
Client-SSL-Cert-Issuer: /DC=neteyelocal/DC=*/O=NetEye/OU=NetEye Root CA/CN=*.neteyelocal
Client-SSL-Cert-Subject: /C=IT/ST=Bolzano/L=Bolzano/O=GlobalSecurity/OU=NeteyeInfluxDB/CN=influxdb.neteyelocal
Client-SSL-Cipher: TLS_AES_256_GCM_SHA384
Client-SSL-Socket-Class: IO::Socket::SSL
Client-Transfer-Encoding: chunked
Request-Id: c2e00fc0-cf34-11ed-8c74-0050569e3eff
X-Influxdb-Build: OSS
X-Influxdb-Version: 1.8.10
X-Request-Id: c2e00fc0-cf34-11ed-8c74-0050569e3eff

{"results":[{"statement_id":0,"series":[{"name":"system","tags":{"host":"neteye4testn1.wp.lan"},"columns":["time","load15"],"values":[[1680205590,1.72]]}]}]}
URL: 'https://influxdb.neteyelocal:8086/query?epoch=s'
Parameters: 'q=SELECT max(load5) AS load5 FROM "telegraf_master"."autogen"."system" WHERE host='neteye4testn1.wp.lan' AND time > (now() - 10m) GROUP BY host;'
======> request send
POST https://influxdb.neteyelocal:8086/query?epoch=s
Authorization: Basic dGVsZWdyYWZfbWFzdGVyX3JlYWRlcjo0bmY3MDkzbm1mODczNGpubWc1
User-Agent: centreon::plugins::backend::http::useragent
Content-Type: application/x-www-form-urlencoded

q=SELECT+max(load5)+AS+load5+FROM+%22telegraf_master%22.%22autogen%22.%22system%22+WHERE+host%3D'neteye4testn1.wp.lan'+AND+time+%3E+(now()+-+10m)+GROUP+BY+host%3B
======> response done
HTTP/1.1 200 OK
Date: Thu, 30 Mar 2023 19:55:03 GMT
Content-Type: application/json
Client-Date: Thu, 30 Mar 2023 19:55:03 GMT
Client-Peer: 192.168.232.34:8086
Client-Response-Num: 2
Client-SSL-Cert-Issuer: /DC=neteyelocal/DC=*/O=NetEye/OU=NetEye Root CA/CN=*.neteyelocal
Client-SSL-Cert-Subject: /C=IT/ST=Bolzano/L=Bolzano/O=GlobalSecurity/OU=NeteyeInfluxDB/CN=influxdb.neteyelocal
Client-SSL-Cipher: TLS_AES_256_GCM_SHA384
Client-SSL-Socket-Class: IO::Socket::SSL
Client-SSL-Warning: Peer certificate not verified
Client-Transfer-Encoding: chunked
Request-Id: c2e4007c-cf34-11ed-8c75-0050569e3eff
X-Influxdb-Build: OSS
X-Influxdb-Version: 1.8.10
X-Request-Id: c2e4007c-cf34-11ed-8c75-0050569e3eff

{"results":[{"statement_id":0,"series":[{"name":"system","tags":{"host":"neteye4testn1.wp.lan"},"columns":["time","load5"],"values":[[1680205590,1.49]]}]}]}
URL: 'https://influxdb.neteyelocal:8086/query?epoch=s'
Parameters: 'q=SELECT max(load1) AS load1 FROM "telegraf_master"."autogen"."system" WHERE host='neteye4testn1.wp.lan' AND time > (now() - 10m) GROUP BY host;'
======> request send
POST https://influxdb.neteyelocal:8086/query?epoch=s
Authorization: Basic dGVsZWdyYWZfbWFzdGVyX3JlYWRlcjo0bmY3MDkzbm1mODczNGpubWc1
User-Agent: centreon::plugins::backend::http::useragent
Content-Type: application/x-www-form-urlencoded

q=SELECT+max(load1)+AS+load1+FROM+%22telegraf_master%22.%22autogen%22.%22system%22+WHERE+host%3D'neteye4testn1.wp.lan'+AND+time+%3E+(now()+-+10m)+GROUP+BY+host%3B
======> response done
HTTP/1.1 200 OK
Date: Thu, 30 Mar 2023 19:55:03 GMT
Content-Type: application/json
Client-Date: Thu, 30 Mar 2023 19:55:03 GMT
Client-Peer: 192.168.232.34:8086
Client-Response-Num: 3
Client-SSL-Cert-Issuer: /DC=neteyelocal/DC=*/O=NetEye/OU=NetEye Root CA/CN=*.neteyelocal
Client-SSL-Cert-Subject: /C=IT/ST=Bolzano/L=Bolzano/O=GlobalSecurity/OU=NeteyeInfluxDB/CN=influxdb.neteyelocal
Client-SSL-Cipher: TLS_AES_256_GCM_SHA384
Client-SSL-Socket-Class: IO::Socket::SSL
Client-SSL-Warning: Peer certificate not verified
Client-Transfer-Encoding: chunked
Request-Id: c2e45a16-cf34-11ed-8c76-0050569e3eff
X-Influxdb-Build: OSS
X-Influxdb-Version: 1.8.10
X-Request-Id: c2e45a16-cf34-11ed-8c76-0050569e3eff

{"results":[{"statement_id":0,"series":[{"name":"system","tags":{"host":"neteye4testn1.wp.lan"},"columns":["time","load1"],"values":[[1680205590,2.13]]}]}]}
status : skipped (no value(s))

And just for reference, here the debug output of the same command line when it works

'/usr/lib/centreon/plugins/centreon_influxdb.pl' \
'--plugin' 'database::influxdb::plugin' \
'--hostname' 'influxdb.neteyelocal' '--port' '8086' '--proto' 'https' '--ssl' 'TLSv1' '--username' 'telegraf_master_reader' '--password' 'MYPASSWORD' '--http-backend' 'lwp' \
'--mode' 'query' '--timeout' '30' \
'--query' 'load1,SELECT max(load1) AS load1 FROM "telegraf_master"."autogen"."system" WHERE host='\''neteye4testn1.wp.lan'\'' AND time > (now() - 10m) GROUP BY host;' \
'--query' 'load5,SELECT max(load5) AS load5 FROM "telegraf_master"."autogen"."system" WHERE host='\''neteye4testn1.wp.lan'\'' AND time > (now() - 10m) GROUP BY host;' \
'--query' 'load15,SELECT max(load15) AS load15 FROM "telegraf_master"."autogen"."system" WHERE host='\''neteye4testn1.wp.lan'\'' AND time > (now() - 10m) GROUP BY host;' \
'--instance' 'host'   \
'--output' 'System Load (1,5,15): %{load1} %{load5} %{load15}' \
'--multiple-output' 'System Load (1,5,15): %{load1}, %{load5}, %{load15}'
OK: System Load (1,5,15): 1.72 1.45 1.67 | 'neteye4testn1.wp.lan#load1'=1.72;;;; 'neteye4testn1.wp.lan#load5'=1.45;;;; 'neteye4testn1.wp.lan#load15'=1.67;;;;
URL: 'https://influxdb.neteyelocal:8086/query?epoch=s'
Parameters: 'q=SELECT max(load1) AS load1 FROM "telegraf_master"."autogen"."system" WHERE host='neteye4testn1.wp.lan' AND time > (now() - 10m) GROUP BY host;'
======> request send
POST https://influxdb.neteyelocal:8086/query?epoch=s
Authorization: Basic dGVsZWdyYWZfbWFzdGVyX3JlYWRlcjo0bmY3MDkzbm1mODczNGpubWc1
User-Agent: centreon::plugins::backend::http::useragent
Content-Type: application/x-www-form-urlencoded

q=SELECT+max(load1)+AS+load1+FROM+%22telegraf_master%22.%22autogen%22.%22system%22+WHERE+host%3D'neteye4testn1.wp.lan'+AND+time+%3E+(now()+-+10m)+GROUP+BY+host%3B
======> response done
HTTP/1.1 200 OK
Date: Thu, 30 Mar 2023 19:58:02 GMT
Content-Type: application/json
Client-Date: Thu, 30 Mar 2023 19:58:02 GMT
Client-Peer: 192.168.232.34:8086
Client-Response-Num: 1
Client-SSL-Cert-Issuer: /DC=neteyelocal/DC=*/O=NetEye/OU=NetEye Root CA/CN=*.neteyelocal
Client-SSL-Cert-Subject: /C=IT/ST=Bolzano/L=Bolzano/O=GlobalSecurity/OU=NeteyeInfluxDB/CN=influxdb.neteyelocal
Client-SSL-Cipher: TLS_AES_256_GCM_SHA384
Client-SSL-Socket-Class: IO::Socket::SSL
Client-Transfer-Encoding: chunked
Request-Id: 2df51688-cf35-11ed-95cc-0050569e3eff
X-Influxdb-Build: OSS
X-Influxdb-Version: 1.8.10
X-Request-Id: 2df51688-cf35-11ed-95cc-0050569e3eff

{"results":[{"statement_id":0,"series":[{"name":"system","tags":{"host":"neteye4testn1.wp.lan"},"columns":["time","load1"],"values":[[1680206260,1.72]]}]}]}
URL: 'https://influxdb.neteyelocal:8086/query?epoch=s'
Parameters: 'q=SELECT max(load5) AS load5 FROM "telegraf_master"."autogen"."system" WHERE host='neteye4testn1.wp.lan' AND time > (now() - 10m) GROUP BY host;'
======> request send
POST https://influxdb.neteyelocal:8086/query?epoch=s
Authorization: Basic dGVsZWdyYWZfbWFzdGVyX3JlYWRlcjo0bmY3MDkzbm1mODczNGpubWc1
User-Agent: centreon::plugins::backend::http::useragent
Content-Type: application/x-www-form-urlencoded

q=SELECT+max(load5)+AS+load5+FROM+%22telegraf_master%22.%22autogen%22.%22system%22+WHERE+host%3D'neteye4testn1.wp.lan'+AND+time+%3E+(now()+-+10m)+GROUP+BY+host%3B
======> response done
HTTP/1.1 200 OK
Date: Thu, 30 Mar 2023 19:58:02 GMT
Content-Type: application/json
Client-Date: Thu, 30 Mar 2023 19:58:02 GMT
Client-Peer: 192.168.232.34:8086
Client-Response-Num: 2
Client-SSL-Cert-Issuer: /DC=neteyelocal/DC=*/O=NetEye/OU=NetEye Root CA/CN=*.neteyelocal
Client-SSL-Cert-Subject: /C=IT/ST=Bolzano/L=Bolzano/O=GlobalSecurity/OU=NeteyeInfluxDB/CN=influxdb.neteyelocal
Client-SSL-Cipher: TLS_AES_256_GCM_SHA384
Client-SSL-Socket-Class: IO::Socket::SSL
Client-SSL-Warning: Peer certificate not verified
Client-Transfer-Encoding: chunked
Request-Id: 2df7ef10-cf35-11ed-95cd-0050569e3eff
X-Influxdb-Build: OSS
X-Influxdb-Version: 1.8.10
X-Request-Id: 2df7ef10-cf35-11ed-95cd-0050569e3eff

{"results":[{"statement_id":0,"series":[{"name":"system","tags":{"host":"neteye4testn1.wp.lan"},"columns":["time","load5"],"values":[[1680205690,1.45]]}]}]}
URL: 'https://influxdb.neteyelocal:8086/query?epoch=s'
Parameters: 'q=SELECT max(load15) AS load15 FROM "telegraf_master"."autogen"."system" WHERE host='neteye4testn1.wp.lan' AND time > (now() - 10m) GROUP BY host;'
======> request send
POST https://influxdb.neteyelocal:8086/query?epoch=s
Authorization: Basic dGVsZWdyYWZfbWFzdGVyX3JlYWRlcjo0bmY3MDkzbm1mODczNGpubWc1
User-Agent: centreon::plugins::backend::http::useragent
Content-Type: application/x-www-form-urlencoded

q=SELECT+max(load15)+AS+load15+FROM+%22telegraf_master%22.%22autogen%22.%22system%22+WHERE+host%3D'neteye4testn1.wp.lan'+AND+time+%3E+(now()+-+10m)+GROUP+BY+host%3B
======> response done
HTTP/1.1 200 OK
Date: Thu, 30 Mar 2023 19:58:02 GMT
Content-Type: application/json
Client-Date: Thu, 30 Mar 2023 19:58:02 GMT
Client-Peer: 192.168.232.34:8086
Client-Response-Num: 3
Client-SSL-Cert-Issuer: /DC=neteyelocal/DC=*/O=NetEye/OU=NetEye Root CA/CN=*.neteyelocal
Client-SSL-Cert-Subject: /C=IT/ST=Bolzano/L=Bolzano/O=GlobalSecurity/OU=NeteyeInfluxDB/CN=influxdb.neteyelocal
Client-SSL-Cipher: TLS_AES_256_GCM_SHA384
Client-SSL-Socket-Class: IO::Socket::SSL
Client-SSL-Warning: Peer certificate not verified
Client-Transfer-Encoding: chunked
Request-Id: 2df83e8a-cf35-11ed-95ce-0050569e3eff
X-Influxdb-Build: OSS
X-Influxdb-Version: 1.8.10
X-Request-Id: 2df83e8a-cf35-11ed-95ce-0050569e3eff

{"results":[{"statement_id":0,"series":[{"name":"system","tags":{"host":"neteye4testn1.wp.lan"},"columns":["time","load15"],"values":[[1680205690,1.67]]}]}]}
System Load (1,5,15): 1.72 1.45 1.67
[root@neteye4testn1 ~]#

Hope this can help.

wp-perc commented 1 year ago

I found this old issue. Closing because is a duplicate of https://github.com/centreon/centreon-plugins/issues/4572.