centreon / centreon-collect

Centreon collect software collection
11 stars 6 forks source link

No Engine statistics and broker statistics because gorgone doesn't start always with cron definitions #1868

Open rmorandell-pgum opened 2 days ago

rmorandell-pgum commented 2 days ago

BUG REPORT INFORMATION

Versions

centreon-broker-selinux-23.10.11-1.el8.x86_64 centreon-plugin-Hardware-Ups-Standard-Rfc1628-Snmp-20241010-130148.el8.noarch centreon-plugins-selinux-0.0.8-083704.el8.x86_64 centreon-widget-live-top10-memory-usage-23.10.0-1.el8.noarch centreon-broker-cbmod-23.10.11-1.el8.x86_64 centreon-plugin-Applications-Protocol-Dns-20241010-130148.el8.noarch centreon-widget-global-health-23.10.0-1.el8.noarch centreon-widget-httploader-23.10.0-1.el8.noarch centreon-trap-23.10.17-1.el8.noarch centreon-license-manager-common-23.10.2-1.el8.noarch centreon-plugin-Applications-Monitoring-Centreon-Database-20241010-130148.el8.noarch centreon-widget-servicegroup-monitoring-23.10.0-1.el8.noarch centreon-clib-23.10.11-1.el8.x86_64 centreon-plugin-Applications-Protocol-Ldap-20241010-130148.el8.noarch centreon-widget-host-monitoring-23.10.1-1.el8.noarch centreon-connector-perl-23.10.11-1.el8.x86_64 centreon-engine-selinux-23.10.11-1.el8.x86_64 centreon-plugin-Applications-Protocol-Http-20241010-130148.el8.noarch centreon-widget-tactical-overview-23.10.0-1.el8.noarch centreon-central-23.10.17-1.el8.noarch centreon-auto-discovery-server-23.10.4-1.el8.noarch centreon-plugin-Applications-Monitoring-Centreon-Poller-20241010-130148.el8.noarch centreon-plugin-Operatingsystems-Linux-Snmp-20241010-130148.el8.noarch centreon-widget-single-metric-23.10.0-1.el8.noarch centreon-widget-engine-status-23.10.0-1.el8.noarch centreon-plugin-Virtualization-VMWare-daemon-3.3.2-1.el8.noarch centreon-gorgone-centreon-config-23.10.10-1.el8.noarch centreon-database-23.10.17-1.el8.noarch centreon-engine-daemon-23.10.11-1.el8.x86_64 centreon-it-edition-extensions-23.10.4-1.el8.noarch centreon-gorgoned-selinux-23.10.10-1.el8.noarch centreon-plugin-Hardware-Printers-Generic-Snmp-20241010-130148.el8.noarch centreon-poller-23.10.17-1.el8.noarch centreon-broker-23.10.11-1.el8.x86_64 centreon-widget-live-top10-cpu-usage-23.10.0-1.el8.noarch centreon-broker-cbd-23.10.11-1.el8.x86_64 centreon-plugin-Applications-Databases-Mysql-20241010-130148.el8.noarch centreon-web-23.10.17-1.el8.noarch centreon-plugin-Applications-Monitoring-Centreon-Central-20241010-130148.el8.noarch centreon-perl-libs-23.10.17-1.el8.noarch centreon-connector-ssh-23.10.11-1.el8.x86_64 centreon-web-selinux-23.10.17-1.el8.noarch centreon-plugin-Operatingsystems-Windows-Snmp-20241010-130148.el8.noarch centreon-widget-hostgroup-monitoring-23.10.0-1.el8.noarch centreon-broker-core-23.10.11-1.el8.x86_64 centreon-connector-23.10.11-1.el8.x86_64 centreon-plugin-Network-Cisco-Standard-Snmp-20241010-130148.el8.noarch centreon-widget-graph-monitoring-23.10.0-1.el8.noarch centreon-23.10.17-1.el8.noarch centreon-pp-manager-23.10.3-1.el8.noarch centreon-license-manager-23.10.2-1.el8.noarch centreon-plugin-Applications-Monitoring-Centreon-Map4-Jmx-20241010-130148.el8.noarch centreon-widget-grid-map-23.10.0-1.el8.noarch centreon-common-23.10.17-1.el8.noarch centreon-plugin-Applications-Protocol-Ftp-20241010-130148.el8.noarch centreon-widget-service-monitoring-23.10.0-1.el8.noarch centreon-gorgone-23.10.10-1.el8.noarch centreon-common-selinux-23.10.17-1.el8.noarch centreon-engine-23.10.11-1.el8.x86_64

Operating System

Oracle Linux 8.

How the the component has been installed and versions

Description

The gorgone statistics module has a yaml file containing the cron definitions to get every 5 minutes the broker and engine statistics.

- name: statistics
  package: "gorgone::modules::centreon::statistics::hooks"
  enable: true
  broker_cache_dir: "/var/cache/centreon//broker-stats/"
  cron:
    - id: broker_stats
      timespec: "*/1 * * * *"
      action: BROKERSTATS
      parameters:
        timeout: 10
    - id: engine_stats
      timespec: "*/1 * * * *"
      action: ENGINESTATS
      parameters:
        timeout: 10

On normal start of gorgone in the log you can see the rows:

2024-11-14 12:09:11 - INFO - [cron] Adding cron definition 'broker_stats' 2024-11-14 12:09:11 - INFO - [cron] Adding cron definition 'engine_stats'

But not always the gorgone is adding this cron definitions. When you try to restart gorgone 20 times 1-2 times the cron definitions are missing what has the effect that in the Plattform status no engine statistics and broker statistics will be shown.

Logs

Here is a log that contains two restarts .. on the first start the cronis missing. On the second restart 30 seconds after the first one the 2 crons has been added correctly.

2024-11-14 11:47:58 - INFO - [core] Initialize server keys
2024-11-14 11:47:58 - INFO - [core] Private key file '/var/lib/centreon-gorgone/.keys/rsakey.priv.pem' loaded
2024-11-14 11:47:58 - INFO - [core] Public key file '/var/lib/centreon-gorgone/.keys/rsakey.pub.pem' loaded
2024-11-14 11:47:58 - INFO - [core] update schema 23.10.0 -> 23.10.0
2024-11-14 11:47:58 - INFO - [core] Module 'action' is loading
2024-11-14 11:47:58 - INFO - [core] Module 'action' is loaded
2024-11-14 11:47:58 - INFO - [core] Module 'httpserver' is loading
2024-11-14 11:47:58 - INFO - [core] Module 'httpserver' is loaded
2024-11-14 11:47:58 - INFO - [core] Module 'cron' is loading
2024-11-14 11:47:58 - INFO - [core] Module 'cron' is loaded
2024-11-14 11:47:58 - INFO - [core] Module 'nodes' is loading
2024-11-14 11:47:58 - INFO - [core] Module 'nodes' is loaded
2024-11-14 11:47:58 - INFO - [core] Module 'proxy' is loading
2024-11-14 11:47:58 - INFO - [core] Module 'proxy' is loaded
2024-11-14 11:47:58 - INFO - [core] Module 'pipeline' is loading
2024-11-14 11:47:58 - INFO - [core] Module 'pipeline' is loaded
2024-11-14 11:47:58 - INFO - [core] Module 'legacycmd' is loading
2024-11-14 11:47:58 - INFO - [core] Module 'legacycmd' is loaded
2024-11-14 11:47:58 - INFO - [core] Module 'engine' is loading
2024-11-14 11:47:58 - INFO - [core] Module 'engine' is loaded
2024-11-14 11:47:58 - INFO - [core] Module 'statistics' is loading
2024-11-14 11:47:58 - INFO - [core] Module 'statistics' is loaded
2024-11-14 11:47:58 - INFO - [core] Module 'audit' is loading
2024-11-14 11:47:58 - INFO - [core] Module 'audit' is loaded
2024-11-14 11:47:58 - INFO - [core] Module 'dbcleaner' is loading
2024-11-14 11:47:58 - INFO - [core] Module 'dbcleaner' is loaded
2024-11-14 11:47:58 - INFO - [core] Gorgoned started
2024-11-14 11:47:58 - INFO - [core] PID 27474
2024-11-14 11:47:58 - INFO - [engine] Create module 'engine' process
2024-11-14 11:47:58 - INFO - [action] Create module 'action' process
2024-11-14 11:47:58 - INFO - [statistics] Create module 'statistics' process
2024-11-14 11:47:58 - INFO - [httpserver] Create module 'httpserver' process
2024-11-14 11:47:58 - INFO - [nodes] Create module 'nodes' process
2024-11-14 11:47:58 - INFO - [audit] Create module 'audit' process
2024-11-14 11:47:58 - INFO - [dbcleaner] Create module 'dbcleaner' process
2024-11-14 11:47:58 - INFO - [proxy] Create module 'proxy' child process for pool id '1'
2024-11-14 11:47:58 - INFO - [proxy] Create module 'proxy' child process for pool id '2'
2024-11-14 11:47:58 - INFO - [proxy] Create module 'proxy' child process for pool id '3'
2024-11-14 11:47:58 - INFO - [proxy] Create module 'proxy' child process for pool id '4'
2024-11-14 11:47:58 - INFO - [proxy] Create module 'proxy' child process for pool id '5'
2024-11-14 11:47:58 - INFO - [proxy] Create module 'proxy' child process for pool id '6'
2024-11-14 11:47:58 - INFO - [proxy] Create module 'proxy' child process for pool id '7'
2024-11-14 11:47:58 - INFO - [proxy] Create module 'proxy' child process for pool id '8'
2024-11-14 11:47:58 - INFO - [legacycmd] Create module 'legacycmd' process
2024-11-14 11:47:58 - INFO - [pipeline] Create module 'pipeline' process
2024-11-14 11:47:58 - INFO - [cron] Create module 'cron' process
2024-11-14 11:47:58 - INFO - [core] Server accepting clients
2024-11-14 11:47:59 - INFO - [core] module key gorgone-action changed
2024-11-14 11:47:59 - INFO - [core] module key gorgone-engine changed
2024-11-14 11:47:59 - INFO - [core] module key gorgone-statistics changed
2024-11-14 11:47:59 - INFO - [core] module key gorgone-proxy-6 changed
2024-11-14 11:47:59 - INFO - [core] module key gorgone-audit changed
2024-11-14 11:47:59 - INFO - [core] module key gorgone-proxy-2 changed
2024-11-14 11:47:59 - INFO - [core] module key gorgone-proxy-4 changed
2024-11-14 11:47:59 - INFO - [core] module key gorgone-proxy-5 changed
2024-11-14 11:47:59 - INFO - [core] module key gorgone-proxy-3 changed
2024-11-14 11:47:59 - INFO - [core] module key gorgone-cron changed
2024-11-14 11:47:59 - INFO - [core] module key gorgone-proxy-1 changed
2024-11-14 11:47:59 - INFO - [core] module key gorgone-pipeline changed
2024-11-14 11:47:59 - INFO - [core] module key gorgone-proxy-8 changed
2024-11-14 11:47:59 - INFO - [core] module key gorgone-dbcleaner changed
2024-11-14 11:47:59 - INFO - [core] module key gorgone-proxy-7 changed
2024-11-14 11:47:59 - INFO - [core] module key gorgone-nodes changed
2024-11-14 11:47:59 - INFO - [proxy] Node '2' is registered
2024-11-14 11:47:59 - INFO - [core] module key gorgone-httpserver changed
2024-11-14 11:47:59 - INFO - [core] module key gorgone-legacycmd changed
2024-11-14 11:48:18 - INFO - [proxy] connect for 2
2024-11-14 11:48:18 - INFO - [clientzmq] gorgone-proxy-1-2 - Client connected successfully to 'tcp://192.168.140.6:5556'
2024-11-14 11:48:18 - INFO - [proxy] Pong received from '2'
2024-11-14 11:48:26 - INFO - [httpserver] 127.0.0.1 GET '/api/internal/constatus' 'centreon::plugins::backend::http::useragent'
2024-11-14 11:48:26 - INFO - [api] Requesting '/api/internal/constatus' [GET]
2024-11-14 11:49:04 - INFO - [httpserver] 27486 has quit
2024-11-14 11:49:04 - INFO - [cron] 27500 has quit
2024-11-14 11:49:08 - INFO - [core] 27474 Receiving order to stop...
2024-11-14 11:49:08 - INFO - [action] 27484 Receiving order to stop...
2024-11-14 11:49:08 - INFO - [action] 27484 has quit
2024-11-14 11:49:08 - INFO - [engine] 27483 Receiving order to stop...
2024-11-14 11:49:08 - INFO - [engine] 27483 has quit
2024-11-14 11:49:09 - INFO - [statistics] 27485 Receiving order to stop...
2024-11-14 11:49:09 - INFO - [statistics] 27485 has quit
2024-11-14 11:49:09 - INFO - [proxy] 27495 Receiving order to stop...
2024-11-14 11:49:09 - INFO - [proxy] 27495 has quit
2024-11-14 11:49:09 - INFO - [proxy] Close control connection
2024-11-14 11:49:09 - INFO - [audit] 27488 has quit
2024-11-14 11:49:09 - INFO - [proxy] 27491 Receiving order to stop...
2024-11-14 11:49:09 - INFO - [proxy] 27491 has quit
2024-11-14 11:49:09 - INFO - [proxy] Close control connection
2024-11-14 11:49:09 - INFO - [proxy] 27493 Receiving order to stop...
2024-11-14 11:49:09 - INFO - [proxy] 27493 has quit
2024-11-14 11:49:09 - INFO - [proxy] Close control connection
2024-11-14 11:49:09 - INFO - [proxy] 27494 Receiving order to stop...
2024-11-14 11:49:09 - INFO - [proxy] 27494 has quit
2024-11-14 11:49:09 - INFO - [proxy] Close control connection
2024-11-14 11:49:09 - INFO - [proxy] 27492 Receiving order to stop...
2024-11-14 11:49:09 - INFO - [proxy] 27492 has quit
2024-11-14 11:49:09 - INFO - [proxy] Close control connection
2024-11-14 11:49:09 - INFO - [proxy] 27497 Receiving order to stop...
2024-11-14 11:49:09 - INFO - [proxy] 27497 has quit
2024-11-14 11:49:09 - INFO - [proxy] Close control connection
2024-11-14 11:49:09 - INFO - [pipeline] -class- 27499 has quit
2024-11-14 11:49:09 - INFO - [proxy] 27490 Receiving order to stop...
2024-11-14 11:49:09 - INFO - [proxy] 27490 has quit
2024-11-14 11:49:09 - INFO - [proxy] Close connection for 2
2024-11-14 11:49:09 - INFO - [proxy] Close internal connection for 2
2024-11-14 11:49:09 - INFO - [proxy] Close control connection
2024-11-14 11:49:09 - INFO - [dbcleaner] 27489 has quit
2024-11-14 11:49:09 - INFO - [nodes] -class- 27487 has quit
2024-11-14 11:49:09 - INFO - [proxy] 27496 Receiving order to stop...
2024-11-14 11:49:09 - INFO - [proxy] 27496 has quit
2024-11-14 11:49:09 - INFO - [proxy] Close control connection
2024-11-14 11:49:09 - INFO - [legacycmd] 27498 has quit
2024-11-14 11:49:13 - INFO - [core] Module 'gorgone::modules::core::action::hooks' is unloaded
2024-11-14 11:49:13 - INFO - [core] Module 'gorgone::modules::core::httpserver::hooks' is unloaded
2024-11-14 11:49:13 - INFO - [core] Module 'gorgone::modules::centreon::nodes::hooks' is unloaded
2024-11-14 11:49:13 - INFO - [core] Module 'gorgone::modules::centreon::audit::hooks' is unloaded
2024-11-14 11:49:13 - INFO - [core] Module 'gorgone::modules::core::dbcleaner::hooks' is unloaded
2024-11-14 11:49:13 - INFO - [core] Module 'gorgone::modules::centreon::legacycmd::hooks' is unloaded
2024-11-14 11:49:13 - INFO - [core] Module 'gorgone::modules::core::pipeline::hooks' is unloaded
2024-11-14 11:49:13 - INFO - [core] Module 'gorgone::modules::core::cron::hooks' is unloaded
2024-11-14 11:49:23 - INFO - [core] Module 'gorgone::modules::centreon::engine::hooks' is unloaded
2024-11-14 11:49:23 - INFO - [core] Module 'gorgone::modules::centreon::statistics::hooks' is unloaded
2024-11-14 11:49:23 - INFO - [core] Quit main process
2024-11-14 11:49:24 - INFO - [core] Initialize server keys
2024-11-14 11:49:24 - INFO - [core] Private key file '/var/lib/centreon-gorgone/.keys/rsakey.priv.pem' loaded
2024-11-14 11:49:24 - INFO - [core] Public key file '/var/lib/centreon-gorgone/.keys/rsakey.pub.pem' loaded
2024-11-14 11:49:24 - INFO - [core] update schema 23.10.0 -> 23.10.0
2024-11-14 11:49:24 - INFO - [core] Module 'action' is loading
2024-11-14 11:49:24 - INFO - [core] Module 'action' is loaded
2024-11-14 11:49:24 - INFO - [core] Module 'httpserver' is loading
2024-11-14 11:49:24 - INFO - [core] Module 'httpserver' is loaded
2024-11-14 11:49:24 - INFO - [core] Module 'cron' is loading
2024-11-14 11:49:24 - INFO - [core] Module 'cron' is loaded
2024-11-14 11:49:24 - INFO - [core] Module 'nodes' is loading
2024-11-14 11:49:24 - INFO - [core] Module 'nodes' is loaded
2024-11-14 11:49:24 - INFO - [core] Module 'proxy' is loading
2024-11-14 11:49:24 - INFO - [core] Module 'proxy' is loaded
2024-11-14 11:49:24 - INFO - [core] Module 'pipeline' is loading
2024-11-14 11:49:24 - INFO - [core] Module 'pipeline' is loaded
2024-11-14 11:49:24 - INFO - [core] Module 'legacycmd' is loading
2024-11-14 11:49:24 - INFO - [core] Module 'legacycmd' is loaded
2024-11-14 11:49:24 - INFO - [core] Module 'engine' is loading
2024-11-14 11:49:24 - INFO - [core] Module 'engine' is loaded
2024-11-14 11:49:24 - INFO - [core] Module 'statistics' is loading
2024-11-14 11:49:24 - INFO - [core] Module 'statistics' is loaded
2024-11-14 11:49:24 - INFO - [core] Module 'audit' is loading
2024-11-14 11:49:24 - INFO - [core] Module 'audit' is loaded
2024-11-14 11:49:24 - INFO - [core] Module 'dbcleaner' is loading
2024-11-14 11:49:24 - INFO - [core] Module 'dbcleaner' is loaded
2024-11-14 11:49:24 - INFO - [core] Gorgoned started
2024-11-14 11:49:24 - INFO - [core] PID 27847
2024-11-14 11:49:24 - INFO - [dbcleaner] Create module 'dbcleaner' process
2024-11-14 11:49:24 - INFO - [audit] Create module 'audit' process
2024-11-14 11:49:24 - INFO - [action] Create module 'action' process
2024-11-14 11:49:24 - INFO - [legacycmd] Create module 'legacycmd' process
2024-11-14 11:49:24 - INFO - [httpserver] Create module 'httpserver' process
2024-11-14 11:49:24 - INFO - [proxy] Create module 'proxy' child process for pool id '1'
2024-11-14 11:49:24 - INFO - [proxy] Create module 'proxy' child process for pool id '2'
2024-11-14 11:49:24 - INFO - [proxy] Create module 'proxy' child process for pool id '3'
2024-11-14 11:49:24 - INFO - [proxy] Create module 'proxy' child process for pool id '4'
2024-11-14 11:49:24 - INFO - [proxy] Create module 'proxy' child process for pool id '5'
2024-11-14 11:49:24 - INFO - [proxy] Create module 'proxy' child process for pool id '6'
2024-11-14 11:49:24 - INFO - [proxy] Create module 'proxy' child process for pool id '7'
2024-11-14 11:49:24 - INFO - [proxy] Create module 'proxy' child process for pool id '8'
2024-11-14 11:49:24 - INFO - [cron] Create module 'cron' process
2024-11-14 11:49:24 - INFO - [statistics] Create module 'statistics' process
2024-11-14 11:49:24 - INFO - [engine] Create module 'engine' process
2024-11-14 11:49:24 - INFO - [nodes] Create module 'nodes' process
2024-11-14 11:49:24 - INFO - [pipeline] Create module 'pipeline' process
2024-11-14 11:49:24 - INFO - [core] Server accepting clients
2024-11-14 11:49:24 - INFO - [core] module key gorgone-audit changed
2024-11-14 11:49:24 - INFO - [core] module key gorgone-action changed
2024-11-14 11:49:24 - INFO - [core] module key gorgone-dbcleaner changed
2024-11-14 11:49:24 - INFO - [core] module key gorgone-proxy-6 changed
2024-11-14 11:49:24 - INFO - [core] module key gorgone-proxy-1 changed
2024-11-14 11:49:24 - INFO - [core] module key gorgone-proxy-7 changed
2024-11-14 11:49:24 - INFO - [core] module key gorgone-httpserver changed
2024-11-14 11:49:24 - INFO - [core] module key gorgone-statistics changed
2024-11-14 11:49:24 - INFO - [core] module key gorgone-proxy-4 changed
2024-11-14 11:49:24 - INFO - [core] module key gorgone-proxy-5 changed
2024-11-14 11:49:24 - INFO - [core] module key gorgone-legacycmd changed
2024-11-14 11:49:24 - INFO - [core] module key gorgone-proxy-3 changed
2024-11-14 11:49:24 - INFO - [core] module key gorgone-cron changed
2024-11-14 11:49:24 - INFO - [core] module key gorgone-proxy-8 changed
2024-11-14 11:49:24 - INFO - [core] module key gorgone-proxy-2 changed
2024-11-14 11:49:24 - INFO - [core] module key gorgone-nodes changed
2024-11-14 11:49:24 - INFO - [core] module key gorgone-engine changed
2024-11-14 11:49:24 - INFO - [core] module key gorgone-pipeline changed
2024-11-14 11:49:24 - INFO - [proxy] Node '2' is registered
2024-11-14 11:49:25 - INFO - [cron] Adding cron definition 'broker_stats'
2024-11-14 11:49:25 - INFO - [cron] Adding cron definition 'engine_stats'
2024-11-14 11:50:00 - INFO - [cron] Launching job 'broker_stats'
2024-11-14 11:50:06 - INFO - [cron] Launching job 'engine_stats'
2024-11-14 11:50:06 - INFO - [statistics] Collecting Engine statistics from target '2'
2024-11-14 11:50:06 - INFO - [statistics] Collecting Engine statistics from target '1'
2024-11-14 11:50:06 - INFO - [proxy] connect for 2
2024-11-14 11:50:06 - INFO - [clientzmq] gorgone-proxy-1-2 - Client connected successfully to 'tcp://192.168.140.6:5556'
2024-11-14 11:50:06 - INFO - [proxy] Received setlogs for '2'
2024-11-14 11:50:14 - INFO - [proxy] Received setlogs for '2'
2024-11-14 11:50:24 - INFO - [proxy] Pong received from '2'
2024-11-14 11:50:44 - INFO - [proxy] Received setlogs for '2'
rmorandell-pgum commented 2 days ago

i found the problem ..

the statistic module adds the cron from his config (yaml) on "run" function on start of the module ..

if (defined($self->{config}->{cron})) { $self->send_internal_action({ action => 'ADDCRON', data => { content => $self->{config}->{cron} } }); }

when the cron module has not finished loading or starting this doesn't work.

workaround for me: move the cron part from the 40-gorgoned.yaml (-name: statistics part) to a own 40-statistics.yaml under cron.d directory.