Status of services (pushed through api) show after icinga2 service was reload

de1m commented 5 years ago

Describe the bug

The error happens only on first start of my container. The container start without any errors und icinga2 can receive data through NSCA (from Windows Server) without errors. I've a cuple of scripts, written in nodejs (icinga2 modul). This scripts create icinga2 objects (hosts and services) and set state of they through icinga2 api. The error is, that the state of this object will be seeable(in icinga2 webgui) after I run the command "service icinga2 reload"

To Reproduce

Start container (show the readme
Create an host and server of them through api
Show the state in webgui

Your Environment

Include as many relevant details about the environment you experienced the problem in

Version used (icinga2 --version): version: r2.10.5-1

Operating System and version:

System information:
Platform: Ubuntu
Platform version: 18.04.2 LTS (Bionic Beaver)
Kernel: Linux
Kernel version: 4.15.0-47-generic
Architecture: x86_64`

Enabled features (icinga2 feature list):

root@monitoring:/# icinga2 feature list
Disabled features: compatlog debuglog elasticsearch gelf ido-myql influxdb livestatus mainlog opentsdb perfdata statusdata syslog
Enabled features: api checker command graphite ido-mysql notification

Icinga Web 2 version and modules (System - About): 2.6.3

Config validation (icinga2 daemon -C):


root@monitoring:/# icinga2 daemon -C
[2019-06-05 08:14:58 +0000] information/cli: Icinga application loader (version: r2.10.5-1)
[2019-06-05 08:14:58 +0000] information/cli: Loading configuration file(s).
[2019-06-05 08:14:58 +0000] information/ConfigItem: Committing config item(s).
[2019-06-05 08:14:58 +0000] information/ApiListener: My API identity: 30b5d7050d6f
[2019-06-05 08:14:58 +0000] warning/ApplyRule: Apply rule 'backup-downtime' (in /etc/icinga2/conf.d/downtimes.conf: 5:1-5:52) for type 'ScheduledDowntime' does not match anywhere!
[2019-06-05 08:14:58 +0000] information/ConfigItem: Instantiated 384 Services.
[2019-06-05 08:14:58 +0000] information/ConfigItem: Instantiated 1 IcingaApplication.
[2019-06-05 08:14:58 +0000] information/ConfigItem: Instantiated 34 Hosts.
[2019-06-05 08:14:58 +0000] information/ConfigItem: Instantiated 3 NotificationCommands.
[2019-06-05 08:14:58 +0000] information/ConfigItem: Instantiated 418 Notifications.
[2019-06-05 08:14:58 +0000] information/ConfigItem: Instantiated 1 NotificationComponent.
[2019-06-05 08:14:58 +0000] information/ConfigItem: Instantiated 3 HostGroups.
[2019-06-05 08:14:58 +0000] information/ConfigItem: Instantiated 1 ApiListener.
[2019-06-05 08:14:58 +0000] information/ConfigItem: Instantiated 1 GraphiteWriter.
[2019-06-05 08:14:58 +0000] information/ConfigItem: Instantiated 1 CheckerComponent.
[2019-06-05 08:14:58 +0000] information/ConfigItem: Instantiated 3 Zones.
[2019-06-05 08:14:58 +0000] information/ConfigItem: Instantiated 1 ExternalCommandListener.
[2019-06-05 08:14:58 +0000] information/ConfigItem: Instantiated 1 Endpoint.
[2019-06-05 08:14:58 +0000] information/ConfigItem: Instantiated 4 ApiUsers.
[2019-06-05 08:14:58 +0000] information/ConfigItem: Instantiated 2 Users.
[2019-06-05 08:14:58 +0000] information/ConfigItem: Instantiated 1 IdoMysqlConnection.
[2019-06-05 08:14:58 +0000] information/ConfigItem: Instantiated 215 CheckCommands.
[2019-06-05 08:14:58 +0000] information/ConfigItem: Instantiated 1 UserGroup.
[2019-06-05 08:14:58 +0000] information/ConfigItem: Instantiated 4 ServiceGroups.
[2019-06-05 08:14:58 +0000] information/ConfigItem: Instantiated 4 TimePeriods.
[2019-06-05 08:14:58 +0000] information/ScriptGlobal: Dumping variables to file '/var/cache/icinga2/icinga2.vars'
[2019-06-05 08:14:58 +0000] information/cli: Finished validating the configuration file(s).

If you run multiple Icinga 2 instances, the zones.conf file (or icinga2 object list --type Endpoint and icinga2 object list --type Zone) from all affected nodes.


## Additional context
I think the error happens because I create the host, then server and set state of this and all of this very fast (in script).
All logs are empty.

dnsmichi commented 5 years ago

set state of they through icinga2 api.

I assume that a new check result is being pushed via API action. If you're manually modifying the state attribute somehow, this won't trigger any runtime updates towards backends (IDO, etc.).

de1m commented 5 years ago

okay, but after I've reload the icinga2 its working correctly. Can I do somephing, for debug or testing?

dnsmichi commented 5 years ago

So you're modifying the state attribute in your library? This is not supported by design.

de1m commented 5 years ago

What do you mean with "state attribute"? "Custom Variables" in Icinga2 webgui? or the "vars" in config file?

object Host "k8s-ext-pods-monitoring" {
        import "passive-host"

        display_name = "k8s-ext-pods-monitoring"
        vars["group"] = "adito"
        vars["server"] = "k8s"
        version = 1559720645.693325
        zone = "30b5d7050d6f"
}

de1m commented 5 years ago

hm... I've changed my Script, that create now this object

object Host "k8s-ext-pods-monitoring" {
        import "passive-host"

        display_name = "k8s-ext-pods-monitoring"
        version = 1560243611.427802
        zone = "30b5d7050d6f"
}

but I get the same error

dnsmichi commented 5 years ago

Hm, I think I misunderstood the meaning of "set a state", which is done via the process-check-result action. The host object and its changed state is visible via REST API, right?

Then I'd point to IDO activation bugs collected in #6012.

lippserd commented 4 years ago

Hi @de1m,

If you still have this problem, please open a new issue. I close this one in the meantime.

For the records, I doubt that it is a good idea to add version and zone to the configuration. But that is just a blind guess.

All the best, Eric

Icinga / icinga2