Missing configuration / environment variable to force the icinga to pick it's api identity

[dwt]

When starting the docker container it will auto create it's own configuration via environment variables. This can be re-executed with icinga2 node wizard which also shows that the correct values are taken from the environment.

In my case, that is 'ops.my.domain' as the ICINGA_CN which leads to generated config files like this:

root@ops:/etc/icinga2# cat constants.conf
[...]
const NodeName = "ops.my.domain"
const ZoneName = "master"
const TicketSalt = "random_salt"

root@ops:/etc/icinga2# cat zones.conf
object Endpoint "ops.my.domain" {}
object Zone "master" {
    endpoints = [ "ops.my.domain" ]
}
[...]

However, when verifying the configuration via icinga2 daemon --validate this setup generates this error:

root@ops:/etc/icinga2# icinga2 daemon --validate
[2021-05-04 08:15:01 +0000] warning/Application: Failed to adjust resource limit for open file handles (RLIMIT_NOFILE) with error "Operation not permitted"
[2021-05-04 08:15:01 +0000] warning/Application: Failed adjust resource limit for number of processes (RLIMIT_NPROC) with error "Operation not permitted"
[2021-05-04 08:15:01 +0000] warning/Application: Failed to adjust resource limit for open file handles (RLIMIT_NOFILE) with error "Operation not permitted"
[2021-05-04 08:15:01 +0000] warning/Application: Failed adjust resource limit for number of processes (RLIMIT_NPROC) with error "Operation not permitted"
[2021-05-04 08:15:01 +0000] information/cli: Icinga application loader (version: r2.12.0-1)
[2021-05-04 08:15:01 +0000] information/cli: Loading configuration file(s).
[2021-05-04 08:15:01 +0000] information/ConfigItem: Committing config item(s).
[2021-05-04 08:15:01 +0000] information/ApiListener: My API identity: 9bc786c2237a
[2021-05-04 08:15:01 +0000] critical/config: Error: Endpoint object for '9bc786c2237a' is missing.
Location: in /etc/icinga2/features-enabled/api.conf: 4:1-4:24
/etc/icinga2/features-enabled/api.conf(2):  * The API listener is used for distributed monitoring setups.
/etc/icinga2/features-enabled/api.conf(3):  */
/etc/icinga2/features-enabled/api.conf(4): object ApiListener "api" {
                                           ^^^^^^^^^^^^^^^^^^^^^^^^
/etc/icinga2/features-enabled/api.conf(5): 
/etc/icinga2/features-enabled/api.conf(6):   ticket_salt = TicketSalt
[2021-05-04 08:15:01 +0000] critical/config: 1 error
[2021-05-04 08:15:01 +0000] critical/cli: Config validation failed. Re-run with 'icinga2 daemon -C' after fixing the config.

AFAIK the important bit is this:

[2021-05-04 08:15:01 +0000] information/ApiListener: My API identity: 9bc786c2237a
[2021-05-04 08:15:01 +0000] critical/config: Error: Endpoint object for '9bc786c2237a' is missing.

where it tells me that icinga seems to think it's api identity is 9bc786c2237aand that it has no endpoint configured. Where does this setting even come from? How do I override it? It doesn't seem to come from the system configuration as that is different

root@ops:/etc/icinga2# hostname
ops
root@ops:/etc/icinga2# hostname -f 
ops.my.doamin

I am missing an environment variable (or config if need be) to force icinga to assume the API identity ops.my.domain.

Maybe there is such a variable, but I can't find it in the documentation of this docker image and also not in the documentation of icinga proper. So, what do you think?

[dwt]

Ping? Is there any additional information I can give to help this along?

[dwt]

ping? I'm still struggling with this problem. Is there any additional context I can provide?

[julianbrost]

So Icinga takes its API identity from the CN of its own certificate. So in case you didn't specify ICINGA_CN when that was generated (usually the first time the container was started), then you'll end up with the hostname as the identity. So in this case you have to regenerate the certificates.

However, I'm surprised that Icinga even finds the certificate as the filename includes the CN. What are the contents of /var/lib/icinga2/certs and did you rename any files there?

[dwt]

That folder contains certificates with the wrong name, i.e. these:

958a9c902c68.crt
958a9c902c68.csr
958a9c902c68.key

No files where renamed here.

The *.crt also has the 'wrong' X509v3 Subject Alternative Name: DNS:958a9c902c68. And no, I did not rename any files there. I will try regenerating the certificate though and see where that takes me.

[dwt]

I have regenerated the certificates (using icinga2 node wizard), with the docker-compose settings

    hostname: ops
    domainname: mydomain.example

And now icinga seems to see the correct identity.

What. I don't get is why the old identity (from the certificate) was found in the past, even though the hostname was already different.

One thing that might help, I had these settings

    hostname: ops
    domainname: ops.mydomain.example

Which where likely wrong (leading icinga to deduce ops.ops.mydomain.example as the CN). Even though the hostname command parsed that correctly.

Still, that does not explain at all why icinga still found the old certificate and deduced the NodeName from that. Is that a separate bug, or something you would like to handle as part of this bug?

[Al2Klimov]

Glad to hear that it works now. Likely the container was started just once without a hostname which ruined it.

The container (...) expects (...) a specific persistent hostname.

– README