[Thruk v3.04 | Icinga2 v2.13.7] no client certificate | unknow ca certificate

ulrichmonji commented 1 year ago

Hello,

Apologize if my post is not in a good place. Do not hesitate to tell where i can post if needed...

My problem is, I would like to connect my icinga2 server as Thruk backend. But i am facing some TLS issues, (no client certificate).

Errors messages

On thruk IHM, when i tried to connect to icinga2 by http type, there is my error on icinga2 server:

[2023-04-20 09:52:50 +0000] information/HttpServerConnection: Request: POST /thruk/cgi-bin/remote.cgi (from [::ffff:172.18.0.3]:55270), user: root, agent: thruk, status: Bad Request).
[2023-04-20 09:52:51 +0000] information/HttpServerConnection: Request: POST /thruk/cgi-bin/remote.cgi (from [::ffff:172.18.0.3]:55272), user: root, agent: thruk, status: Bad Request).
[2023-04-20 09:52:57 +0000] information/HttpServerConnection: Request: POST /thruk/cgi-bin/remote.cgi (from [::ffff:172.18.0.3]:55274), user: root, agent: thruk, status: Bad Request).
[2023-04-20 09:52:57 +0000] information/HttpServerConnection: Request: POST /thruk/cgi-bin/remote.cgi (from [::ffff:172.18.0.3]:55276), user: root, agent: thruk, status: Bad Request).

On thruk IHM, when i tried to connect to icinga2 by livestatus type, there is my error on icinga2 server:

[2023-04-20 10:09:53 +0000] information/HttpServerConnection: Request: POST /thruk/cgi-bin/remote.cgi (from [::ffff:172.18.0.3]:55436), user: root, agent: thruk, status: Bad Request).
[2023-04-20 10:09:53 +0000] critical/ApiListener: Client TLS handshake failed (from [::ffff:172.18.0.3]:55438): http request
[2023-04-20 10:09:54 +0000] critical/ApiListener: Client TLS handshake failed (from [::ffff:172.18.0.3]:55440): http request
[2023-04-20 10:09:55 +0000] critical/ApiListener: Client TLS handshake failed (from [::ffff:172.18.0.3]:55444): http reques

Platform caracteristics

I am working on docker containers, i created two containers, One is icinga-master via icinga2 image and the other is Thruk via custom image builded. Both of them ran on the same bridge docker network.

Icinga configuration

Launching icinga containers:

docker network create icinga 
docker run -d --network icinga --name icinga-master -h icinga-master -p 9000:5665 -v icinga-master:/data -e ICINGA_MASTER=1 icinga/icinga2:latest

icinga2 version :


icinga@icinga-master:/$ icinga2 --version 
icinga2 - The Icinga 2 network monitoring daemon (version: v2.13.7)

Copyright (c) 2012-2023 Icinga GmbH (https://icinga.com/) License GPLv2+: GNU GPL version 2 or later https://gnu.org/licenses/gpl2.html This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law.

System information: Platform: Debian GNU/Linux Platform version: 11 (bullseye) Kernel: Linux Kernel version: 3.10.0-1160.88.1.el7.x86_64 Architecture: x86_64

Build information: Compiler: GNU 10.2.1 Build host: buildkitsandbox OpenSSL version: OpenSSL 1.1.1n 15 Mar 2022

* Operating System and version
```bash
icinga@icinga-master:/$ cat /etc/os-release 
PRETTY_NAME="Debian GNU/Linux 11 (bullseye)"
NAME="Debian GNU/Linux"
VERSION_ID="11"
VERSION="11 (bullseye)"
VERSION_CODENAME=bullseye
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"

Icinga2 Container kernel

icinga@icinga-master:/$ uname  -a 
Linux icinga-master 3.10.0-1160.88.1.el7.x86_64 #1 SMP Tue Mar 7 15:41:52 UTC 2023 x86_64 GNU/Linux

Enabled features (icinga2 feature list)

icinga@icinga-master:/$ icinga2 feature list
Disabled features: debuglog elasticsearch gelf graphite icingadb ido-mysql ido-pgsql influxdb influxdb2 mainlog opentsdb perfdata syslog
Enabled features: api checker notification

Config validation (icinga2 daemon -C)

icinga@icinga-master:/$ icinga2 daemon -C
[2023-04-20 09:15:01 +0000] information/cli: Icinga application loader (version: v2.13.7)
[2023-04-20 09:15:01 +0000] information/cli: Loading configuration file(s).
[2023-04-20 09:15:01 +0000] information/ConfigItem: Committing config item(s).
[2023-04-20 09:15:01 +0000] information/ApiListener: My API identity: icinga-master
[2023-04-20 09:15:02 +0000] information/ConfigItem: Instantiated 1 Host.
[2023-04-20 09:15:02 +0000] information/ConfigItem: Instantiated 2 NotificationCommands.
[2023-04-20 09:15:02 +0000] information/ConfigItem: Instantiated 1 Downtime.
[2023-04-20 09:15:02 +0000] information/ConfigItem: Instantiated 12 Notifications.
[2023-04-20 09:15:02 +0000] information/ConfigItem: Instantiated 1 IcingaApplication.
[2023-04-20 09:15:02 +0000] information/ConfigItem: Instantiated 2 HostGroups.
[2023-04-20 09:15:02 +0000] information/ConfigItem: Instantiated 3 Zones.
[2023-04-20 09:15:02 +0000] information/ConfigItem: Instantiated 1 Endpoint.
[2023-04-20 09:15:02 +0000] information/ConfigItem: Instantiated 1 NotificationComponent.
[2023-04-20 09:15:02 +0000] information/ConfigItem: Instantiated 1 CheckerComponent.
[2023-04-20 09:15:02 +0000] information/ConfigItem: Instantiated 1 ApiUser.
[2023-04-20 09:15:02 +0000] information/ConfigItem: Instantiated 1 ApiListener.
[2023-04-20 09:15:02 +0000] information/ConfigItem: Instantiated 244 CheckCommands.
[2023-04-20 09:15:02 +0000] information/ConfigItem: Instantiated 3 TimePeriods.
[2023-04-20 09:15:02 +0000] information/ConfigItem: Instantiated 1 UserGroup.
[2023-04-20 09:15:02 +0000] information/ConfigItem: Instantiated 1 User.
[2023-04-20 09:15:02 +0000] information/ConfigItem: Instantiated 11 Services.
[2023-04-20 09:15:02 +0000] information/ConfigItem: Instantiated 3 ServiceGroups.
[2023-04-20 09:15:02 +0000] information/ConfigItem: Instantiated 1 ScheduledDowntime.
[2023-04-20 09:15:02 +0000] information/ScriptGlobal: Dumping variables to file '/var/cache/icinga2/icinga2.vars'
[2023-04-20 09:15:02 +0000] information/cli: Finished validating the configuration file(s).

Thruk Configuration

Thruk Dockerfile:

FROM ubuntu:22.04
# https://download.thruk.org/pkg/v3.04/ubuntu22.04/amd64/
COPY packages /tmp/
COPY docker-entrypoint.sh /usr/local/bin/
RUN apt-get update -y
RUN apt-get install apache2 libapache2-mod-fcgid -y
RUN dpkg -i /tmp/libthruk_3.00_ubuntu22.04_amd64.deb \
      /tmp/thruk-base_3.04-1_ubuntu22.04_amd64.deb \
      /tmp/thruk_3.04-1_ubuntu22.04_amd64.deb \
      /tmp/thruk-plugin-reporting_3.04-1_ubuntu22.04_amd64.deb || true
RUN apt-get install -f -y
EXPOSE 80
# ENTRYPOINT ["/etc/init.d/apache2", "start"]
ENTRYPOINT ["/usr/local/bin/docker-entrypoint.sh"]

Launching Thruk containers:

docker run -dit --name thruk -h thruk --network icinga -p 80:80 --restart=always my-thruk-image

Thruk version

root@thruk:/# thruk --version
thruk v3.04

Copying icinga2 certificates to thruk containers

[icinga@devopsbox icinga2]$ docker cp icinga-master:/var/lib/icinga2/certs/ca.crt  . 
Successfully copied 3.584kB to /home/icinga/icinga2/.
[icinga@devopsbox icinga2]$ docker cp icinga-master:/var/lib/icinga2/certs/icinga-master.crt  . 
Successfully copied 3.584kB to /home/icinga/icinga2/.
[icinga@devopsbox icinga2]$ docker cp icinga-master:/var/lib/icinga2/certs/icinga-master.key  . 
Successfully copied 5.12kB to /home/icinga/icinga2/.
[icinga@devopsbox icinga2]$ docker cp ./ca.crt thruk:/tmp/
Successfully copied 3.584kB to thruk:/tmp/
[icinga@devopsbox icinga2]$ docker cp ./icinga-master.crt thruk:/tmp/
Successfully copied 3.584kB to thruk:/tmp/
[icinga@devopsbox icinga2]$ docker cp ./icinga-master.key thruk:/tmp/
Successfully copied 5.12kB to thruk:/tmp/

Permission on icinga certificates on thruk containers (Read permissions seted) root@thruk:/# ls /tmp/ -l

total 29556
-rw-r--r--. 1 5665 5665     1720 Apr 20 09:09 ca.crt
-rw-r--r--. 1 5665 5665     1757 Apr 20 09:09 icinga-master.crt
-rw-r--r--. 1 5665 5665     3243 Apr 20 09:09 icinga-master.key

I added Icinga CA to trustore of thruk containers :

root@thruk:/# cp /tmp/ca.crt  /usr/local/share/ca-certificates/
root@thruk:/# update-ca-certificates 
Updating certificates in /etc/ssl/certs...
rehash: warning: skipping ca-certificates.crt,it does not contain exactly one certificate or CRL
1 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.

I Enabled Icinga feature on Thruk config (/etc/thruk/thruk.conf)

root@thruk:/# grep enable_icinga_features  /etc/thruk/thruk.conf
#enable_icinga_features = 0
enable_icinga_features = 1

I created an agent ticket for icinga API

root@thruk:/# curl -k -s -S -i -u root:828d95c660d274c3 -H 'Accept: application/json' \
-X POST 'https://icinga-master:5665/v1/actions/generate-ticket' \
-d '{ "cn": "icinga-agent", "pretty": true }'
HTTP/1.1 200 OK
Server: Icinga/v2.13.7
Content-Type: application/json
Content-Length: 258

{
  "results": [
      {
          "code": 200,
          "status": "Generated PKI ticket '963ad4231128b81e2f25a5e9a1fc0709d0ec45a5' for common name 'icinga-agent'.",
          "ticket": "963ad4231128b81e2f25a5e9a1fc0709d0ec45a5"
      }
  ]
}

Here is my Thruk config (/etc/thruk/thruk_local.conf)

root@thruk:/# cat /etc/thruk/thruk_local.conf 
ssl_verify_hostnames = 0
<Component Thruk::Backend>
<peer>
    name    = icinga2-livestatus
    id      = 36a9e
    type    = livestatus
    <options>
        peer          = tls://icinga-master:5665
    </options>
</peer>
<peer>
    name    = icinga-http
    type    = http
    <options>
        peer          = https://root:828d95c660d274c3@icinga-master:5665  # apiser credentials are: login: root, password: 828d95c660d274c3
        auth          = 963ad4231128b81e2f25a5e9a1fc0709d0ec45a5  # ticket generated
        ca_file       = /tmp/ca.crt
        cert          = /tmp/icinga-master.crt
        key           = /tmp/icinga-master.key
    </options>
</peer>
</Component>

sni commented 1 year ago

thruk cannot talk to the icinga api, Thruk uses livestatus only. You need to enable the livestatus module and connect to that one. No certificate required for that.

ulrichmonji commented 1 year ago

Thanks for your reply,

In fact, it is Ok via livestatus Module. The module were not installed on my icinga2 containers, i rebuilt a new icinga2 image which contains this module and ewerything works fine

On future release, is Thruk could talk to icinga api, or it is not on roadmap ?

Regards

sni commented 1 year ago

There are no plans to support the icinga api. Livestatus is way faster and better.

sni / Thruk