Macro to generate Icinga host config

[n-rodriguez]

Hi there!

I've created a small jinja macro to convert YAML to Icinga config : https://github.com/jbox-web/icinga2-macro-formula/blob/master/icinga2-macro/macro.jinja

You convert this to this.

It's fully understood by Icinga2 and it's tested :)

There is one caveat : it cannot render empty hashes, so empty_hash: {} and ['a', 'sample', 'array', { with_empty_hash: {} } won't be rendered.

[myii]

Hi, @n-rodriguez! None of the links are working, they're all 404s -- perhaps the repo hasn't been set to public access?

[n-rodriguez]

@myii fixed

[myii]

@n-rodriguez Looks powerful! I don't use Icinga myself but I know @alxwr has done a lot of this formula. Let's see what he thinks about this.

[alxwr]

@n-rodriguez Interesting approach. I like the idea of just mapping data. This would make Icinga2 config "pillarstack-able". :-)

I've got to ask some questions to get a better understanding though:

• What problem are you trying to solve here?
• Is there are necessity (or valid and common use case) for empty hashes in Icinga2 config?
• Does that force us to have mapping only without the possibility of added intelligence (or sane defaults)?
  i.e. the default for http_address should come from the hosts's address, http_vhost's should come from the host's name, etc. (I plan on implementing sane defaults in the future and don't want this to be blocked.) So, do you see a possibility to still add some sort of pre-processing before mapping the data to YAML?

[alxwr]

@myii Thanks for the hint!

[n-rodriguez]

This would make Icinga2 config "pillarstack-able". :-)

Actually that's what I do

What problem are you trying to solve here?

I want to easily extend Icinga hosts checks

Is there are necessity (or valid and common use case) for empty hashes in Icinga2 config?

I don't think so, and until now I live with it :)

Does that force us to have mapping only without the possibility of added intelligence (or sane defaults)? i.e. the default for http_address should come from the hosts's address, http_vhost's should come from the host's name, etc. (I plan on implementing sane defaults in the future and don't want this to be blocked.) So, do you see a possibility to still add some sort of pre-processing before mapping the data to YAML?

Actually I generate a pillar structure which is directly dumpable to Icinga config. The generation of this pillar structure takes care of using sane default values.

[n-rodriguez]

Example from real life :

I generate this in my pillar and directly dump it :

my_server:
  monitoring:
    enabled: true
    config:
      raid_soft: true
      disks:
        - /dev/sda
        - /dev/sdb

      enable_basic_checks: true

      load:
        critical: 32.0,16.0,10.0
        warning: 16.0,10.0,5.0

      network:
        private:
          fqdn: web2.example.corp
          ipv4: 10.1.0.7

        public:
          fqdn: web2.example.net
          ipv4: xxx.xxx.xxx.xxx
          ipv6: 'xxxx:xxxx:xxxx:xxxx::'

      nfs:
        mounts:
          - "/data/apps/foo/shared/storage/app"
          - "/data/apps/foo/shared/storage/logs"
          - "/data/apps/bar/shared/docs_pdf"
          - "/data/apps/bar/shared/docs_pdf_sent"
          - "/data/apps/baz/shared/documents"
          - "/data/apps/baz/shared/ftp"
          - "/data/apps/baz/shared/tutos"
          - "/data/apps/biz/shared/images"

      partitions:
        - "/"
        - "/boot"
        - "/boot/efi"
        - "/data"

      php:
        versions:
          - "5.6"
          - "7.3"

      server:
        os: "Debian GNU/Linux 10 (buster)"
        type: physical

      services:
        - beamium
        - exim
        - fail2ban
        - nfs-client
        - nginx
        - noderig
        - openssh
        - php
        - telegraf

      ssh:
        port: 22

      web_apps:
        - name: "web_app1/ipv4"
          address: "xxx.xxx.xxx.xxx"
          vhost: "web_app1.example.net"
          uri: "/check.php"
          ssl: true
          expect_body_regex: "OK"

        - name: "web_app2/ipv4"
          address: "yyy.yyy.yyy.yyy"
          vhost: "web_app2.example.net"
          uri: "/check.php"
          ssl: true
          expect_body_regex: "OK"

        - name: "web_app2/ipv6"
          address: "zzzz:zzzz:zzzz:zzzz::zzzz"
          vhost: "web_app2.example.net"
          uri: "/check.php"
          ssl: true
          expect_body_regex: "OK"

then in Icinga :

object Host "web2.example.net" {
  # Base config
  import   "generic-host"
  address  = "10.1.0.7"
  address6 = "xxxx:xxxx:xxxx:xxxx::"

  vars.config["enable_basic_checks"] = true
  vars.config["raid_soft"] = true
  vars.config["disks"] = ["/dev/sda", "/dev/sdb"]
  vars.config["partitions"] = ["/", "/boot", "/boot/efi", "/data"]
  vars.config["services"] = ["beamium", "exim", "fail2ban", "nfs-client", "nginx", "noderig", "openssh", "php", "telegraf"]
  vars.config["web_apps"] = [
  {
    name = "web_app1/ipv4"
    vhost = "web_app1.example.net"
    address = "xxx.xxx.xxx.xxx"
    uri = "/check.php"
    ssl = true
    expect_body_regex = "OK"
  },
  {
    name = "web_app2/ipv4"
    vhost = "web_app2.example.net"
    address = "yyy.yyy.yyy.yyy"
    uri = "/check.php"
    ssl = true
    expect_body_regex = "OK"
  },
  {
    name = "web_app2/ipv6"
    vhost = "web_app2.example.net"
    address = "zzzz:zzzz:zzzz:zzzz::zzzz"
    uri = "/check.php"
    ssl = true
    expect_body_regex = "OK"
  }]
  vars.config["load"]["warning"] = "16.0,10.0,5.0"
  vars.config["load"]["critical"] = "32.0,16.0,10.0"
  vars.config["network"]["public"]["ipv4"] = "xxx.xxx.xxx.xxx"
  vars.config["network"]["public"]["ipv6"] = "xxxx:xxxx:xxxx:xxxx::"
  vars.config["network"]["public"]["fqdn"] = "web2.example.net"
  vars.config["network"]["private"]["ipv4"] = "10.1.0.7"
  vars.config["network"]["private"]["fqdn"] = "web2.example.corp"
  vars.config["server"]["type"] = "physical"
  vars.config["server"]["os"] = "Debian GNU/Linux 10 (buster)"
  vars.config["nfs"]["mounts"] = ["/data/apps/foo/shared/storage/app", "/data/apps/foo/shared/storage/logs", "/data/apps/bar/shared/docs_pdf", "/data/apps/bar/shared/docs_pdf_sent", "/data/apps/baz/shared/documents", "/data/apps/baz/shared/ftp", "/data/apps/baz/shared/tutos", "/data/apps/biz/shared/images"]
  vars.config["ssh"]["port"] = 22
  vars.config["php"]["versions"] = ["5.6", "7.3"]
}

apply Service for (web_app in host.vars.config.web_apps) {
  import "generic-service"
  name                  = "nginx/ssl-certificate/" + web_app.name
  display_name          = "Nginx | SSL Certificate | " + web_app.name
  check_command         = "http"
  vars.http_ssl         = web_app.ssl
  vars.http_vhost       = web_app.vhost
  vars.http_sni         = web_app.ssl
  vars.http_uri         = web_app.uri
  vars.http_address     = web_app.address

  # this is what we check: valid certificate?
  vars.http_certificate = 14

  vars.notification_period = "9to5"

  assign where host.address && web_app.ssl
}

apply Service for (web_app in host.vars.config.web_apps) {
  import "generic-service"
  name                  = "nginx/vhost/" + web_app.name
  display_name          = "Nginx | vhost | " + web_app.name
  check_command         = "http"
  vars.http_ssl         = web_app.ssl
  vars.http_vhost       = web_app.vhost
  vars.http_sni         = web_app.ssl
  vars.http_uri         = web_app.uri
  vars.http_address     = web_app.address

  # add additional vars
  if (web_app.auth_pair) {
    vars.http_auth_pair = web_app.auth_pair
  }

  # this is what we check: valid ip <=> vhost association
  vars.http_headerstring = "X-App-Name: " + web_app.vhost

  # test if content match (if available)
  if (web_app.expect_body_regex) {
    vars.http_expect_body_regex = web_app.expect_body_regex
  }

  assign where host.address
}

[n-rodriguez]

Does that force us to have mapping only without the possibility of added intelligence (or sane defaults)?

Consider this macro as a serializer so you can only pass a hash (like JSON or YAML) and dump it. (btw passing only an array is a non-sense in this context).

[alxwr]

@n-rodriguez Thanks for the fine explanation! LGTM! Can we use this as the default to serialize our Pillar data to Icinga2 config? (I don't want to maintains two separate ways of mapping.)

[n-rodriguez]

Can we use this as the default to serialize our Pillar data to Icinga2 config?

Sure!