Roadmap Contribution Request - Feature handling, CA, Cert generation, roles structure etc.

Hello @flokoe

as you are the new maintainer we are happy to see some change and results on the project. We @lbetz (maintainer of the icinga puppet module) and I want to contribute and help out on a few topics. We have some time and motivation to spare.

We had a meeting the other day and discussed a few things, which we want to change or add. Please feel free to comment on them, add your notes and share your thoughts.

Extract the repo management into an extra role.

In future we think that the role to install and configure Icinga 2 won't be the last one. We need to consider, that after Icinga 2, we need the following roles to cover all Icinga 2 components.

icingaweb2
icingadb
icinga-redis
maybe more?

So to keep the repo management consistent over all icinga* roles we think its best to extract the part and publish it into an separate role. Then it would be possible to use it as a dependency for the roles.

Certificate Generation for Clients/Agents

As Icinga 2 provides a few ways to generate certificates for clients, we had the discussion which way would be the best to choose. Ansible provides a key feature which enables us to easily generate certificates on the CA system.

So we suggest to generate and sign the certificates on the CA system by using delegate_to and provide them to the client.

Manage "role" in Icinga Environment

On the topic how we define a client or master in Ansible we discussed it's best to let the user choose which features to use. As many our environments are so uniq and therefore we can't define how a master, satellite or agent should look like.

So we decided we should stick with using features to design the Icinga instance.

In addition to this, the CA isn't a feature in Icinga terms, but something we need to take care like a feature. There should be a variable to define if the system contains the Icinga 2 CA. i2_ca = true

Icinga 2 config, objects, attributes and parser

Config Path

Icinga 2 provides the possibility to distribute monitoring over multiple locations, but also as a single instance if the monitoring environment isn't that big or there are no remote/DMZ location.

So the user need to choose if he uses a single instance or want to use Icinga 2 distributed. In this case we thought about how we want to provide the monitoring configuration over Ansible.

First of all, the user need a variable to choose whether he wants to define the configuration in zones.d, custom.d directory or none if the API maybe is used.

  i2_objects_dir: custom.d/
  or
  i2_objects_dir: zones.d/

Monitoring Code/Config

With this information we know now, where to put monitoring configuration files per default. Then we need to give the user the ability to use its own structure. Of course we can generate the configuration in one big blob and dump it into the configuration directory but in case anyone want to debug something manually it won't be easy and frustrating.

So we came up with the following example:

  i2_objects:
    - path: "{{ i2_objects_dir }}/satellite/hosts.conf"
      objects:
        - Host:                       #object type
            client01.localdomain:     #object name
              display_name: client01  #object attributes
              [...]
        - Host:
            client02.localdomain:
              display_name: client02
              [...]
        - Service:
            disk:
              check_command: disk
              host_name: client01.localdomain
              [...]
              apply:

For every object in this array we will prepare the path and generate the file with the defined objects. The user can choose which objects he/she needs in the specific file, this enables complete control over the structure and configuration.

Parser

I already started to create something like a parser https://github.com/Icinga/ansible-icinga2/blob/master/templates/object_attributes.j2, because the quoting is in many cases very important. For the Puppet module there's a custom parser provided, for Ansible it should be similar. The parser should be working like a filter plugin and will return a string.

Basically we would try to replicate the parser from the puppet module https://github.com/Icinga/puppet-icinga2/blob/master/lib/puppet_x/icinga2/utils.rb and rewrite it for python.

Feel free to add your thoughts and comments. We would be happy to help out!

mkay

Hi! :)

Thank you for your comment.

as you are the new maintainer we are happy to see some change and results on the project.

I totally understand that. Unfortunately I have had not enough time to dig into it.

We @lbetz (maintainer of the icinga puppet module) and I want to contribute and help out on a few topics. We have some time and motivation to spare.

Sure, I am happy to accept any help!

In future we think that the role to install and configure Icinga 2 won't be the last one.

I totally agree. To stuff all the different projects into one role would be very bad and an antipattern to single purpose roles.

So to keep the repo management consistent over all icinga* roles we think its best to extract the part and publish it into an separate role.

Actually, this is a very good idea. I did not consider that one may want to install for example only icingadb on a separate host.

All other topics you mentioned are perfectly valid and interesting, and I kind of thought about them too. And I think it is important that we clarify how we want the icinga2 roles to work.

Here are some questions/topics I came up with (in random order and with no guarantee of completeness):

How much should we separate the roles (repo, agent/satellite, control plane vs all in one)?
What is the expected scope of all the roles? What should the covered?
Freedom/customization vs. opinionated settings/functionality/defaults
Should the role manage object configuration (host and services, only hosts or only the install/basic conf of the daemon?)
If we manage objects should we use the api, director or files?
What top down strategy?
apply vs settings per object (what strategy)?
should Ansible manage object confs (could cause very complex conf/var files)
conf location? All on master, synced to all nodes?
Support of HA setup?
What about the roles of T-Systems and Bodo Schulz?
How to tell Ansible which host should assume which role (one big dictionary or per host var file)?
automated signing of new nodes

Some things you mentioned partly answer some of those questions, but there are still many answers missing.

Before we start working on new features, we should try to solve most of the questions. If you are interested, I would be happy to have a chat with both of you to discuss the best way of action.

But we can continue via text as well if this is more convenient for you.

Best regards

Hi flokoe

here a few thoughts on your questions. I've worked a bit on the role lately.

How much should we separate the roles (repo, agent/satellite, control plane vs all in one)?

I think it's best to give the user the opportunity to do everything possible in icinga 2 with the icinga2 role.

As I said, one role for the repo to reduce the overheat. A role for icinga 2, which can configure every object(type) and every configfile. A role for icingaweb2 and maybe but thats future talk, a role for icingadb.

What is the expected scope of all the roles? What should the covered?

For Icinga 2 everything, included monitoring objects. Everything we configure in Icinga 2 are objects, if the parsing and generating of those things are done, everything can be configured.

automated signing of new nodes

Generation of certificates is mandatory. As I described

Should the role manage object configuration (host and services, only hosts or only the install/basic conf of the daemon?)

Yes

If we manage objects should we use the api, director or files?

Director has its own automation, we will generate "normal" files with objects. Which are in human readable form to easily debug the result.

apply vs settings per object (what strategy)?

Everything is possible, it's just the configuration which will be translated from yaml to icinga2 objects.

conf location? All on master, synced to all nodes?

As it is required by Icinga 2, the configuration will be stored on the master node and then synced via the zones replication. Configuration will be stored in /etc/icinga2/zones.d

Support of HA setup?

It's only a setting in the configuration object. For example: enable_ha on the ido feature. As we are able to generate every object with the parser, I say yes.

How to tell Ansible which host should assume which role (one big dictionary or per host var file)?

There is only configuration which defines the state of the host, so every host will be getting the role and the configuration to enable or disable the local features and configuration.

In addition I mentioned that we want to have a dictionary for monitoring objects, see the example in the issue. This dictionary can be defined on the master, or somebody can also merge it from multiple locations if he's capable.

should Ansible manage object confs (could cause very complex conf/var files)

The parser is nearly ready and yes it's a bit complex but doesn't change that much. And they will be human readable for debugging purpose.

Just a few thoughts, feel free to give your imput. Best Regards

Obviously I am not able to work on this. I am terribly sorry. I am resigning and wish you much success!

No problem, we started a new project under https://github.com/Icinga/ansible-collection-icinga

Icinga / ansible-icinga2