Doc on splitting facts for custom vars if array/json/hash

[TheFlyingCorpse]

Any suggestions on how to split facts in arrays to get an output where

Input Example:

[{"Name":"ADSync","StartType":2,"Status":4,"CanShutdown":true,"CanStop":true,"CanPauseAndContinue":false},
{"Name":"AeLookupSvc","StartType":3,"Status":1,"CanShutdown":false,"CanStop":false,"CanPauseAndContinue":false},
{"Name":"ALG","StartType":3,"Status":1,"CanShutdown":false,"CanStop":false,"CanPauseAndContinue":false},
{"Name":"AppIDSvc","StartType":3,"Status":1,"CanShutdown":false,"CanStop":false,"CanPauseAndContinue":false},
{"Name":"Appinfo","StartType":3,"Status":1,"CanShutdown":false,"CanStop":false,"CanPauseAndContinue":false}]

Desired output is a key based on the name with the key/value pairs as a subset, example:

vars.windows_service["ADSync"] = {
    name = "ADSync"
    starttype = 2
    status = 4
    canshutdown = true
    canstop = true
    canpauseandcontinue = false
}

The reasoning behind this is to let Puppet discover services that icinga can monitor. I have a script that emits the services on Windows to Puppet, classifying them (Standard windows service, role (AD/IIS) or 3rd party, letting us quickly monitor anything we dont know about), the same is easily done for IIS' websites and AppPools amongst other examples. With Director we can then almost automate an approval process for everything in Icinga2, letting everything come in "unclassified" and be monitored from day 0, letting the NOC take care of classifying if its production, dev or test by a process going on after its already being monitored, rather than trying to figure out what is on the box. :+1: :+1: :+1:

[Thomas-Gelf]

Sorry for the late response, I didn't have notifications enabled for this repo. If I correctly understood what you are asking for, it should be enough to add Custom variable -> windows_service -> from facts.your_special_fact - and you are all done. Or do I miss something?

[TheFlyingCorpse]

No, you got it :+1:

Would it be possible to use sync rules on the keys, say if I had a database over SLA / extra options per object? I have a concept brewing where I can allow the NOC to take over all classificiations/adjusting for me, where they just assign a classification (production, test, acceptance, development) to a service and it will get added to notifications and such from there. The difficulty is storing it between runs when using "replace" parameter to purge parameters, or even on merge due to the services being made by auto apply. Since this is on the "service" and not on the object read from puppet/the source it is a bit difficult perhaps.

An option here is to just write a custom script and query puppetdb directly, do the magic to find the services and use that as the source for Icinga Director to import services with the extra attributes/columns from.

[Thomas-Gelf]

I have a working prototype for something I currently use to call "host-driven-services". This consists of:

• the possibility to use Puppet or a similar tool to ship a hash/dictionary with service definitions directly on the host. All you are allowed to define in that hash are service names, used templates and custom variables
• an apply rule automagically rolled out by Director fitting those objects in case at least one of those exists
• a GUI extension showing those services like "normal" services directly on the host
• the possibility to override all provided custom variables for those services in case corresponding fields have been defined for the involved check commands or service templates This "allow-to-override" mechanism is strongly opinionated and the related logic makes part of the mentioned apply rule.

Would something like this fit your use-case?

[TheFlyingCorpse]

Yes. It sounds like the use case fits, seeing the NOC is able to override the parameters.

How would this work with sync when objects are removed? Thats a major hurdle now when using several sources, if its no longer in Puppet, it can still be "active" from racktables or ldap, causing partial objects before its cleaned up there as well.

[Thomas-Gelf]

An object does not "know" where it comes from and it should not. "Purge" deletes all objects that vanished at a specific source since the last Sync run of the very same Sync rule. Objects that have been touched during this sync will not be deleted. This works perfectly as long as different sources ship different objects. Last week I used 15 different sources and sync rules for hosts in a single, no problem so far.

When the same objects get properties from different sources, you should combine them in a single Sync rule and everything should still work as expected. For this feature when you decided to "replace" existing objects it would mean that the "host-driven-services" variable would go away, as Puppet would no longer ship them. As of how it is implemented right now, your property overrides would be preserved. They wouldn't hurt, but they would still be there and still match if you later decide to re-activate those objects. But to me this should be considered a bug, when services go away all related properties should also.

Please let me know your thoughts, or try to better describe how it should behave to perfectly fit your personal preferences. As I didn't yet publish it there is still a chance to get heard ;-)

[mjbrooks]

+1

Count me in on having an interest in this. There needs to be an option to let puppet drive defaults/initial services and other discoverables while allowing for changes to be made from the Director UI for things like thresholds etc. without having puppet stomp on them afterwards.

Is there a queue in the dev tracker that we should be on, or is that just for the main stuff and we just use the github issue tracker for things like this?

[Thomas-Gelf]

Hi @TheFlyingCorpse, @mjbrooks!,

Host-driven services have been dropped, as IMO the prototype was too abstract and confusing - and technically it didn't differ from single services synchronized to single hosts. What remained our consent in this issue was IMO the desire to let Puppet define initial properties while still allowing the user to override them in the GUI. And of course without loosing those changes on the next sync, but preserving the possibility to sync new default values.

Now, Service Variable Overrides have improved a lot recently, and Director 1.3.0 (been tagged today) got a bunch of improvements for that kind of workflows. There is a "hidden" feature that might be of interest for you. When synchronizing single services to single hosts, please try to set the property use_var_overrides to true (or 'y').

Then go to those hosts, choose 'services' - and let me know whether what you see is what you expected ;-)

Cheers, Thomas

[mjbrooks]

Oooooo, that sounds pretty good @Thomas-Gelf

As soon as I can get over to Director 1.3.0 I'll have to check it out.

Best, Matthew

[TheFlyingCorpse]

Sadly if you use "replace" on hosts, you nuke the director overrides on every sync, unless you set it to merge, now the import source if it has a field that had data that is now null, it wont be overwritten as null. 👎

[Thomas-Gelf]

@TheFlyingCorpse: could you please open a related issue in the Director issue tracker? This should be easy to fix

[Thomas-Gelf]

Closing this issue, as we are ways off-topic. Apart from "replace" which should be solved I guess we covered all other mentioned tasks. In case I missed something please open a new issue!

Cheers, Thomas