dnsmichi
commented
6 years ago Hm, those services know nothing from each other.
Some ideas from the #osmc hackathon (@bobapple & @urso)
- https://github.com/Icinga/icingabeat/blob/master/icingabeat.full.yml allows for the instance name and custom field addition. Maybe this could be used to dump the data and filter later on.
- Elasticsearch feature as an alternative does not support all the events Icingabeat does, neither is has Ha support
- Calculating a fingerprint for a message is hard - the timestamps might differ between the two cluster nodes. The output, state, etc. are not unique, they may occur like 5 seconds later.
I'd say for an immediate trial, go for custom fields or the name and experiment with that on de-duplication in Logstash.
bobapple
manishkmbl0
Expected Behavior
Two icingabeats sending data to two logstash listeners, only one input gets inserted to elasticsearch.
Current Behavior
Without configuration the data will be written twice, with the elasticsearch fingerprint configuration (https://www.elastic.co/guide/en/logstash/current/plugins-filters-fingerprint.html) it should only write it once (fingerprint already exists if icingabeat #1 was faster)
The problem with the fingerprint is that icingabeat has a lot of different field so I can't just set a fingerprint for "message".
Possible Solution
Write if else for every icingabeat field if not empty and then set a fingerprint. Not sure if this is the best way, there might be an easier solution from icinga direct?
Steps to Reproduce (for bugs)
Context
At the moment e run two icingabeats on two icingas but one is not running. If icingabeat on #1 is not working we will start #2 , a clean icingabeat cluster would be nice.