[database::elasticsearch::restapi::plugin] - new mode: custom request

[joschi99]

It should be nice to have a mode which allows to send custom request and to validate the response. This mode allows to interrogate a Elasticsearch database and define thresholds/alarms for specific events/datas.

Example: Windows Event Logs Get alarm if there are more then x failed logins for a user This kind of control is not possible, interrogating Event Log from Active Directory when there are more then 1 AD Server

Inputs

• time range
• index
• custom query or saved search
• thresholds

[Sims24]

Hi @joschi99 ,

Could you share a curl request and the answer from Elastic API?

We are thinking about it but we want to gather various use cases to ensure we can offer something generic.

Thanks, Simon

[joschi99]

Hi @Sims24,

I will try to give you two sample

how many failed logins we have in the last hour? criteria's:

• Index: win_evt-*
• Channel: Security
• Event ID: 4625
• Time: last hour

Elastic query :

get win_evt-*/_search
{
  "aggs": {},
  "size": 0,
  "query": {
    "bool": {
      "must": [],
      "filter": [
        {
          "bool": {
            "filter": [
              {
                "bool": {
                  "should": [
                    {
                      "match": {
                        "Channel": "Security"
                      }
                    }
                  ],
                  "minimum_should_match": 1
                }
              },
              {
                "bool": {
                  "should": [
                    {
                      "match": {
                        "EventID": 4625
                      }
                    }
                  ],
                  "minimum_should_match": 1
                }
              }
            ]
          }
        },
        {
          "range": {
            "@timestamp": {
              "gte": "2021-01-19T08:12:39.446Z",
              "lte": "2021-01-19T09:12:39.446Z",
              "format": "strict_date_optional_time"
            }
          }
        }
      ],
      "should": [],
      "must_not": []
    }
  }
}

Elastic output (value: 212):

{
  "took" : 7,
  "timed_out" : false,
  "_shards" : {
    "total" : 15,
    "successful" : 15,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 212,
      "relation" : "eq"
    },
    "max_score" : null,
    "hits" : [ ]
  }
}

how many failed logins we have in the last hour group by Top 10 user? criteria's:

• Index: win_evt-*
• Channel: Security
• Event ID: 4625
• Time: last hour
• aggregation by TargetUserName, count desc
• size: 10 (Top 10)

Elastic query :

get win_evt-*/_search
{
  "aggs": {
    "2": {
      "terms": {
        "field": "TargetUserName",
        "order": {
          "_count": "desc"
        },
        "size": 10
      }
    }
  },
  "size": 0,
  "stored_fields": [
    "*"
  ],
  "query": {
    "bool": {
      "must": [],
      "filter": [
        {
          "bool": {
            "filter": [
              {
                "bool": {
                  "should": [
                    {
                      "match": {
                        "Channel": "Security"
                      }
                    }
                  ],
                  "minimum_should_match": 1
                }
              },
              {
                "bool": {
                  "should": [
                    {
                      "match": {
                        "EventID": 4625
                      }
                    }
                  ],
                  "minimum_should_match": 1
                }
              }
            ]
          }
        },
        {
          "range": {
            "@timestamp": {
              "gte": "2021-01-19T08:21:55.050Z",
              "lte": "2021-01-19T09:21:55.050Z",
              "format": "strict_date_optional_time"
            }
          }
        }
      ],
      "should": [],
      "must_not": []
    }
  }
}

Elastic output :

{
  "took" : 16,
  "timed_out" : false,
  "_shards" : {
    "total" : 15,
    "successful" : 15,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 214,
      "relation" : "eq"
    },
    "max_score" : null,
    "hits" : [ ]
  },
  "aggregations" : {
    "2" : {
      "doc_count_error_upper_bound" : 0,
      "sum_other_doc_count" : 0,
      "buckets" : [
        {
          "key" : "user1",
          "doc_count" : 176
        },
        {
          "key" : "user2",
          "doc_count" : 14
        },
        {
          "key" : "user3",
          "doc_count" : 6
        },
        {
          "key" : "user4",
          "doc_count" : 4
        },
        {
          "key" : "user5",
          "doc_count" : 1
        }
      ]
    }
  }
}

If you like we can do a websession, I can produce a lot of examples

[Sims24]

Thanks, we will definitely take a look at it. We'll let you know if we need help from you.

Simon

[mullergg]

We need this features too. I can also help if you need.

[Sims24]

Hey guys @joschi99 & @mullergg,

Finally ... We chose to develop a more flexible collection mode to cover monitoring of various APIs and data formats.

You can now use the apps::protocols::http::plugin and a collection mode to do that.

perl centreon_plugins.pl --plugin=apps::protocols::http::plugin --mode=collection --config='/etc/centreon-engine/http_collection/elastic-collection-config.json'
OK: number of events: 5 | 'filebeat.events.count'=5;;;0;

Here is a sample config file:

{
    "mapping": {},
    "http": {
        "cache": {
            "enable": false,
            "reload": 5
        },
        "requests": [
            {
                "name": "secevents",
                **"hostname": "10.25.10.91",**
                "proto": "http",
                "port": "9200",
                **"endpoint": "/filebeat-*/_search",**
                **"headers": ["Accept: application/json", "Content-Type:application/json"],**
                "timeout": 5,
                "backend": "curl",
                **"rtype": "json",**
                "payload": {
                    "type": "file",
                    **"value": "/etc/centreon-engine/http_collection/elastic-ssh-error-aPACHE.json"**
                },
                "parse": [
                    {
                        "name": "hits",
                        **"path": "$.hits.total.value",**
                        "entries": [
                            { "id": "total" }
                        ]
                    }
                ]
            }
        ]
    },
    "selection": [
        {
            "name": "secevents",
            "perfdatas": [
                { "nlabel": "filebeat.events.count", "value": "%(http.tables.seceventsHits.[0].total)", "min": 0 }
            ],
            "formatting": {
                "printf_msg":"number of events: %s",
                "printf_var":[
                    "%(http.tables.seceventsHits.[0].total)"
                ],
                "display_ok": true
            }
        }
    ]
}

The /etc/centreon-engine/http_collection/elastic-ssh-error-aPACHE.json file can contain any elastic query like the ones @joschi99 shared. E.g

{
  "aggs": {},
  "size": 0,
  "query": {
    "bool": {
      "must": [],
      "filter": [
        {
          "bool": {
            "filter": [
              {
                "bool": {
                  "should": [
                    {
                      "match": {
                        "Channel": "Security"
                      }
                    }
                  ],
                  "minimum_should_match": 1
                }
              },
              {
                "bool": {
                  "should": [
                    {
                      "match": {
                        "EventID": 4625
                      }
                    }
                  ],
                  "minimum_should_match": 1
                }
              }
            ]
          }
        },
        {
          "range": {
            "@timestamp": {
              "gte": "2021-01-19T08:12:39.446Z",
              "lte": "2021-01-19T09:12:39.446Z",
              "format": "strict_date_optional_time"
            }
          }
        }
      ],
      "should": [],
      "must_not": []
    }
  }
}

I bolded some very important part of the config file, I'm writing a blog post about it but it will take me some additional time.

Feel free to share any feedbacks!

[Sims24]

@joschi99 extra question, can we consider this as a solution for this issue too?

[joschi99]

I think this could be a valid solution, but I will need to check the solution and update you asap

[Sims24]

Great, let me know.

[mullergg]

@Sims24 and @joschi99 I test it today and to me this solution isn't enough.

I have multiple elastic cluster on different environment (like developpment, test, integration etc....) with this solution I need to have a config file for each and manage it in a different way. But I have the hosts informations in centreon.

Or maybe I've missing something or misunderstand something. This solution could be the trick if we can replace the informations in the config file with the Centreon Macro.

[Sims24]

@mullergg thanks for the feedback, we will implement a constant option logic to help you with this. Exactly like we did for SNMP for example. So we will be OK I think.

[joschi99]

Hi @Sims24, did you have any news for this request?

[garnier-quentin]

Now with the mode, you can use constants:

       "hostname": "%(constants.hostname)",

And you can set in command line with the option: --constant='hostname=10.0.0.1'

[haytxy]

hello, @Sims24 did you manage to provide a blog post regarding this issue.?