Handling of hyphen (dash, minus) character in hostnames

mattpoel commented 6 years ago

First of all, thanks a lot for this great IcingaWeb2 module!

The hosts I'm monitoring do contain the - character one or multiple times in their hostnames like the following examples:

WDW-TEST01
WDW-TEST02
CHR-STG-D01

The - character in kibana / elasticsearch splits the search term and would provide you all results for WDW and CHR. Therefore, I have to put my search term in quotes when searching in kibana, e.g.:

syslog_hostname: "WDW-TEST01"

I tried to somehow squeeze in the double quotes into the IcingaWeb2 elasticsearch configuration, but was not successful on receiving the correct result. It is still returning "all" log entries:

syslog_hostname="{host.name}"

Does the elasticsearch filter configuration currently somehow support search for terms in double quotes?

Expected Behavior

Filter configuration should provide a possibility to put search term in double quotes to provide the possibility to search for hostnames with a - in it.

Current Behavior

Haven't found a way how to specify the filter configuration to respect - in hostname term for elasticsearch.

Steps to Reproduce (for bugs)

Try to search for a string / host with a - in it.

Context

Currently it is not possible to display a correct result as my syslog elasticsearch entries do not contain the hosts IP (syslog proxy / load balancer in the front).

Your Environment

Module version (System - About): 1.0.0
Icinga Web 2 version and modules (System - About): 2.5.1
Icinga 2 version (icinga2 --version): r2.8.1-1
Operating System and version: CentOS Linux release 7.3.1611
Webserver, PHP versions: Apache 2.4.6-45 / PHP 7.1.8-1
Elasticsearch / Kibana: 5.5.2-1

lippserd commented 6 years ago

Hi,

Thanks for the report. Is it possible to escape the hyphen with a backslash maybe? Anyway, I'll have a look at this one asap.

Best regards, Eric

mattpoel commented 6 years ago

Hi Eric,

thanks a lot!

As far as kibana is executing the query, the correct result will only be returned if the search term is put in double-quotes:

syslog_hostname: "WDW-TEST01"

The search will still be case-insensitive.

Escaping the - with just a backslash doesn't return the proper result for me. Tried the following with a hard-coded hostname in the Event Types configuration:

syslog_hostname=WDW\-TEST01

Best regards, Matt

jediblair commented 5 years ago

Try using the .keyword type (this might depend on your elasticsearch version, it's working for 6.x here) - so syslog_hostname.keyword={host.name} The other option is configuring your index mapping so the hostname field is "keyword" in elasticsearch - https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping.html Cheers, Blair

swood1980 commented 5 years ago

I have found that when using the analyze_wildcard=true you also need to use default_operator=AND.

In the file library/Elasticsearch/FilterRenderer.php

I have made the following change adding in default_operator=AND as below.

if ($sign === '=' || $sign === '!=') { return array( 'query_string' => array( 'default_field' => $column, 'query' => $value, 'analyze_wildcard' => true, 'default_operator' => "AND" ) ); }

This appears to fix the issue with hyphens in hostnames as the wildcard is split in to tokens. Hostname foo-bar-one is split into search for hostnames with foo AND bar AND one within the field. The default being OR will search fields containing any of those fields and hence return multiple wrong results.

You then need to force the function to count the as an array by splitting the host search into 2 section. I use hostname and then * as the second part of the array.

filter = "host={host.name} *&type=rsyslog&severity!=notice&severity!=info&severity!=debug"

Icinga / icingaweb2-module-elasticsearch