Filter not working as expected

CodingMush commented 6 years ago

Hello everybody, I like the module very well, unfortunately I still have some problems. The ELK stack is set up and the data is displayed, unfortunately more than desired. When selecting a host, the entries of other hosts are also displayed.

Unfortunately, beat.hostname outputs only the CN and not the FQDN, the hosts are specified in Icinga (host.name) with FQDN. Therefore, in the Filebeat.yml configuration beat.name was specified with FQDN and the Event Types Filter in Icinga adjusted.

Expected Behavior

beat.name = host1.sub.top-level.de host.name = host1.sub.top-level.de beat.name = host.name (true)  show the matched host

Current Behavior

beat.name = host1.sub.top-level.de host.name = host1.sub.top-level.de Displays host1.sub.top-level, but also host2.sub.top-level.de

Steps to Reproduce (for bugs)

/etc/filebeat/filebeat.yml

================================ General =====================================

name: host1.sub.top-level.de
Icinga – Modules – Elasticsearch Event Types -> Index: logstash-* Event Types -> Filter: beat.name = {host.name} (Event Types -> Filter: beat.hostname = {host.name} | {host.display_name} was also tried) Event Types -> Fields: input_type, source, message, beat.name

Your Environment

Module version (System - About): businessprocess 2.1.0 cube 1.0.1 director master doc 2.5.3 elasticsearch 1.0.0 grafana 1.2.5 map 1.0.4 monitoring 2.5.3 nagvis 0.0.1 vsphere 1.1.0
Icinga Web 2 version and modules (System - About): Icinga Web 2 Version - 2.5.3
Icinga 2 version (icinga2 --version): icinga2 - The Icinga 2 network monitoring daemon (version: r2.8.4-1)
Elasticsearch version (curl http://localhost:9200): "version" : { "number" : "6.2.4", "build_hash" : "ccec39f", "build_date" : "2018-04-12T20:37:28.497551Z", "build_snapshot" : false, "lucene_version" : "7.2.1", "minimum_wire_compatibility_version" : "5.6.0", "minimum_index_compatibility_version" : "5.0.0"
Operating System and version: System information: Platform: Ubuntu Platform version: 16.04.4 LTS (Xenial Xerus) Kernel: Linux Kernel version: 4.4.0-21-generic Architecture: x86_64
Webserver, PHP versions:

CodingMush commented 6 years ago

Update: Works with IP Filter (received_from={host.address}) as expected. Maybe related with: Wrong results when {host.name} includes a minus (-) character #34

jediblair commented 5 years ago

Check my reply here - https://github.com/Icinga/icingaweb2-module-elasticsearch/issues/26#issuecomment-427200530

cbrassel commented 5 years ago

Hi, I'm just trying to integrate elastic logs with icinga2 and I have the same trouble with FQDN hosts.

So I have tried to use host.keyword=${host_name} but the query returned by the module is not working in elastic. Filter returned by module : {"_source":["@timestamp","host","message","severity","facility"], "query":{"match":{"host.keyword":"$w102.xx.yyy.zzz"}}, "from":0, "size":25, "sort":{"@timestamp":"desc"}} is returning nothing, but this one is working as expected : {"_source":["@timestamp","host","message","severity","facility"], "query":{"match":{"host.keyword":"w102.xx.yyy.zzz"}}, "from":0, "size":25, "sort":{"@timestamp":"desc"}}

The only change is the $ sign at the beginning of the hostname.

Can any body tell me where to avoid using the $ sign ?

Regard's Claude

jediblair commented 5 years ago

Don't think you need the $ at all.. just get rid of it ?

jediblair commented 5 years ago

in the filter field just put host.keyword={host_name} ? eg have a look at the screenshot on issue 34? https://github.com/Icinga/icingaweb2-module-elasticsearch/issues/34

cbrassel commented 5 years ago

thank you jediblair 👍

I'm new to icinga .. and I have entered host.keyword=${host_name} .... to much bash scripting I suppose ..

Sorry

lippserd commented 5 years ago

Duplicate of #26

Icinga / icingaweb2-module-elasticsearch