Set fielddata=true on [beat.hostname]

[nicolasbadia]

First, thanks for sharing this!

I followed the instructions, but I get this error :

Fielddata is disabled on text fields by default. Set fielddata=true on [beat.hostname] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead.

Also, the dashboard looks like this, with shards failing :

Any idea of what I missed?

Note that the only thing I changed is that I defined this as output:

else if [postfix] {
    elasticsearch {
      hosts => localhost
      manage_template => false
      index => "%{[@metadata][beat]}-postfix-%{[@metadata][version]}-%{+yyyy.MM}-%{[host][name]}"
    }
  }

Also no that I'm using nginx and mysql plugins.

[ActionScripted]

Glad someone's using this!

Taking at a look at the error I wonder if maybe file beat has changed how they're handling host names. From the config(s) we're using beat.hostname but it sounds like maybe that field has changed or is different than the code in this repo is expecting.

To troubleshoot, I'd suggest deleting all resources created by this repo (anything with a name that includes "[Filebeat Postfix]"), check your Filebeat field names and configuration and then edit the files you're using from the repo to use the newer Filebeat field name(s) if they're different.

I might also check any customizations made to logstash or filebeat. Going through the Elastic forums it does look like other folks have had a similar issue in the past but the resolution doesn't seem clear: https://discuss.elastic.co/t/kibana-dashboard-error-set-fielddata-true-on-fieldname/82812/12

The changes you've made should be totally fine; I don't think this error is related to those.

[ActionScripted]

Maybe also helpful:

• https://discuss.elastic.co/t/kibana-dashboard-error-set-fielddata-true-on-fieldname/76775/3
• https://github.com/elastic/kibana/issues/9898
• https://github.com/elastic/beats/issues/6399

One common suggestion is to make sure you've gone through the full Filebeat setup including loading templates: https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-template.html

Manually from the Filebeat :

# I think this is the path, not sure
cd /usr/share/filebeat

# Default template install
./filebeat setup --template -E output.logstash.enabled=false -E 'output.elasticsearch.hosts=["localhost:9200"]'

# Default dashboard setup
filebeat setup --dashboards

[nicolasbadia]

Thanks a lot for the quick answer. I did a few changes and it is now working properly.

I think the main issue were that my logstash config files were not in a proper order.

Also, looks like my index "%{[@metadata][beat]}-postfix-%{[@metadata][version]}-%{+yyyy.MM}-%{[host][name]}" were problematic. "%{[@metadata][beat]}-%{[@metadata][version]}-%{+yyyy.MM}-%{[host][name]}-postfix" fixed an issue.

The only not working visualization is "Message total" which say: Could not locate that index-pattern-field (id: postfix_queueid)

Any idea for this last issue ?

[nicolasbadia]

Found it, just had to refresh the indexes... Awesome dashboard by the way 👍

[ActionScripted]

@nicolasbadia thanks for following up with the fix! Glad it's working.

[li-adrienloiseau]

Hello @nicolasbadia / @ActionScripted can you share you're configuration ? I'm in the same issue...

1. Could not locate that index-pattern-field (id: postfix_queueid)
2. beat.hostname is empty

[phuongkhpt]

Pls help me

[ActionScripted]

@phuongkhpt have you tried making the changes suggested in the red areas for fieldata=true? What version(s) of the Elastic components are you using?