search

cleesmith / unifiedbeat

Unifiedbeat reads records from Unified2 binary files generated by network intrusion detection software and indexes the records in Elasticsearch.
MIT License
30 stars 14 forks source link

Raw Fields don't appear to be being indexed #11

Closed dleblanc-vidcruiter closed 7 years ago

dleblanc-vidcruiter commented 7 years ago

I'm using the latest unified beat, ES 2.4. I've imported the unifiedbeat.template.json file and can see the template in ES.

My Logstash config is as follows: elasticsearch { hosts => ["localhost:9200"] sniffing => true manage_template => true index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}" document_type => "%{[@metadata][type]}" }

This puts the data in the an index that matches the naming pattern specified in unifiedbeat.template.json

The Raw fields however don't appear to be available in the index when looking in the kibana4 visualizations screen. Other indexes such as the default logstash index which also specifies raw fields (using dynamic mapping) do work and the raw fields are fully available for use in the visualization tab.

cleesmith commented 7 years ago

As this project is not used very often ... not very often by me anyways, as we switched to a near real-time approach instead of stuffing a gazillion records into ES and then hopelessly searching through that ... this isn't a priority for me. If you want to fix it and then issue a pull request, that's great.