logstash-plugins / logstash-codec-netflow

Apache License 2.0
78 stars 87 forks source link

Fortigate Firewall Appliction_ID #150

Closed FracKenA closed 6 years ago

FracKenA commented 6 years ago

This is related to ticket #116

I am getting the following errors.

`[2018-07-20T10:31:02,216][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"netflow-2018.07.20", :_type=>"doc", :_routing=>nil}, #], :response=>{"index"=>{"_index"=>"netflow-2018.07.20", "_type"=>"doc", "_id"=>"4W5QuGQBJdPGiGEr0_VV", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [netflow.application_id]", "caused_by"=>{"type"=>"number_format_exception", "reason"=>"multiple points"}}}}}

[2018-07-20T10:31:02,216][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"netflow-2018.07.20", :_type=>"doc", :_routing=>nil}, #], :response=>{"index"=>{"_index"=>"netflow-2018.07.20", "_type"=>"doc", "_id"=>"4m5QuGQBJdPGiGEr0_VV", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [netflow.application_id]", "caused_by"=>{"type"=>"number_format_exception", "reason"=>"multiple points"}}}}}

[2018-07-20T10:31:02,217][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"netflow-2018.07.20", :_type=>"doc", :_routing=>nil}, #], :response=>{"index"=>{"_index"=>"netflow-2018.07.20", "_type"=>"doc", "_id"=>"425QuGQBJdPGiGEr0_VV", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [netflow.application_id]", "caused_by"=>{"type"=>"number_format_exception", "reason"=>"multiple points"}}}}}`

Using 4.0.2 of the codec

logstash-codec-netflow (4.0.2)

and this is my logstash.yml

`modules:

Any feedback would be appreciated.

jorritfolmer commented 6 years ago

This isn't an issue with the Netflow codec, the error comes from Elasticsearch.

It looks very similar to this comment, which means you'll need to upgrade Elastiflow to support the change in application_id: https://github.com/robcowart/elastiflow/issues/100#issuecomment-402224134