joriws
commented
7 years ago I've a bit similar but with Cisco Netflow NBAR.
[2017-10-24T12:50:33,277][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"netflow-2017.10.24", :_type=>"netflow", :_routing=>nil}, 2017-10-24T12:50:10.000Z 10.10.10.6 %{message}], :response=>{"index"=>{"_index"=>"netflow-2017.10.24", "_type"=>"netflow", "_id"=>"AV9Obzo8FEc_QuhC3MaT", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [netflow.application_id]", "caused_by"=>{"type"=>"number_format_exception", "reason"=>"For input string: \"13:0\""}}}}}
I tried to change from netflow.template.json the application id from "integer" to "keyword" but it did not help.
With vanilla netflow config (not elastiflow) everything worked fine (no filter section, just input + output config)..
Atroskelis
robcowart
I am having an issue with NetFlow v9 from a FortiGate device. I have narrowed it down to the below error.
This is the actual output from the FortiGate for the application id:
While the template calls for an integer. When using a standard vanilla input filter for NetFlow it successfully consumes into elastisearch. Do you have any ideas on what I need to correct to make this function correctly?
Any assistance you have would be greatly appreciated.
This is the error from the logstash logs
[2017-10-16T11:51:59,840][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"netflow-2017.10.16", :_type=>"netflow", :_routing=>nil}, 2017-10-16T15:45:25.000Z 10.10.240.50 %{message}], :response=>{"index"=>{"_index"=>"netflow-2017.10.16", "_type"=>"netflow", "_id"=>"AV8l4m6tGh0qZlWmKC7q", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [netflow.application_id]", "caused_by"=>{"type"=>"number_format_exception", "reason"=>"For input string: \"20:53068615909376\""}}}}}This is the output from the vanilla NetFlow config.
{ "_index": "t-netflow-2017.10.16", "_type": "netflow", "_id": "AV8mB_WpGh0qZlWmKrub", "_score": 1, "_source": { "@version": "1", "host": "10.10.240.50", "netflow": { "output_snmp": 12, "forwarding_status": { "reason": 0, "status": 1 }, "in_pkts": 1, "ipv4_dst_addr": "10.11.0.21", "first_switched": "2017-10-16T16:15:42.999Z", "flowset_id": 258, "l4_src_port": 57392, "version": 9, "application_id": "20:53068615909376", "flow_seq_num": 3311142, "ipv4_src_addr": "10.10.11.1", "in_bytes": 75, "protocol": 17, "flow_end_reason": 2, "last_switched": "2017-10-16T16:17:24.000Z", "input_snmp": 11, "out_pkts": 1, "out_bytes": 75, "l4_dst_port": 53 }, "@timestamp": "2017-10-16T16:17:23.000Z", "type": "netflow" }, "fields": { "netflow.first_switched": [ 1508170542999 ], "@timestamp": [ 1508170643000 ], "netflow.last_switched": [ 1508170644000 ] } }