search

robcowart / elastiflow

Network flow analytics (Netflow, sFlow and IPFIX) with the Elastic Stack
Other
2.48k stars 592 forks source link

Logstach Netflow IPFIX decode error #209

Closed leonel-santos closed 5 years ago

leonel-santos commented 5 years ago

Hi. A problem of this kind is happening to me too.

When I export IPFIX flows via TCP or UDP to logstach within the same network, no problem. I can see the IPFIX flows in Elastiflow Kibana Dashboards.

When I have the IPFIX exporter on a different network than the ELK server, I can only decode IPFIX flows exported via UDP to the logstach. TCP dosen't work.

The error:

[2018-11-10T13:32:47,679][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"elastiflow-3.3.0-2018.11.10", :_type=>"doc", :_routing=>nil}, #<LogStash::Event:0x161c6c73>], :response=>{"index"=>{"_index"=>"elastiflow-3.3.0-2018.11.10", "_type"=>"doc", "_id"=>"r1DT_WYBq-d9FyrfLfzt", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [node.ipaddr]", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"'_gateway' is not an IP string literal."}}}}}

It seems that there is some modification in the IP address field and it is replaced by a DNS name associated with it.

Can someone help. Thanks in advance.

robcowart commented 5 years ago

The Logstash TCP input used to resolve the name of all clients connecting to it. This caused a number of issues and may have been the problem here. There was a fix that added a feature to disable these lookups, which I have added to my development branch and will include in the next ElastiFlow release.