Closed luis-nagaki closed 5 years ago
What results do you get if use ElastiFlow unmodified?
Please check firewall-cmd allow incoming port for netflow
I will try unmodified output and report back @robcowart , firewall is off not even installed @xUmaRix
@robcowart used this which works w/ other output integrations. and still no index i also tried adding a tag to netflow and did an if condition on output for it no index either.
output {
stdout { codec => rubydebug }
elasticsearch {
hosts => ["localhost:9200"]
index => "elastiflow-3.3.0-%{+YYYY.MM.dd}"
template_name => "elastiflow-3.3.0"
template_overwrite => true
manage_template => true
}
}
also used
id => "output_elasticsearch_single"
Not change
[2018-11-15T11:11:44,735][WARN ][logstash.codecs.netflow ] Invalid netflow packet received (value '0' not as expected for obj.records[2].flowset_length)
[2018-11-15T11:11:44,854][WARN ][logstash.codecs.netflow ] Reduced-size encoding for uint32 is larger than uint32 {:field=>[:uint32, :conn_id], :length=>8}
[2018-11-15T11:12:27,151][WARN ][logstash.codecs.netflow ] Can't (yet) decode flowset id 512 from observation domain id 524288, because no template to decode it with has been received. This message will usually go away after 1 minute.
Basically the Logstash Netflow Codec has trouble with the records it is being sent. You may need to open issue here... https://github.com/logstash-plugins/logstash-codec-netflow
A PCAP of the flow records will likely be necessary to troubleshoot.
Not sure its a codec issue, as I got it working with this before http://www.routereflector.com/2017/07/elk-as-a-free-netflow-ipfix-collector-and-visualizer/
But then I saw ElastiFlow and fell in love, just trying to get it working. I removed all previous netflow configs
Any thoughts? Tried out sflow, getting this and no index yet [2018-11-16T13:13:41,140][WARN ][logstash.codecs.sflow ] Unknown record entreprise 0, format 7
Worked FYI if it was a fresh install
Using ELK 6.4.x I have elastiflow installed and inputs up, but elasticsearch is not receiving anything.
I have the follow output
I also tried just if [type] == "netflow" or [type] == "ipfix" etc and nothing. no error msgs so far but not data. If it matters, its data from a Juniper device
All on single node ELK
I am going to try and do [event][type] if statement. I did a stdout on that section for IF netflow and see nothing out. so I dont think its picking up [netflow]