search

robcowart / elastiflow

Network flow analytics (Netflow, sFlow and IPFIX) with the Elastic Stack
Other
2.48k stars 592 forks source link

vFlow: Golang Collector #247

Closed pukkita closed 5 years ago

pukkita commented 5 years ago

Just came across this project...

looks like a promising integration into Elastiflow to get rid of logstash :smile:

High-performance, scalable and reliable IPFIX, sFlow and Netflow collector (written in pure Golang).

https://github.com/VerizonDigital/vflow/blob/master/README.md

robcowart commented 5 years ago

Thanks for sharing @pukkita. I have been aware of this project for quite some time. The challenge with vFlow is that its decoder doesn't support a lot of the PEN-specific IPFIX fields (as well as vendor-specific Netflow fields) that are so valuable in a lot of security use-cases. For performance management/billing use-cases it is probably sufficient.

The other challenge with the options currently available is that they do very little additional processing and enrichment of the raw flow records. It is this enrichment that is important to many of ElastiFlow's features. So even if one was to start with vFlow or one of the other options (including Filebeat's new netflow input), you would still have to send the output through Logstash to add all of the necessary enrichment. I want to deliver all of this functionality in a single, high-performance, yet configurable binary. I also want to support additional outputs, such as InfluxDB (which, BTW, I think a lot of user's will prefer over an Elasticsearch backend, as it provides better performance and uses SIGNIFICANTLY LESS storage).