elastic / logstash

Logstash - transport and process your logs, events, or other data
https://www.elastic.co/products/logstash
Other
60 stars 3.5k forks source link

Deprecating the NetFlow module #11104

Open acchen97 opened 5 years ago

acchen97 commented 5 years ago

We're deprecating the Logstash NetFlow module in favor of the newer Filebeat NetFlow module. I'd suggest we put up a banner at the top of the documentation page.

Proposed language: "This module has been deprecated. We recommend using the Filebeat NetFlow module which is compliant with the Elastic Common Schema (ECS)."

karenzone commented 5 years ago

I've completed the documentation work. Leaving open so we can eval and address code side.

fredbcode commented 5 years ago

Hello, Did you make performance tests with filebeat ? Maybe I missed something but I'm unable to have the same results (with same server and same requests of course) https://github.com/elastic/beats/issues/14112

fredbcode commented 5 years ago

Tests here, the load is reduced

On top command the load average was, more or less, 14/20 with logstash and only 3/4 with filebeat

1500% cpu usage with logstash 300% filebeat -> 24 cores, 132 Go of memory

Big difference, same hardware, same os, same load, just docker images logstah/filebeat switched

jsvd commented 5 years ago

hi @fredbcode good to know the performance and experience on the beats side is better, which makes us more comfortable with this decision :)

fredbcode commented 5 years ago

Hello, Did you make performance tests with filebeat ? Maybe I missed something but I'm unable to have the same results (with same server and same requests of course) elastic/beats#14112

Result with 7.4.0

fredbcode commented 5 years ago

Tests here, the load is reduced

On top command the load average was, more or less, 14/20 with logstash and only 3/4 with filebeat

1500% cpu usage with logstash 300% filebeat -> 24 cores, 132 Go of memory

Big difference, same hardware, same os, same load, just docker images logstah/filebeat switched

Result with 7.4.1 SNAPSHOT