Adding an ability of being more selective when setting up netflow module

[zentavr]

It would be great to be able to set up udp input for netflow collector more verbosely, like:

1. workers - by default is 2 regarding logstash-input-udp-3.1.2/lib/logstash/inputs/udp.rb
2. receive_buffer_bytes - by default is 212992 in Ubuntu 16.04
3. queue_size - by default is 2000 regarding logstash-input-udp-3.1.2/lib/logstash/inputs/udp.rb

[jordansissel]

It would prefer is users did not find the need to tune such things. Can we improve the defaults instead?

Also, receive buffer and queue size arfor basically the same concepts. Tuning both seems like a strange thing. Again I would prefer better defaults, not more options.

On Mon, Oct 30, 2017 at 3:20 PM Andrey Miroshnichenko < notifications@github.com> wrote:

It would be great to be able to set up udp input for netflow collector more verbosely, like:

1. workers - by default is 2 regarding logstash-input-udp-3.1.2/lib/logstash/inputs/udp.rb
2. receive_buffer_bytes - by default is 212992 in Ubuntu 16.04
3. queue_size - by default is 2000 regarding logstash-input-udp-3.1.2/lib/logstash/inputs/udp.rb

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/elastic/logstash/issues/8547, or mute the thread https://github.com/notifications/unsubscribe-auth/AAIC6tERSFV_JpiCOVqBGcxz1FUfJoxUks5sxku1gaJpZM4QL8B- .

[zentavr]

@jordansissel: ok, but even in the documentation authors say that:

For high-performance production environments the configuration below will decode up to 6000 flows/sec on an 8 CPU instance. If your total flowrate exceeds 6000 flows/sec, you should use multiple Logstash instances.

input {
udp {
port                 => 2055
receive_buffer_bytes => 16777216
codec                => netflow
workers              => 6
}

Make sure to increase the Linux kernel receive buffer limit:

# sysctl -w net.core.rmem_max=$((1024*1024*16))

I see no reasons for users to provide this feature to do. Instead what needs to be done is to fetch and set up all the things manually instead of using the module.

[untergeek]

Instead what needs to be done is to fetch and set up all the things manually instead of using the module.

That is actually the recommended path if you need more customization. The benefit of the using a module comes from the combination of ease of setup plus preconfigured dashboards. Modules aren't meant to be the be-all, end-all for power users.