Set up file descriptors and mmap

[rmarting]

SUMMARY

Kafka uses a very large number of files and a large number of sockets to communicate with the clients. All of this requires a relatively high number of available file descriptors. Red Hat Enterprise Linux ships with only 1,024 file descriptors allowed per process. This is too low for Kafka. Values around 100k are very common in production environments.

The collection should allow to set up that value for RHEL environments, and also allow to update the vm.max_map_count variables.

References:

• How to set ulimit values
• What is the parameter "max_map_count" and does it affect the server performance?

ISSUE TYPE

• Feature Idea

[rpelisse]

@rmarting what would be a sane default value for vm.max_map_count?

[rmarting]

In production environments we are speaking to have values around 100k, however, there is not a magic number. So, we define the default value for RHEL environments (around 64k), or we can define 1048576 as default value. But this number requires a better analysis of the environment of the Kafka.

HTH

[rpelisse]

@rmarting I think with the ulimit and the kernel parameters tweaks, we have everything we need, right? We can close this issue, don't you think?

[rmarting]

Implemented by #69.