search

sous-chefs / kafka

Development repository for the kafka cookbook
https://supermarket.chef.io/cookbooks/kafka
Apache License 2.0
91 stars 105 forks source link

Set ulimit number by default #79

Closed spuder closed 9 years ago

spuder commented 9 years ago

I think the default value for the ulimit should be set by default.

After days of crashes, and various conversations in the IRC chanel, I discovered that kafka was exceeding the maximum number of open files.

Someone pointed out that this cookbook can set the ulimit right inside of the upstart file.

https://github.com/bmhatfield/chef-ulimit/issues/38

Even users with simple configurations will run out of open files. Should the limit be set by default?

mthssdrbrg commented 9 years ago

My opinion is that the limit shouldn't be set by default. Personally I prefer when cookbooks makes as little decisions as possible on their own, and where things are as configurable as possible (to an extent when it just becomes cumbersome to use).

Furthermore, I believe that it's pretty difficult to set a reasonable default limit as it depends on what kind of infrastructure we're talking about (i.e. shared resources using something like Mesos or a system where Kafka is the only thing running). Why would 64k be better than 32k for example?

spuder commented 9 years ago

Agreed that the cookbook shouldn't set a system wide limit. But since this limit is only for the kafka user, the influence should be minimal.

Perhaps there is some middle ground? Not setting a default leads to a bad out of the box user experience because kafka will work for a short period and then crash. It took me a long time to figure out why the cookbook wasn't just working out of the box. Maybe a warning in the README would be appropriate?

mthssdrbrg commented 9 years ago

A heads up section in the README would definitely be appropriate.

lock[bot] commented 4 years ago

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.