Closed ibanner56 closed 9 years ago
Note that the line numbers done correspond to the master branch, since we added a few imports for other tasks. The specific lines according to the stack trace for our version are:
if (!addr.isEmpty) {
offset = twitterToScalaFuture(client.offset(topicName, tu._2, OffsetRequest.LatestTime)).map(_.offsets.head).recover {
case e => Logger.warn("Could not connect to partition leader " + addr + ". Error message: " + e.getMessage); 0L
}
}
Apparently this is due to the ulimit being too low. 1024 is too small. We're going to boost out ulimit to the max for our server and see if it resolves the issue.
Seems to have extended the length of time we can run before the issue reappears, however the issue still persists.
I got the same error.
Looks like the web-console is leaking socket. I saw the number of TCP connections with Kafka brokers keep growing as the server runs. Not sure if it's problem of the web-console or my Kafka/Zookeepers.
I have the same problem with a 3node zookeeper/kafka cluster
For each: "[debug] application - Getting partition log sizes for topic test from partition leaders" the number of established sessions towards kafka grows by 15:
netstat -an | grep ESTA | grep 9092 | wc -l 15 netstat -an | grep ESTA | grep 9092 | wc -l 30 netstat -an | grep ESTA | grep 9092 | wc -l 45 netstat -an | grep ESTA | grep 9092 | wc -l 60 netstat -an | grep ESTA | grep 9092 | wc -l 75
This is from a fresh restart without browsing the GUI
This is just another version of #30.
Service crashes with the following stack strace:
Here is the function it seems to be failing in, as mentioned at the top of the stack trace (from Util.scala):