yahoo / CMAK

CMAK is a tool for managing Apache Kafka clusters
Apache License 2.0
11.84k stars 2.5k forks source link

java.lang.OutOfMemoryError: Java heap space #352

Open dasrintu opened 7 years ago

dasrintu commented 7 years ago

Hi ,we are having JVM (MEMEORY)issue with our kafkamanager. Starting the kafkaManager giving -Exception in thread "KafkaManagedOffsetCache" java.lang.OutOfMemoryError: Java heap space

we have 3 zookeeper server on the configuration file of KM Kindly let us know how to increase the memory of the Kafka Manager.

patelh commented 7 years ago

did you increase the heap?

dasrintu commented 7 years ago

No,i am not sure which file does have the JVM configured. I have checked "application.conf" file,but look like it doesn't have memory info.. But i have checked -/kmsrc/sbt file,this file has "default_jvm_opts_common" option,but as far as i know -. This folder contains the source files used to build the package and will not contain the runtime configurations.

alexsavio commented 7 years ago

Hi,

We have 4 computers in total, 4 with Kafka Broker, and 3 with Zookeeper.

We are seeing that Kafka-Manager is consuming the whole Java Heap and using a lot of CPU, one core is always at 100%. Is this a known issue?

The configuration we are using is:

# The application languages
# ~~~~~
play.i18n.langs=["en"]

play.http.requestHandler = "play.http.DefaultHttpRequestHandler"
play.http.context = "/"
play.application.loader=loader.KafkaManagerLoader

kafka-manager.zkhosts="{{ zookeeper_hosts }}"
kafka-manager.zkhosts=${?ZK_HOSTS}
pinned-dispatcher.type="PinnedDispatcher"
pinned-dispatcher.executor="thread-pool-executor"
application.features=["KMClusterManagerFeature","KMTopicManagerFeature","KMPreferredReplicaElectionFeature","KMReassignPartitionsFeature"]

akka {
  loggers = ["akka.event.slf4j.Slf4jLogger"]
  loglevel = "INFO"
}

basicAuthentication.enabled=true
basicAuthentication.username="{{ kafka_manager_user }}"
basicAuthentication.password="{{ kafka_manager_password }}"
basicAuthentication.realm="Kafka-Manager"
basicAuthentication.excluded=["/api/health"] # ping the health of your instance without authentification

kafka-manager.consumer.properties.file=${?CONSUMER_PROPERTIES_FILE}

And the heap size we are using:

kafka_manager_heap_s: 2g
kafka_manager_heap_x: 4g

For us it is strange that it is using so much CPU, however we love the project and it is very useful!

Any insight? Thanks a lot.

Cheers, Alex

joshbranham commented 6 years ago

@alexsavio where were you able to set the manager heap settings?

wjw870970 commented 4 years ago

hi dasrintu Did you have resolved the problem about java heap space

andrewkowalik commented 4 years ago

In case anyone else searches this issue.

Heap can be set at runtime as a Java arg.

bin/cmak -J-Xmx1g

@patelh Can we close this issue?