search

yahoo / CMAK

CMAK is a tool for managing Apache Kafka clusters
Apache License 2.0
11.84k stars 2.5k forks source link

Kafkamanager support for kafka 2.3.1 version #695

Open anirudhreddy6 opened 5 years ago

anirudhreddy6 commented 5 years ago

We just deployed a new kafkacluster with 2.3.1 version. Trying to add the cluster in kafkamamager ,versions are till only 2.2.0. Tried configuring it with 2.2.0. Comes back with

"Yikes! Ask timed out on [ActorSelection[Anchor(akka://kafka-manager-system/), Path(/user/kafka-manager/kafka53/kafka-state)]] after [2000 ms]. Message of type [kafka.manager.model.ActorModel$KSGetBrokers$] was sent by [Actor[akka://kafka-manager-system/user/kafka-manager/kafka53#-1669570313]]. A typical reason for AskTimeoutException is that the recipient actor didn't send a reply."

Is kafkamanager currently supporting this version? Any update would be helpful. Thanks

austinnichols101 commented 5 years ago

Also see #665

henhiskan commented 5 years ago

Got same error, but I think my problem is related to the API version 3 not being supported. From logs:

2019-11-11 21:33:09,135 - [WARN] k.m.a.c.KafkaManagedOffsetCache - Failed to process a message from offset topic on cluster kafka-kafka.data!
org.apache.kafka.common.KafkaException: Unknown group metadata version 3
    at kafka.manager.utils.one10.GroupMetadataManager$.schemaForGroupValue(GroupMetadataManager.scala:554)
    at kafka.manager.utils.one10.GroupMetadataManager$.readGroupMessageValue(GroupMetadataManager.scala:726)
    at kafka.manager.actor.cluster.KafkaManagedOffsetCache.$anonfun$run$3(KafkaStateActor.scala:377)
    at kafka.manager.actor.cluster.KafkaManagedOffsetCache.$anonfun$run$3$adapted(KafkaStateActor.scala:325)
    at scala.util.Success.foreach(Try.scala:253)
    at kafka.manager.actor.cluster.KafkaManagedOffsetCache.run(KafkaStateActor.scala:325)
    at java.lang.Thread.run(Thread.java:748)
hcbraun commented 4 years ago

Getting the same with Kafka 2.4.0 and manager 2.0.0.2. I think it is caused by change in "Consumer Instance Owner" on the brokers.

2020-01-09 06:20:04,036 - [WARN] k.m.a.c.KafkaManagedOffsetCache - Failed to process a message from offset topic on cluster kafka!
org.apache.kafka.common.KafkaException: Unknown group metadata version 3
    at kafka.manager.utils.one10.GroupMetadataManager$.schemaForGroupValue(GroupMetadataManager.scala:554)
    at kafka.manager.utils.one10.GroupMetadataManager$.readGroupMessageValue(GroupMetadataManager.scala:726)
    at kafka.manager.actor.cluster.KafkaManagedOffsetCache.$anonfun$run$3(KafkaStateActor.scala:377)
    at kafka.manager.actor.cluster.KafkaManagedOffsetCache.$anonfun$run$3$adapted(KafkaStateActor.scala:325)
    at scala.util.Success.foreach(Try.scala:253)
    at kafka.manager.actor.cluster.KafkaManagedOffsetCache.run(KafkaStateActor.scala:325)
    at java.lang.Thread.run(Thread.java:748)
116davinder commented 4 years ago

upgrade CMAK to 3.x it should be fixed, I was using CMAK 3.x with Kafka 2.3 and 2.4 .

poyyi commented 4 years ago

I'm using cmak 3.0.0.4 with Kafka version 2.3.1....Lags are not showing up on consumers... using SASL_SSL with plain ....i don't see dropdown in kafkaversion to support 2.3.0 or 2.3.1 i tired 3 different version 2.0.0.0,CPBEX,and 3.0.0.4 ..running into same problem is something wrong in zookeeper? Errors: org.apache.kafka.common.errors.TimeoutException: Failed to get offsets by times in 30000ms Any help is appreciated Thanks