[akka://kafka-manager-system/user/kafka-manager/mycluster/broker-view] Failed to get topic metrics for broker BrokerIdentity(2,node.server.com,9092,9999) WARNING arguments left: 1

[ashishsoni]

Please let me know how can i enable the JMX as it keep saying below error and i cant see the metric in the UI

[error] k.m.KafkaJMX$ - Failed to connect to service:jmx:rmi:///jndi/rmi://NODE6.sever.com:9999/jmxrmi java.rmi.ConnectException: Connection refused to host: 127.0.1.1; nested exception is: java.net.ConnectException: Connection refused at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:619) ~[na:1.7.0_79] at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:216) ~[na:1.7.0_79] at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:202) ~[na:1.7.0_79] at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:129) ~[na:1.7.0_79] at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source) ~[na:1.7.0_79] Caused by: java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) ~[na:1.7.0_79] at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) ~[na:1.7.0_79] at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) ~[na:1.7.0_79] at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) ~[na:1.7.0_79] at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) ~[na:1.7.0_79] [ERROR] [05/19/2015 13:56:45.492] [pool-14-thread-1] [akka://kafka-manager-system/user/kafka-manager/mycluster/broker-view] Failed to get topic metrics for broker BrokerIdentity(2,NODE6.server.com,9092,9999) WARNING arguments left: 1

[patelh]

Check the quickstart for jmx: https://kafka.apache.org/08/quickstart.html

[patelh]

@ashishsoni has this been resolved? Any update?

[sslavic]

Was hit by same issue. After fixing bug in BrokerViewCacheActor (see #89), error cause was printed:

[ERROR] [06/26/2015 01:08:28.086] [pool-14-thread-7] [akka://kafka-manager-system/user/kafka-manager/local/broker-view] Failed to get broker metrics for BrokerIdentity(1,broker1,9092,9001)
java.rmi.ConnectException: Connection refused to host: 127.0.1.1; nested exception is:
    java.net.ConnectException: Operation timed out
    at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:619)
    at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:216)
    at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:202)
    at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:130)
    at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source)
    at javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2432)
    at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:308)
    at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:270)
    at kafka.manager.KafkaJMX$.doWithConnection(KafkaJMX.scala:36)
    at kafka.manager.BrokerViewCacheActor$$anonfun$updateView$1$$anonfun$apply$6$$anonfun$apply$13$$anonfun$apply$14$$anonfun$apply$2.apply$mcV$sp(BrokerViewCacheActor.scala:194)
    at kafka.manager.BrokerViewCacheActor$$anonfun$updateView$1$$anonfun$apply$6$$anonfun$apply$13$$anonfun$apply$14$$anonfun$apply$2.apply(BrokerViewCacheActor.scala:193)
    at kafka.manager.BrokerViewCacheActor$$anonfun$updateView$1$$anonfun$apply$6$$anonfun$apply$13$$anonfun$apply$14$$anonfun$apply$2.apply(BrokerViewCacheActor.scala:193)
    at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
    at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.ConnectException: Operation timed out
    at java.net.PlainSocketImpl.socketConnect(Native Method)
    at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:345)
    at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
    at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
    at java.net.Socket.connect(Socket.java:589)
    at java.net.Socket.connect(Socket.java:538)
    at java.net.Socket.<init>(Socket.java:434)
    at java.net.Socket.<init>(Socket.java:211)
    at sun.rmi.transport.proxy.RMIDirectSocketFactory.createSocket(RMIDirectSocketFactory.java:40)
    at sun.rmi.transport.proxy.RMIMasterSocketFactory.createSocket(RMIMasterSocketFactory.java:148)
    at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:613)
    ... 16 more

My Kafka brokers were running in VMs, and from exception stacktrace I understood that broker JVM RMI server was using loopback interface IP address on VMs as hostname, preventing remote RMI/JMX connections to work. Then I configured KAFKA_JMX_OPTS besides everything set to it in kafka-run-class.sh also to set -Djava.rmi.server.hostname to Kafka broker's advertised host name - after those changes, and restart of brokers I was able to connect with jconsole to brokers.

kafka-manager confused me a bit even after that - with no topics created in the cluster, broker view page displays error message.

[tebriel]

I had this same problem, was running in Vagrant, the /etc/hosts file listed 127.0.0.1 hostname hostname, after removing that, I no longer had an issue.

[MonkeyDLuphy]

after edit the /etc/hosts

as below: "use broker.ip instead of 127.0.0.1" localhost localhost.localdomain

then the error never shows again but,i didn't know if there exist any risk after i editing /etc/hosts file

[Siouffy]

Just had the same error and the fix by @MonkeyDLuphy works 100% On the target broker that u can't reach from kafka-manager vi /etc/hosts comment/remove: 127.0.0.1 localhost add: localhost localhost.localdomain next to the machines ip then restart the broker :)

[lonfme]

Open file ./bin/kafka-server-start.sh to add text:

[superleo-cn]

Facing the same issue and after trying all of the above ways, still cannot make it out.

[orderedchaosdev]

So I know this is old but you should know rmi picks a random port normally you can add -Dcom.sun.management.jmxremote.rmi.port=8998 to make it static instead so that way you can open that port and the jmx port and it will connected this is in addition the the other options listed above

[nikostrongioglou]

So is there a clearly defined approach if one is trying to connect to a dockerised Kafka instance via Kafka Manager? I believe that there is no point in enumerating the entirety of docker-compose file configurations I have tried but for sure none of them has worked...

(JAVA_OPTS, JMX_PORT, JMX_HOST, EXT_HOST)

May somebody post a Kafka manager accessible Kafka docker-compose file?

Thanks

[jbz]

Our brokers run on VMs and on changing IPs. We bind their hostnames and localhost to 127.0.0.1 to have permanence in /etc/hosts (and to guarantee local processes can look up and reach local ports). It seems like this relies on the remote host having its external - non-loopback - IP as the answer to the remote host's own lookup of its hostname. This is...not optimal. Is there any way to force kafka-manager to use DNS names to connect to brokers, rather than trying this dance with the cluster self-reported IP lookups? It doesn't make sense to me that a process is relying on an IP lookup that was performed by a remote host to make a connection to that host. The entire purpose of DNS is to permit localized network address overrides and handling.

Hm, going to try setting java.rmi.server.useLocalHostname to true on the broker and see if that forces it to report the DNS name vs. the IP to the zookeeper server.

[xjliang2007]

maybe you can set your /etc/hosts file at broker server, change the 127.0.0.1 your-hostname to intranet-ip hostname, and then restart your kafka app