didi / KnowStreaming

一站式云原生实时流数据平台,通过0侵入、插件化构建企业级Kafka服务,极大降低操作、存储和管理实时流数据门槛
https://knowstreaming.com
GNU Affero General Public License v3.0
6.9k stars 1.28k forks source link

kafka配置了跨网段,JMX不能指定访问的地址? #502

Closed lixworld closed 1 year ago

lixworld commented 2 years ago

kafka3.2部署在内网,做了内外网分流配置(其中外网做了端口19093的NAT映射):

listeners=CLIENT://:9092,EXTERNAL://:19093
advertised.listeners=CLIENT://10.2.0.113:9092,EXTERNAL://14.23.111.111:19093
inter.broker.listener.name=CLIENT

并且设置了 JMX_PORT=9999,但9999端口没有映射到外网访问。

LogiKM跟kafka一样部署在内网服务器,集群访问的JMX是外网的地址:14.23.111.111:9999,导致连接失败。

java.io.IOException: Failed to retrieve RMIServer stub: javax.naming.ServiceUnavailableException [Root exception is java.rmi.ConnectException: Connection refused to host: 14.23.111.111; 
nested exception is: 
        java.net.ConnectException: Connection timed out (Connection timed out)]
        at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:369)
        at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:270)
        at com.xiaojukeji.kafka.manager.common.utils.jmx.JmxConnectorWrap.createJmxConnector(JmxConnectorWrap.java:137)
        at com.xiaojukeji.kafka.manager.common.utils.jmx.JmxConnectorWrap.safeCreateJmxConnector(JmxConnectorWrap.java:105)
        at com.xiaojukeji.kafka.manager.common.utils.jmx.JmxConnectorWrap.checkJmxConnectionAndInitIfNeed(JmxConnectorWrap.java:74)
        at com.xiaojukeji.kafka.manager.service.service.impl.JmxServiceImpl.getTopicMetrics(JmxServiceImpl.java:167)
        at com.xiaojukeji.kafka.manager.service.service.impl.JmxServiceImpl.getTopicMetrics(JmxServiceImpl.java:133)
        at com.xiaojukeji.kafka.manager.service.service.impl.JmxServiceImpl$1.call(JmxServiceImpl.java:96)
        at com.xiaojukeji.kafka.manager.service.service.impl.JmxServiceImpl$1.call(JmxServiceImpl.java:93)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

JMX能指定访问内网地址 10.2.0.113:9999 吗?

ZQKC commented 2 years ago

辛苦提供一份Broker在ZK注册的元数据,在ZK的/brokers/ids/{brokerId}这个节点,我们看一下如何解析。

对这个数据的解析的代码是在: https://github.com/didi/LogiKM/blob/master/kafka-manager-common/src/main/java/com/xiaojukeji/kafka/manager/common/zookeeper/znode/brokers/BrokerMetadata.java

也欢迎提PR,优化对这种场景的支持。

lixworld commented 2 years ago

辛苦提供一份Broker在ZK注册的元数据,在ZK的/brokers/ids/{brokerId}这个节点,我们看一下如何解析。

对这个数据的解析的代码是在: https://github.com/didi/LogiKM/blob/master/kafka-manager-common/src/main/java/com/xiaojukeji/kafka/manager/common/zookeeper/znode/brokers/BrokerMetadata.java

也欢迎提PR,优化对这种场景的支持。

[zk: 192.168.0.18:12181(CONNECTED) 6] get /brokers/ids/1
{"listener_security_protocol_map":{"CLIENT":"PLAINTEXT","EXTERNAL":"PLAINTEXT"},"endpoints":["CLIENT://10.2.0.113:9092","EXTERNAL://14.23.111.111:19093"],"jmx_port":9999,"features":{},"host":"10.2.0.113","timestamp":"1656986540469","port":9092,"version":5}
ZQKC commented 1 year ago

https://github.com/didi/KnowStreaming/blob/master/docs/dev_guide/%E8%A7%A3%E5%86%B3%E8%BF%9E%E6%8E%A5JMX%E5%A4%B1%E8%B4%A5.md

v3.0.0-beta.1版本已进行优化解决,可以尝鲜使用一下

ZQKC commented 1 year ago

无更多反馈,关闭该问题~