search

didi / KnowStreaming

一站式云原生实时流数据平台，通过0侵入、插件化构建企业级Kafka服务，极大降低操作、存储和管理实时流数据门槛
https://knowstreaming.com
GNU Affero General Public License v3.0
6.99k stars 1.28k forks source link

zookeeper开启了ACL权限认证，zookeeper的认证配置的在哪设置 #1061

Open zhaoyu810373097 opened 1 year ago

zhaoyu810373097 commented 1 year ago

zookeeper配合kafka都开启的sasl权限认证，kafka可以设置sasl认证配置，zookeeper在哪里设置

ZQKC commented 1 year ago

zookeeper配合kafka都开启的sasl权限认证，kafka可以设置sasl认证配置，zookeeper在哪里设置

见doc目录下的文档：https://github.com/didi/KnowStreaming/blob/master/docs/dev_guide/%E6%8E%A5%E5%85%A5ZK%E5%B8%A6%E8%AE%A4%E8%AF%81Kafka%E9%9B%86%E7%BE%A4.md

zhaoyu810373097 commented 1 year ago

我的zookeeper用的是org.apache.kafka.common.security.plain.PlainLoginModule,其它配置按照文档中进行配置，出现下面报错 2023-06-30 14:26:59.939 [MetadataTaskTP-6-thread-21] ERROR class=c.x.k.s.k.c.utils.zookeeper.FourLetterWordUtil||method=executeFourLetterCmd||clusterPhyId=2||host=192.168.22.123||port=2181||cmd=srvr||secure=true||timeout=5000||errMsg=exception! javax.net.ssl.SSLHandshakeException: Remote host terminated the handshake at sun.security.ssl.SSLSocketImpl.handleEOF(SSLSocketImpl.java:1379) at sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1206) at sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1107) at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:400) at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:372) at com.xiaojukeji.know.streaming.km.common.utils.zookeeper.FourLetterWordUtil.send4LetterWord(FourLetterWordUtil.java:94) at com.xiaojukeji.know.streaming.km.common.utils.zookeeper.FourLetterWordUtil.executeFourLetterCmd(FourLetterWordUtil.java:48) at com.xiaojukeji.know.streaming.km.core.service.zookeeper.impl.ZookeeperServiceImpl.getFromZookeeperCluster(ZookeeperServiceImpl.java:150) at com.xiaojukeji.know.streaming.km.core.service.zookeeper.impl.ZookeeperServiceImpl.listFromZookeeper(ZookeeperServiceImpl.java:53) at com.xiaojukeji.know.streaming.km.task.kafka.metadata.SyncZookeeperTask.processClusterTask(SyncZookeeperTask.java:33) at com.xiaojukeji.know.streaming.km.task.kafka.metadata.AbstractAsyncMetadataDispatchTask.lambda$asyncProcessSubTask$0(AbstractAsyncMetadataDispatchTask.java:33) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Suppressed: java.net.SocketException: Broken pipe (Write failed) at java.net.SocketOutputStream.socketWrite0(Native Method) at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:111) at java.net.SocketOutputStream.write(SocketOutputStream.java:155) at sun.security.ssl.SSLSocketOutputRecord.encodeAlert(SSLSocketOutputRecord.java:81) at sun.security.ssl.TransportContext.fatal(TransportContext.java:358) at sun.security.ssl.TransportContext.fatal(TransportContext.java:270) at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:410) ... 12 common frames omitted Caused by: java.io.EOFException: SSL peer shut down incorrectly at sun.security.ssl.SSLSocketInputRecord.read(SSLSocketInputRecord.java:471) at sun.security.ssl.SSLSocketInputRecord.readHeader(SSLSocketInputRecord.java:460) at sun.security.ssl.SSLSocketInputRecord.decode(SSLSocketInputRecord.java:159) at sun.security.ssl.SSLTransport.decode(SSLTransport.java:110) at sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1198) ... 14 common frames omitted

ZQKC commented 1 year ago

我的zookeeper用的是org.apache.kafka.common.security.plain.PlainLoginModule,其它配置按照文档中进行配置，出现下面报错 2023-06-30 14:26:59.939 [MetadataTaskTP-6-thread-21] ERROR class=c.x.k.s.k.c.utils.zookeeper.FourLetterWordUtil||method=executeFourLetterCmd||clusterPhyId=2||host=192.168.22.123||port=2181||cmd=srvr||secure=true||timeout=5000||errMsg=exception! javax.net.ssl.SSLHandshakeException: Remote host terminated the handshake at sun.security.ssl.SSLSocketImpl.handleEOF(SSLSocketImpl.java:1379) at sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1206) at sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1107) at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:400) at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:372) at com.xiaojukeji.know.streaming.km.common.utils.zookeeper.FourLetterWordUtil.send4LetterWord(FourLetterWordUtil.java:94) at com.xiaojukeji.know.streaming.km.common.utils.zookeeper.FourLetterWordUtil.executeFourLetterCmd(FourLetterWordUtil.java:48) at com.xiaojukeji.know.streaming.km.core.service.zookeeper.impl.ZookeeperServiceImpl.getFromZookeeperCluster(ZookeeperServiceImpl.java:150) at com.xiaojukeji.know.streaming.km.core.service.zookeeper.impl.ZookeeperServiceImpl.listFromZookeeper(ZookeeperServiceImpl.java:53) at com.xiaojukeji.know.streaming.km.task.kafka.metadata.SyncZookeeperTask.processClusterTask(SyncZookeeperTask.java:33) at com.xiaojukeji.know.streaming.km.task.kafka.metadata.AbstractAsyncMetadataDispatchTask.lambda$asyncProcessSubTask$0(AbstractAsyncMetadataDispatchTask.java:33) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Suppressed: java.net.SocketException: Broken pipe (Write failed) at java.net.SocketOutputStream.socketWrite0(Native Method) at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:111) at java.net.SocketOutputStream.write(SocketOutputStream.java:155) at sun.security.ssl.SSLSocketOutputRecord.encodeAlert(SSLSocketOutputRecord.java:81) at sun.security.ssl.TransportContext.fatal(TransportContext.java:358) at sun.security.ssl.TransportContext.fatal(TransportContext.java:270) at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:410) ... 12 common frames omitted Caused by: java.io.EOFException: SSL peer shut down incorrectly at sun.security.ssl.SSLSocketInputRecord.read(SSLSocketInputRecord.java:471) at sun.security.ssl.SSLSocketInputRecord.readHeader(SSLSocketInputRecord.java:460) at sun.security.ssl.SSLSocketInputRecord.decode(SSLSocketInputRecord.java:159) at sun.security.ssl.SSLTransport.decode(SSLTransport.java:110) at sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1198) ... 14 common frames omitted

ZK的四字命令可以使用么？看看在终端如何使用四字命令

zhaoyu810373097 commented 1 year ago

06f7bf50f7faed07dcc40925ef8d9e5(1)(1)

ZQKC commented 1 year ago

06f7bf50f7faed07dcc40925ef8d9e5(1)(1)

是KS的一个BUG，后续修复一下，老哥有兴趣的话，也欢迎贡献PR。

WhiteStart commented 1 year ago

修改 KnowStreaming 的启动脚本

KnowStreaming/bin/startup.sh 中的 47 行的 JAVA_OPT 中追加如下设置

-Djava.security.auth.login.config=/xxx/zk_client_jaas.conf

请问这里采用docker运行的话应该如何配置

ZQKC commented 1 year ago

修改 KnowStreaming 的启动脚本

KnowStreaming/bin/startup.sh 中的 47 行的 JAVA_OPT 中追加如下设置

-Djava.security.auth.login.config=/xxx/zk_client_jaas.conf

请问这里采用docker运行的话应该如何配置

对容器不是很了解。 觉得可以参照application.yml这个文件怎么生效的来增加zk_client_jaas.conf这个文件（觉得可以使用ADD命令加进来），先试着看能否成功加进来，然后再在startup.sh里面，增加-Djava.security.auth.login.config=/xxx/zk_client_jaas.conf这个参数。