Closed natedogs911 closed 7 years ago
Also, I'm working on some scripts to automate the launch of the KDC, Kafka/zookeeper instance to be used in a set of docker images etc. not sure if you are interested in a PR if I can get it all working smoothly
Can you see if Kafka is hitting Kerby to get a ticket? You should be able to see some output on the Kerby console when Kafka is started.
here's an example KDC log,
Running org.apache.coheigea.bigdata.kerberos.kafka.KafkaKerbyTest
KDC ready on port: 56632
Oct 24, 2017 2:15:43 PM org.apache.kerby.kerberos.kerb.server.request.KdcRequest preauth
INFO: The preauth data is empty.
Oct 24, 2017 2:15:43 PM org.apache.kerby.kerberos.kerb.server.KdcHandler handleRecoverableException
INFO: KRB error occurred while processing request:Additional pre-authentication required
Oct 24, 2017 2:15:44 PM org.apache.kerby.kerberos.kerb.server.request.AsRequest issueTicket
INFO: AS_REQ ISSUE: authtime 1508879744127,client@spot.local for krbtgt/spot.local@spot.local
Oct 24, 2017 2:15:44 PM org.apache.kerby.kerberos.kerb.server.request.TgsRequest issueTicket
INFO: TGS_REQ ISSUE: authtime 1508879744227,client for zookeeper/localhost@spot.local
Oct 24, 2017 2:34:55 PM org.apache.kerby.kerberos.kerb.server.impl.DefaultInternalKdcServerImpl doStop
INFO: Default Internal kdc server stopped.
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1,200.324 sec
Running org.apache.coheigea.bigdata.kerberos.hadoop.HadoopKerbyTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0 sec
Running org.apache.coheigea.bigdata.kerberos.hive.HiveKerbyTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0 sec
Running org.apache.coheigea.bigdata.kerberos.storm.StormKerbyTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0 sec
What I meant was look at the Kerby window + start the Kafka broker and then see if there is some output in the Kerby window to indicate that it issued a token successfully.
from the log it seems that zookeeper authenticates, but only when the Kafka instance fires up.
I don't see service tickets when running klist
, only krbtgt/local.
the sequence of events are:
start kdc
able to kinit with client
start zookeeper instance last log from zk:
[2017-10-25 23:08:01,850] INFO binding to port 0.0.0.0/0.0.0.0:2181 (org.apache.zookeeper.server.NIOServerCnxnFactory)
everything is stable at this point
start kafka, after a few seconds fatal error:
[2017-10-25 23:10:33,685] FATAL Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
org.I0Itec.zkclient.exception.ZkAuthFailedException: Authentication failure
at org.I0Itec.zkclient.ZkClient.waitForKeeperState(ZkClient.java:947)
at org.I0Itec.zkclient.ZkClient.waitUntilConnected(ZkClient.java:924)
at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:1231)
at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:157)
at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:131)
at kafka.utils.ZkUtils$.createZkClientAndConnection(ZkUtils.scala:103)
at kafka.utils.ZkUtils$.apply(ZkUtils.scala:85)
at kafka.server.KafkaServer.initZk(KafkaServer.scala:338)
at kafka.server.KafkaServer.startup(KafkaServer.scala:191)
at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:38)
at kafka.Kafka$.main(Kafka.scala:65)
at kafka.Kafka.main(Kafka.scala)
log from zookeeper at that time
[2017-10-25 23:10:33,658] INFO Accepted socket connection from /127.0.0.1:52559 (org.apache.zookeeper.server.NIOServerCnxnFactory)
[2017-10-25 23:10:33,670] INFO Client attempting to establish new session at /127.0.0.1:52559 (org.apache.zookeeper.server.ZooKeeperServer)
[2017-10-25 23:10:33,671] INFO Creating new log file: log.46 (org.apache.zookeeper.server.persistence.FileTxnLog)
[2017-10-25 23:10:33,678] INFO Established session 0x15f574b6d700000 with negotiated timeout 6000 for client /127.0.0.1:52559 (org.apache.zookeeper.server.ZooKeeperServer)
[2017-10-25 23:10:33,681] INFO Processed session termination for sessionid: 0x15f574b6d700000 (org.apache.zookeeper.server.PrepRequestProcessor)
[2017-10-25 23:10:33,684] INFO Closed socket connection for client /127.0.0.1:52559 which had sessionid 0x15f574b6d700000 (org.apache.zookeeper.server.NIOServerCnxn)
When I start Zookeeper, I see the following output in the Kerby console:
INFO: The preauth data is empty. Oct 26, 2017 12:37:47 PM org.apache.kerby.kerberos.kerb.server.KdcHandler handleRecoverableException INFO: KRB error occurred while processing request:Additional pre-authentication required Oct 26, 2017 12:37:48 PM org.apache.kerby.kerberos.kerb.server.request.AsRequest issueTicket INFO: AS_REQ ISSUE: authtime 1509017868123,zookeeper/localhost@kafka.apache.org for krbtgt/kafka.apache.org@kafka.apache.org
and in the Zookeeper console:
Client Principal = zookeeper/localhost@kafka.apache.org Server Principal = krbtgt/kafka.apache.org@kafka.apache.org Session Key = EncryptionKey: keyType=17 keyBytes (hex dump)= 0000: AD 3B F2 15 82 50 86 20 23 ED 9F 29 82 69 9B C8 .;...P. #..).i..
Forwardable Ticket false Forwarded Ticket false Proxiable Ticket false Proxy Ticket false Postdated Ticket false Renewable Ticket false Initial Ticket false Auth Time = Thu Oct 26 12:37:48 IST 2017 Start Time = Thu Oct 26 12:37:48 IST 2017 End Time = Fri Oct 27 12:37:48 IST 2017 Renew Till = null Client Addresses Null . (org.apache.zookeeper.Login) [2017-10-26 12:37:48,155] INFO TGT valid starting at: Thu Oct 26 12:37:48 IST 2017 (org.apache.zookeeper.Login) [2017-10-26 12:37:48,155] INFO TGT expires: Fri Oct 27 12:37:48 IST 2017 (org.apache.zookeeper.Login)
Are you sure that you see no output from Zookeeper as per above?
Thank you, I found the error in my zookeeper client configuration. I had been over the config 4-5 times to verify I had everything correct, however I had missed the krb5.conf variable for zookeeper was pointing to the wrong directory.
now its just to testing the client configuration at application level.
Hello,
I'm following this post: http://coheigea.blogspot.com/2017/05/securing-apache-kafka-with-kerberos.html
and have a kdc running on my local machine (Mac OS high sierra)
the scenario is as follows:
client
NOTE: some logs captured at different times, but same error messages regardless
zookeeper: