Kafka connection native mode failing with kerberos

Describe the bug

App is trying connecting to kafka with kerberos auth in native mode and is receiving error. 2021-08-12 20:28:25,309 WARN [org.apa.kaf.com.net.Selector] (smallrye-kafka-consumer-thread-0) [Consumer clientId=eviq-test, groupId=eviq5555-test] Unexpected error from dcmidph02k004-a.epg.nam.gm.com/10.127.129.67; closing connection: java.lang.NullPointerException at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:229) at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:196) at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:203) at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator.lambda$createSaslToken$1(SaslClientAuthenticator.java:534)

Expected behavior

The expectation is to connect to kafka in native mode as it does in jvm mode the following are setting Trying to connect to kafka using kerberos with settings below:

kafka.ssl.truststore.location=/work/truststore.jks kafka.ssl.truststore.password=xxxxx kafka.ssl.endpoint.identification.algorithm=https kafka.sasl.mechanism=GSSAPI kafka.security.protocol=SASL_SSL kafka.sasl.jaas.config=com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true doNotPrompt=true keyTab="file:/work/sample.keytab" storeKey=true useTicketCache=false debug=true principal="user@user.com"; kafka.sasl.kerberos.service.name=kafka mp.messaging.incoming.events.connector=smallrye-kafka mp.messaging.incoming.events.topic=testtopic mp.messaging.incoming.events.group.id=test mp.messaging.incoming.events.client.id=test mp.messaging.incoming.events.key.deserializer=org.apache.kafka.common.serialization.StringDeserializer mp.messaging.incoming.events.value.deserializer=org.apache.kafka.common.serialization.StringDeserializer mp.messaging.incoming.events.enable.auto.commit=false mp.messaging.incoming.events.auto.offset.reset=earliest quarkus.ssl.native=true quarkus.native.additional-build-args=--initialize-at-run-time=org.apache.kafka.common.security.kerberos.KerberosLogin, -J-Djava.security.krb5.conf=/etc/krb5.conf, -J-Djava.security.krb5.realm=xxx, -J-Djava.security.krb5.kdc=xxxx, -J-Djavax.net.ssl.trustStore=/work/truststore.jks,-J-Djavax.net.ssl.trustStorePassword=xxx quarkus.native.enable-all-security-services=true quarkus.security.security-providers=SunRsaSign,SunJCE,SunJGSS,SunSASL

Actual behavior

The app is failing with nullpointer exception . 2021-08-12 20:28:25,309 WARN [org.apa.kaf.com.net.Selector] (smallrye-kafka-consumer-thread-0) [Consumer clientId=test, groupId=test] Unexpected error from xxxxxxxxxx; closing connection: java.lang.NullPointerException at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:229) at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:196) at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:203) at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator.lambda$createSaslToken$1(SaslClientAuthenticator.java:534) at java.security.AccessController.doPrivileged(AccessController.java:145) at javax.security.auth.Subject.doAs(Subject.java:36) at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator.createSaslToken(SaslClientAuthenticator.java:534) at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator.sendSaslClientToken(SaslClientAuthenticator.java:433) at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator.sendInitialToken(SaslClientAuthenticator.java:332) at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator.authenticate(SaslClientAuthenticator.java:273) at org.apache.kafka.common.network.KafkaChannel.prepare(KafkaChannel.java:181) at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:543) at org.apache.kafka.common.network.Selector.poll(Selector.java:481) at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:561) at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:265) at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:236) at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:215) at org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureCoordinatorReady(AbstractCoordinator.java:245) at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.poll(ConsumerCoordinator.java:480)

How to Reproduce?

No response

Output of `uname -a` or `ver`

No response

Output of `java -version`

GraalVM version (if different from Java)

No response

Quarkus version or git rev

2.1.2

Build tool (ie. output of `mvnw --version` or `gradlew --version`)

No response

Additional information

The following is the setting:

kafka.ssl.truststore.location=/work/truststore.jks kafka.ssl.truststore.password=xxxxx kafka.ssl.endpoint.identification.algorithm=https kafka.sasl.mechanism=GSSAPI kafka.security.protocol=SASL_SSL kafka.sasl.jaas.config=com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true doNotPrompt=true keyTab="file:/work/sample.keytab" storeKey=true useTicketCache=false debug=true principal="user@user.com"; kafka.sasl.kerberos.service.name=kafka mp.messaging.incoming.events.connector=smallrye-kafka mp.messaging.incoming.events.topic=testtopic mp.messaging.incoming.events.group.id=test mp.messaging.incoming.events.client.id=test mp.messaging.incoming.events.key.deserializer=org.apache.kafka.common.serialization.StringDeserializer mp.messaging.incoming.events.value.deserializer=org.apache.kafka.common.serialization.StringDeserializer mp.messaging.incoming.events.enable.auto.commit=false mp.messaging.incoming.events.auto.offset.reset=earliest quarkus.ssl.native=true quarkus.native.additional-build-args=--initialize-at-run-time=org.apache.kafka.common.security.kerberos.KerberosLogin, -J-Djava.security.krb5.conf=/etc/krb5.conf, -J-Djava.security.krb5.realm=xxx, -J-Djava.security.krb5.kdc=xxxx, -J-Djavax.net.ssl.trustStore=/work/truststore.jks,-J-Djavax.net.ssl.trustStorePassword=xxx quarkus.kafka.health.enabled=false quarkus.native.enable-all-security-services=true quarkus.security.security-providers=SunRsaSign,SunJCE,SunJGSS,SunSASL

The below is the log until we get the error 2021-08-12 20:28:24,573 INFO [org.apa.kaf.cli.con.ConsumerConfig] (main) ConsumerConfig values: allow.auto.create.topics = false auto.commit.interval.ms = 5000 auto.offset.reset = earliest bootstrap.servers = [xxxxxxx] check.crcs = true 2021-08-12 20:28:24,574 DEBUG [org.apa.kaf.cli.con.KafkaConsumer] (main) [Consumer clientId=test, groupId=test] Initializing the Kafka consumer Debug is true storeKey true useTicketCache false useKeyTab true doNotPrompt true ticketCache is null isInitiator true KeyTab is /work/D_HUB.keytab refreshKrb5Config is false principal is xxxxxxxxxxxx tryFirstPass is false useFirstPass is false storePass is false clearPass is false principal is xxxxxxxxx Will use keytab Commit Succeeded

2021-08-12 20:28:25,113 INFO [org.apa.kaf.com.sec.aut.AbstractLogin] (main) Successfully logged in. 2021-08-12 20:28:25,113 DEBUG [org.apa.kaf.com.sec.ker.KerberosLogin] (main) [Principal=xxxxxxxxx]: It is a Kerberos ticket
2021-08-12 20:28:25,117 INFO [io.quarkus] (main) Profile prod activated. 2021-08-12 20:28:25,117 INFO [io.quarkus] (main) Installed features: [cdi, kafka-client, resteasy, resteasy-jackson, security, smallrye-context-propagation, smallrye-reactive-messaging, smallrye-reactive-messaging-kafka, vertx] 2021-08-12 20:28:25,152 DEBUG [org.apa.kaf.com.net.Selector] (smallrye-kafka-consumer-thread-0) [Consumer clientId=test, groupId=test5555-test] Created socket with SO_RCVBUF = 65536, SO_SNDBUF = 131072, SO_TIMEOUT = 0 to node -4 2021-08-12 20:28:25,156 DEBUG [org.apa.kaf.cli.NetworkClient] (smallrye-kafka-consumer-thread-0) [Consumer clientId=test, groupId=test5555-test] Completed connection to node -4. Fetching API versions. 2021-08-12 20:28:25,241 DEBUG [org.apa.kaf.com.net.SslTransportLayer] (smallrye-kafka-consumer-thread-0) [SslTransportLayer channelId=-4 key=channel=java.nio.channels.SocketChannel[connection-pending remote=xxxxxxxxxxx:9094], selector=sun.nio.ch.EPollSelectorImpl@56a25b20, interestOps=8, readyOps=0] SSL handshake completed successfully with peerHost 'xxxx' peerPort 9094 peerPrincipal 'xxxxxxxxxxxxxxxxxx' cipherSuite 'TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384' 2021-08-12 20:28:25,242 DEBUG [org.apa.kaf.com.sec.aut.SaslClientAuthenticator] (smallrye-kafka-consumer-thread-0) [Consumer clientId=test, groupId=test] Set SASL client state to RECEIVE_APIVERSIONS_RESPONSE 2021-08-12 20:28:25,280 DEBUG [org.apa.kaf.com.sec.aut.SaslClientAuthenticator] (smallrye-kafka-consumer-thread-0) [Consumer clientId=test, groupId=test] Set SASL client state to SEND_HANDSHAKE_REQUEST 2021-08-12 20:28:25,281 DEBUG [org.apa.kaf.com.sec.aut.SaslClientAuthenticator] (smallrye-kafka-consumer-thread-0) [Consumer clientId=test, groupId=test] Set SASL client state to RECEIVE_HANDSHAKE_RESPONSE 2021-08-12 20:28:25,309 DEBUG [org.apa.kaf.com.sec.aut.SaslClientAuthenticator] (smallrye-kafka-consumer-thread-0) [Consumer clientId=test, groupId=test] Set SASL client state to INITIAL 2021-08-12 20:28:25,309 WARN [org.apa.kaf.com.net.Selector] (smallrye-kafka-consumer-thread-0) [Consumer clientId=test, groupId=test] Unexpected error from xxxxxxxxxx; closing connection: java.lang.NullPointerException at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:229) at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:196) at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:203) at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator.lambda$createSaslToken$1(SaslClientAuthenticator.java:534) at java.security.AccessController.doPrivileged(AccessController.java:145) at javax.security.auth.Subject.doAs(Subject.java:36)

quarkusio / quarkus