Error connecting to Kafka with SCRAM-SHA-512 mechanism when running as native

abishai commented 2 years ago

The issue looks like https://github.com/micronaut-projects/micronaut-kafka/issues/264 but with another connection options.

Expected Behavior

Successful connection to kafka server

Actual Behaviour

org.apache.kafka.common.errors.SaslAuthenticationException: Failed to configure SaslClientAuthenticator
Caused by: org.apache.kafka.common.errors.SaslAuthenticationException: Failed to create SaslClient with mechanism SCRAM-SHA-512
15:28:33.816 [kafka-producer-network-thread | producer-2] INFO  o.a.k.c.network.SaslChannelBuilder - [Producer clientId=producer-2] Failed to create channel due to 
org.apache.kafka.common.errors.SaslAuthenticationException: Failed to configure SaslClientAuthenticator
Caused by: org.apache.kafka.common.errors.SaslAuthenticationException: Failed to create SaslClient with mechanism SCRAM-SHA-512
15:28:33.816 [kafka-producer-network-thread | producer-2] WARN  o.apache.kafka.clients.NetworkClient - [Producer clientId=producer-2] Error connecting to node a3f29642800cd4fc488efa01ad323788-1018565300.eu-west-1.elb.amazonaws.com:9094 (id: -1 rack: null)

After I enabled debug in kafka library, the latest successful log was:

15:28:33.816 [pool-1-thread-2] DEBUG o.a.k.c.s.a.SaslClientAuthenticator - [Consumer clientId=<snip>, groupId=<snip>] Creating SaslClient: client=null;service=kafka;serviceHostname=<snip>.eu-west-1.elb.amazonaws.com;mechs=[SCRAM-SHA-512]

I believe, we received null here https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/common/security/authenticator/SaslClientAuthenticator.java#L221 because Sasl library was unable to find mechanism requested.

Steps To Reproduce

Default micronaut-kafka example, compiled natively with following settings. Can be verified without actual kafka server setup, the error occurs before connection.

kafka:
  bootstrap.servers: [server]
  security.protocol: SASL_PLAINTEXT
  sasl.jaas.config: org.apache.kafka.common.security.scram.ScramLoginModule required username="[my-key]" password="[my-password]";
  sasl.mechanism: SCRAM-SHA-512

Environment Information

OpenJDK11

Example Application

No response

Version

3.2.3

abishai commented 2 years ago

Strangely, but additional security provider wasn't loaded into java Security, with following hints I was able to connect

-H:AdditionalSecurityProviders=org.apache.kafka.common.security.scram.internals.ScramSaslClientProvider

{
    "name": "org.apache.kafka.common.security.scram.ScramLoginModule",
    "allDeclaredConstructors": true,
    "allDeclaredMethods": true
  },
  {
    "name": "org.apache.kafka.common.security.scram.ScramSaslClient",
    "allDeclaredConstructors": true,
    "allDeclaredMethods": true
  },
  {
    "name": "org.apache.kafka.common.security.scram.internals.ScramSaslClient$ScramSaslClientFactory",
    "allDeclaredConstructors": true,
    "allDeclaredMethods": true
  },

dstepanov commented 2 years ago

Would you be able to contribute a sample app for future testing in https://github.com/micronaut-projects/micronaut-kafka/tree/master/tests ?

micronaut-projects / micronaut-kafka