search

aws / aws-msk-iam-auth

Enables developers to use AWS Identity and Access Management (IAM) to connect to their Amazon Managed Streaming for Apache Kafka (Amazon MSK) clusters.
Apache License 2.0
138 stars 65 forks source link

NullPointException when building StsClient with overrideEndpoint #165

Open li-wang2 opened 2 months ago

li-wang2 commented 2 months ago

Hi team,

I got the following error when connecting Kafka with IAM authentication by using version 2.1.0

  ...
Caused by: org.apache.kafka.common.KafkaException: java.lang.NullPointerException: The URI scheme of endpointOverride must not be null.
    at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:184)
    at org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:192)
    at org.apache.kafka.common.network.ChannelBuilders.clientChannelBuilder(ChannelBuilders.java:81)
    at org.apache.kafka.clients.ClientUtils.createChannelBuilder(ClientUtils.java:117)
    at org.apache.kafka.clients.ClientUtils.createNetworkClient(ClientUtils.java:217)
    ... 31 common frames omitted
Caused by: java.lang.NullPointerException: The URI scheme of endpointOverride must not be null.
    at software.amazon.awssdk.utils.Validate.paramNotNull(Validate.java:156)
    at software.amazon.awssdk.core.client.builder.SdkDefaultClientBuilder.endpointOverride(SdkDefaultClientBuilder.java:502)
    at software.amazon.msk.auth.iam.internals.MSKCredentialProvider$ProviderBuilder.getStsClientBuilder(MSKCredentialProvider.java:289)
    at software.amazon.msk.auth.iam.internals.MSKCredentialProvider$ProviderBuilder.createSTSRoleCredentialProvider(MSKCredentialProvider.java:342)
    at software.amazon.msk.auth.iam.internals.MSKCredentialProvider$ProviderBuilder.lambda$getStsRoleProvider$4(MSKCredentialProvider.java:330)
    at java.base/java.util.Optional.map(Optional.java:260)
    at software.amazon.msk.auth.iam.internals.MSKCredentialProvider$ProviderBuilder.getStsRoleProvider(MSKCredentialProvider.java:307)
    at software.amazon.msk.auth.iam.internals.MSKCredentialProvider$ProviderBuilder.getProviders(MSKCredentialProvider.java:254)
    at software.amazon.msk.auth.iam.internals.MSKCredentialProvider.<init>(MSKCredentialProvider.java:108)
    at software.amazon.msk.auth.iam.internals.MSKCredentialProvider.<init>(MSKCredentialProvider.java:104)
    at software.amazon.msk.auth.iam.IAMClientCallbackHandler.lambda$configure$1(IAMClientCallbackHandler.java:54)
    at java.base/java.util.Optional.map(Optional.java:260)
    at software.amazon.msk.auth.iam.IAMClientCallbackHandler.configure(IAMClientCallbackHandler.java:54)
    at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:151)
    ... 35 common frames omitted

I used the below config for the connection :-

security.protocol = SASL_SSL sasl.mechanism = AWS_MSK_IAM sasl.jaas.config = software.amazon.msk.auth.iam.IAMLoginModule required  awsRoleArn=fake-role-arn awsStsRegion=ap-southeast-1; sasl.client.callback.handler.class = software.amazon.msk.auth.iam.IAMClientCallbackHandler

It looks like if awsStsRegion not equal to aws-global, will create the override endpoint with URI.create("sts." + stsRegion + ".amazonaws.com"), and this will cause the NullPointException at at software.amazon.awssdk.utils.Validate.paramNotNull(Validate.java:156) because the override endpoint have no https:// prefix.

Could someone please help to look at this issue?

jvdadda commented 2 months ago

Here is the PR to fix it, it will needs a maintainer review/merge/publish : https://github.com/aws/aws-msk-iam-auth/pull/166

li-wang2 commented 2 months ago

Here is the PR to fix it, it will needs a maintainer review/merge/publish : #166

Great, thanks a lot!

kamlakant commented 2 months ago

Is this issue a regression? Is there any older version of aws-msk-iam-auth where it was working? We are currently stuck because of this. Any workarounds are appreciated.

jvdadda commented 2 months ago

Is this issue a regression? Is there any older version of aws-msk-iam-auth where it was working? We are currently stuck because of this. Any workarounds are appreciated.

@kamlakant Yes, it is coming from SDK V2 migration, with 2.0.x you should not have the problem

tjmatosplanet commented 1 month ago

I´m having this problem with Amazon MSK when using the software.amazon.msk.auth.iam.IAMClientCallbackHandler someone could help on this?

jvdadda commented 1 month ago

I´m having this problem with Amazon MSK when using the software.amazon.msk.auth.iam.IAMClientCallbackHandler someone could help on this?

@tjmatosplanet You can use the 2.0.x instead of 2.1.x until a maintainer publish the fix