search

aws / aws-msk-iam-auth

Enables developers to use AWS Identity and Access Management (IAM) to connect to their Amazon Managed Streaming for Apache Kafka (Amazon MSK) clusters.
Apache License 2.0
145 stars 67 forks source link

NullPointException when building StsClient with overrideEndpoint #165

Closed li-wang2 closed 4 months ago

li-wang2 commented 7 months ago

Hi team,

I got the following error when connecting Kafka with IAM authentication by using version 2.1.0

  ...
Caused by: org.apache.kafka.common.KafkaException: java.lang.NullPointerException: The URI scheme of endpointOverride must not be null.
    at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:184)
    at org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:192)
    at org.apache.kafka.common.network.ChannelBuilders.clientChannelBuilder(ChannelBuilders.java:81)
    at org.apache.kafka.clients.ClientUtils.createChannelBuilder(ClientUtils.java:117)
    at org.apache.kafka.clients.ClientUtils.createNetworkClient(ClientUtils.java:217)
    ... 31 common frames omitted
Caused by: java.lang.NullPointerException: The URI scheme of endpointOverride must not be null.
    at software.amazon.awssdk.utils.Validate.paramNotNull(Validate.java:156)
    at software.amazon.awssdk.core.client.builder.SdkDefaultClientBuilder.endpointOverride(SdkDefaultClientBuilder.java:502)
    at software.amazon.msk.auth.iam.internals.MSKCredentialProvider$ProviderBuilder.getStsClientBuilder(MSKCredentialProvider.java:289)
    at software.amazon.msk.auth.iam.internals.MSKCredentialProvider$ProviderBuilder.createSTSRoleCredentialProvider(MSKCredentialProvider.java:342)
    at software.amazon.msk.auth.iam.internals.MSKCredentialProvider$ProviderBuilder.lambda$getStsRoleProvider$4(MSKCredentialProvider.java:330)
    at java.base/java.util.Optional.map(Optional.java:260)
    at software.amazon.msk.auth.iam.internals.MSKCredentialProvider$ProviderBuilder.getStsRoleProvider(MSKCredentialProvider.java:307)
    at software.amazon.msk.auth.iam.internals.MSKCredentialProvider$ProviderBuilder.getProviders(MSKCredentialProvider.java:254)
    at software.amazon.msk.auth.iam.internals.MSKCredentialProvider.<init>(MSKCredentialProvider.java:108)
    at software.amazon.msk.auth.iam.internals.MSKCredentialProvider.<init>(MSKCredentialProvider.java:104)
    at software.amazon.msk.auth.iam.IAMClientCallbackHandler.lambda$configure$1(IAMClientCallbackHandler.java:54)
    at java.base/java.util.Optional.map(Optional.java:260)
    at software.amazon.msk.auth.iam.IAMClientCallbackHandler.configure(IAMClientCallbackHandler.java:54)
    at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:151)
    ... 35 common frames omitted

I used the below config for the connection :-

security.protocol = SASL_SSL sasl.mechanism = AWS_MSK_IAM sasl.jaas.config = software.amazon.msk.auth.iam.IAMLoginModule required  awsRoleArn=fake-role-arn awsStsRegion=ap-southeast-1; sasl.client.callback.handler.class = software.amazon.msk.auth.iam.IAMClientCallbackHandler

It looks like if awsStsRegion not equal to aws-global, will create the override endpoint with URI.create("sts." + stsRegion + ".amazonaws.com"), and this will cause the NullPointException at at software.amazon.awssdk.utils.Validate.paramNotNull(Validate.java:156) because the override endpoint have no https:// prefix.

Could someone please help to look at this issue?

jvdadda commented 7 months ago

Here is the PR to fix it, it will needs a maintainer review/merge/publish : https://github.com/aws/aws-msk-iam-auth/pull/166

li-wang2 commented 7 months ago

Here is the PR to fix it, it will needs a maintainer review/merge/publish : #166

Great, thanks a lot!

kamlakant commented 7 months ago

Is this issue a regression? Is there any older version of aws-msk-iam-auth where it was working? We are currently stuck because of this. Any workarounds are appreciated.

jvdadda commented 7 months ago

Is this issue a regression? Is there any older version of aws-msk-iam-auth where it was working? We are currently stuck because of this. Any workarounds are appreciated.

@kamlakant Yes, it is coming from SDK V2 migration, with 2.0.x you should not have the problem

tjmatosplanet commented 6 months ago

I´m having this problem with Amazon MSK when using the software.amazon.msk.auth.iam.IAMClientCallbackHandler someone could help on this?

jvdadda commented 6 months ago

I´m having this problem with Amazon MSK when using the software.amazon.msk.auth.iam.IAMClientCallbackHandler someone could help on this?

@tjmatosplanet You can use the 2.0.x instead of 2.1.x until a maintainer publish the fix

NikolayMetchev commented 4 months ago

Any news here?

jvdadda commented 4 months ago

@NikolayMetchev The fix is included inside the last version (2.2.0), the problem should be no more here.

@sidyag I think you can close the issue

guanlin84 commented 4 months ago

Closing the issue as it's been fixed in 2.2.0.

github-actions[bot] commented 4 months ago

This issue is now closed. Comments on closed issues are hard for our team to see. If you need more assistance, please open a new issue that references this one.