search

aws / aws-msk-iam-auth

Enables developers to use AWS Identity and Access Management (IAM) to connect to their Amazon Managed Streaming for Apache Kafka (Amazon MSK) clusters.
Apache License 2.0
137 stars 65 forks source link

Cannot connect to publicly accessible MSK using IAM credentials #172

Open abhishek-parative opened 1 month ago

abhishek-parative commented 1 month ago

I am following the instructions set here https://docs.aws.amazon.com/msk/latest/developerguide/iam-access-control.html and here https://docs.aws.amazon.com/msk/latest/developerguide/create-topic.html to access my MSK instance.

However, my IAM credentials are not being picked up.

This is what my ~/.aws/config file looks like:

[default]
sso_session = FOOBAR
sso_account_id = 0123456789012
sso_role_name = AdministratorAccess
region = us-west-2
output = json

[profile sample]
sso_session = FOOBAR
sso_account_id = 0123456789012
sso_role_name = AdministratorAccess
sso_region = us-west-2
region = us-west-2
output = json
sso_start_url = foobar_url
sso_registration_scopes = sso:account:access

This is what my client.properties looks like

security.protocol=SASL_SSL
sasl.mechanism=AWS_MSK_IAM
sasl.jaas.config = software.amazon.msk.auth.iam.IAMLoginModule required awsProfileName="sample";
sasl.client.callback.handler.class=software.amazon.msk.auth.iam.IAMClientCallbackHandler

I also added the aws-msk-iam-auth-1.1.1-all.jar to the kafka/libs folder.

Error Trace:

[2024-05-10 11:23:32,005] ERROR [AdminClient clientId=adminclient-1] Connection to node -1 (REDACTED) failed authentication due to: An error: (java.security.PrivilegedActionException: javax.security.sasl.SaslException: Failed to find AWS IAM Credentials [Caused by aws_msk_iam_auth_shadow.com.amazonaws.SdkClientException: Unable to load AWS credentials from any provider in the chain: [software.amazon.msk.auth.iam.internals.EnhancedProfileCredentialsProvider@1149414c: The SSO session associated with this profile has expired or is otherwise invalid. To refresh this SSO session run aws sso login with the corresponding profile., aws_msk_iam_auth_shadow.com.amazonaws.auth.AWSCredentialsProviderChain@581c9ec8: Unable to load AWS credentials from any provider in the chain: [EnvironmentVariableCredentialsProvider: Unable to load AWS credentials from environment variables (AWS_ACCESS_KEY_ID (or AWS_ACCESS_KEY) and AWS_SECRET_KEY (or AWS_SECRET_ACCESS_KEY)), SystemPropertiesCredentialsProvider: Unable to load AWS credentials from Java system properties (aws.accessKeyId and aws.secretKey), WebIdentityTokenCredentialsProvider: You must specify a value for roleArn and roleSessionName, software.amazon.msk.auth.iam.internals.EnhancedProfileCredentialsProvider@6a2a7c1c: The SSO session associated with this profile has expired or is otherwise invalid. To refresh this SSO session run aws sso login with the corresponding profile., aws_msk_iam_auth_shadow.com.amazonaws.auth.EC2ContainerCredentialsProviderWrapper@60cc5251: Failed to connect to service endpoint: ]]]) occurred when evaluating SASL token received from the Kafka Broker. Kafka Client will go to AUTHENTICATION_FAILED state. (org.apache.kafka.clients.NetworkClient)
...
Caused by: javax.security.sasl.SaslException: Failed to find AWS IAM Credentials [Caused by aws_msk_iam_auth_shadow.com.amazonaws.SdkClientException: Unable to load AWS credentials from any provider in the chain: [software.amazon.msk.auth.iam.internals.EnhancedProfileCredentialsProvider@1149414c: The SSO session associated with this profile has expired or is otherwise invalid. To refresh this SSO session run aws sso login with the corresponding profile., aws_msk_iam_auth_shadow.com.amazonaws.auth.AWSCredentialsProviderChain@581c9ec8: Unable to load AWS credentials from any provider in the chain: [EnvironmentVariableCredentialsProvider: Unable to load AWS credentials from environment variables (AWS_ACCESS_KEY_ID (or AWS_ACCESS_KEY) and AWS_SECRET_KEY (or AWS_SECRET_ACCESS_KEY)), SystemPropertiesCredentialsProvider: Unable to load AWS credentials from Java system properties (aws.accessKeyId and aws.secretKey), WebIdentityTokenCredentialsProvider: You must specify a value for roleArn and roleSessionName, software.amazon.msk.auth.iam.internals.EnhancedProfileCredentialsProvider@6a2a7c1c: The SSO session associated with this profile has expired or is otherwise invalid. To refresh this SSO session run aws sso login with the corresponding profile., aws_msk_iam_auth_shadow.com.amazonaws.auth.EC2ContainerCredentialsProviderWrapper@60cc5251: Failed to connect to service endpoint: ]]]
    at software.amazon.msk.auth.iam.internals.IAMSaslClient.generateClientMessage(IAMSaslClient.java:148)
    at software.amazon.msk.auth.iam.internals.IAMSaslClient.evaluateChallenge(IAMSaslClient.java:95)
    at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator.lambda$createSaslToken$1(SaslClientAuthenticator.java:534)
    at java.base/java.security.AccessController.doPrivileged(Native Method)
    at java.base/javax.security.auth.Subject.doAs(Subject.java:423)
    at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator.createSaslToken(SaslClientAuthenticator.java:534)
    at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator.sendSaslClientToken(SaslClientAuthenticator.java:433)
    at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator.sendInitialToken(SaslClientAuthenticator.java:332)
    at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator.authenticate(SaslClientAuthenticator.java:273)
    at org.apache.kafka.common.network.KafkaChannel.prepare(KafkaChannel.java:181)
    at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:543)
    at org.apache.kafka.common.network.Selector.poll(Selector.java:481)
    at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:571)
    at org.apache.kafka.clients.admin.KafkaAdminClient$AdminClientRunnable.processRequests(KafkaAdminClient.java:1413)
    at org.apache.kafka.clients.admin.KafkaAdminClient$AdminClientRunnable.run(KafkaAdminClient.java:1344)
    at java.base/java.lang.Thread.run(Thread.java:829)

I reauthenticated using aws sso login --profile sample and no dice

Cyberness commented 1 month ago

Same problem with MSF on version 2.1.0. Rollback to version 1.1.9 solved the issue.

abhishek-parative commented 1 month ago

When I use the aws-msk-iam-auth-1.1.9-all.jar, I get the following error: To use SSO OIDC related properties in the 'sample' profile, the 'ssooidc' service module must be on the class path.

Stack trace:

[2024-05-10 14:38:15,817] WARN Exception loading credentials. Retry Attempts: 0 (software.amazon.msk.auth.iam.internals.MSKCredentialProvider)
aws_msk_iam_auth_shadow.com.amazonaws.SdkClientException: Unable to load AWS credentials from any provider in the chain: [software.amazon.msk.auth.iam.internals.EnhancedProfileCredentialsProvider@713f1d49:

**To use SSO OIDC related properties in the 'sample' profile, the 'ssooidc' service module must be on the class path.**

aws_msk_iam_auth_shadow.com.amazonaws.auth.AWSCredentialsProviderChain@460ddbd6: Unable to load AWS credentials from any provider in the chain: [EnvironmentVariableCredentialsProvider: Unable to load AWS credentials from environment variables (AWS_ACCESS_KEY_ID (or AWS_ACCESS_KEY) and AWS_SECRET_KEY (or AWS_SECRET_ACCESS_KEY)), SystemPropertiesCredentialsProvider: Unable to load AWS credentials from Java system properties (aws.accessKeyId and aws.secretKey), WebIdentityTokenCredentialsProvider: You must specify a value for roleArn and roleSessionName, software.amazon.msk.auth.iam.internals.EnhancedProfileCredentialsProvider@6c1c84bc: To use SSO OIDC related properties in the 'default' profile, the 'ssooidc' service module must be on the class path., aws_msk_iam_auth_shadow.com.amazonaws.auth.EC2ContainerCredentialsProviderWrapper@71f1a25b: Failed to connect to service endpoint: ]]
    at aws_msk_iam_auth_shadow.com.amazonaws.auth.AWSCredentialsProviderChain.getCredentials(AWSCredentialsProviderChain.java:142)
    at software.amazon.msk.auth.iam.internals.MSKCredentialProvider.loadCredentialsWithRetry(MSKCredentialProvider.java:154)
    at software.amazon.msk.auth.iam.internals.MSKCredentialProvider.getCredentials(MSKCredentialProvider.java:141)
    at software.amazon.msk.auth.iam.IAMClientCallbackHandler.handleCallback(IAMClientCallbackHandler.java:100)
    at software.amazon.msk.auth.iam.IAMClientCallbackHandler.handle(IAMClientCallbackHandler.java:77)
    ...
sbernardo commented 3 weeks ago

I had same problem, I found this https://github.com/aws/aws-sdk-java-v2/issues/2961#issuecomment-1012991545 , seems need to update some libraries.

This solved for me as well:

Same problem with MSF on version 2.1.0. Rollback to version 1.1.9 solved the issue.

But maybe into version 2.1.1 updating aws libraries solves as well