search

aws / aws-msk-iam-auth

Enables developers to use AWS Identity and Access Management (IAM) to connect to their Amazon Managed Streaming for Apache Kafka (Amazon MSK) clusters.
Apache License 2.0
137 stars 65 forks source link

AWS MSK Kafka Connect issues #81

Closed JussiLem closed 1 year ago

JussiLem commented 1 year ago

Hi,

I've been trying to use a custom connector with Kafka Connect working by using the AWS console.

Not sure if related to IAM Role and policies, but I'm getting this error on the connectors log stream:

[Worker-0016cbeff960e3da2] Caused by: javax.security.sasl.SaslException: Failed to find AWS IAM Credentials [Caused by aws_msk_iam_auth_shadow.com.amazonaws.SdkClientException: Unable to load AWS credentials from any provider in the chain: [aws_msk_iam_auth_shadow.com.amazonaws.auth.AWSCredentialsProviderChain@707b13a: Unable to load AWS credentials from any provider in the chain: [EnvironmentVariableCredentialsProvider: Unable to load AWS credentials from environment variables (AWS_ACCESS_KEY_ID (or AWS_ACCESS_KEY) and AWS_SECRET_KEY (or AWS_SECRET_ACCESS_KEY)), SystemPropertiesCredentialsProvider: Unable to load AWS credentials from Java system properties (aws.accessKeyId and aws.secretKey), WebIdentityTokenCredentialsProvider: You must specify a value for roleArn and roleSessionName, software.amazon.msk.auth.iam.internals.EnhancedProfileCredentialsProvider@7adb92bf: Profile file contained no credentials for profile 'default': ProfileFile(profiles=[]), aws_msk_iam_auth_shadow.com.amazonaws.auth.EC2ContainerCredentialsProviderWrapper@41f935e: Internal Server Error (Service: null; Status Code: 500; Error Code: null; Request ID: null; Proxy: null)]]]

Any ideas how to proceed?

Thanks!

dude0001 commented 1 year ago

@JussiLem what version of aws-msk-iam-auth are you using? Where are you running Kafka Connect? Can you share your connector and worker configurations?

JussiLem commented 1 year ago

Hi,

Thanks for your response. I actually got it working already, just forgot to update it. I'm using the managed MSK Connect and in the end this issue was related to IAM Role and policy. Also for some reason it wasn't clear for me that the aws-msk-iam-auth library was handled by the managed Connect so that created some confusion in the beginning.

aljoshare commented 9 months ago

@JussiLem I know it's closed but can you tell me what the solution was? I have exactly the same problem and I'm running out of ideas.