MSK: Pod Identity support

Issue submitter TODO list

[X] I've looked up my issue in FAQ
[X] I've searched for an already existing issues here
[X] I've tried running main-labeled docker image and the issue still persists there
[X] I'm running a supported version of the application which is listed here

Describe the bug (actual behavior)

Hi Team,

Earlier I reported this issue - https://github.com/kafbat/kafka-ui/issues/287. Though we didnt get a chance t use MSK after that, but now I have deployed akfka ui in AWS Rosa but it seems the pod identity issue is not resolved.

I can still see it is giving the same error

Unable to load credentials from any of the providers in the chain AwsCredentialsProviderChain(credentialsProviders=[AwsCredentialsProviderChain(credentialsProviders=[EnvironmentVariableCredentialsProvider(), SystemPropertyCredentialsProvider(), WebIdentityTokenCredentialsProvider(), ProfileCredentialsProvider(profileName=default, profileFile=ProfileFile(sections=[])), ContainerCredentialsProvider()])]) : [AwsCredentialsProviderChain(credentialsProviders=[EnvironmentVariableCredentialsProvider(), SystemPropertyCredentialsProvider(), WebIdentityTokenCredentialsProvider(), ProfileCredentialsProvider(profileName=default, profileFile=ProfileFile(sections=[])), ContainerCredentialsProvider()]): Unable to load credentials from any of the providers in the chain AwsCredentialsProviderChain(credentialsProviders=[EnvironmentVariableCredentialsProvider(), SystemPropertyCredentialsProvider(), WebIdentityTokenCredentialsProvider(), ProfileCredentialsProvider(profileName=default, profileFile=ProfileFile(sections=[])), ContainerCredentialsProvider()]) : [EnvironmentVariableCredentialsProvider(): Unable to load credentials from system settings. Access key must be specified either via environment variable (AWS_ACCESS_KEY_ID) or system property (aws.accessKeyId)., SystemPropertyCredentialsProvider(): Unable to load credentials from system settings. Access key must be specified either via environment variable (AWS_ACCESS_KEY_ID) or system property (aws.accessKeyId)., WebIdentityTokenCredentialsProvider(): Not authorized to perform sts:AssumeRoleWithWebIdentity (Service: Sts, Status Code: 403, Request ID: 81e6f31f-2ca4-XXXXXXXacca), ProfileCredentialsProvider(profileName=default, profileFile=ProfileFile(sections=[])): Profile file contained no credentials for profile 'default': ProfileFile(sections=[]), ContainerCredentialsProvider(): Cannot fetch credentials from container - neither AWS_CONTAINER_CREDENTIALS_FULL_URI or AWS_CONTAINER_CREDENTIALS_RELATIVE_URI environment variables are set.]] at software.amazon.awssdk.core.exception.SdkClientException$BuilderImpl.build(SdkClientException.java:111) at software.amazon.awssdk.auth.credentials.AwsCredentialsProviderChain.resolveCredentials(AwsCredentialsProviderChain.java:130) at software.amazon.msk.auth.iam.internals.MSKCredentialProvider.loadCredentialsWithRetry(MSKCredentialProvider.java:175) at software.amazon.msk.auth.iam.internals.MSKCredentialProvider.resolveCredentials(MSKCredentialProvider.java:162) at software.amazon.msk.auth.iam.IAMClientCallbackHandler.handleCallback(IAMClientCallbackHandler.java:99) at software.amazon.msk.auth.iam.IAMClientCallbackHandler.handle(IAMClientCallbackHandler.java:77) at software.amazon.msk.auth.iam.internals.IAMSaslClient.generateClientMessage(IAMSaslClient.java:139) at software.amazon.msk.auth.iam.internals.IAMSaslClient.evaluateChallenge(IAMSaslClient.java:96) at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator.lambda$createSaslToken$1(SaslClientAuthenticator.java:534)

Expected behavior

It should able to connect MSK using Multi vpc cross account IAM role

Your installation details

Pull latest image created a helm chat deployed it in out Rosa env

Steps to reproduce

Create MSK Enable Multi vpc for IAM update cluster policy in MSK and Client end (Rosa) Pull latest image created a helm chat and provided MSK details deployed it in AWS Rosa env

Screenshots

Logs

Unable to load credentials from any of the providers in the chain AwsCredentialsProviderChain(credentialsProviders=[AwsCredentialsProviderChain(credentialsProviders=[EnvironmentVariableCredentialsProvider(), SystemPropertyCredentialsProvider(), WebIdentityTokenCredentialsProvider(), ProfileCredentialsProvider(profileName=default, profileFile=ProfileFile(sections=[])), ContainerCredentialsProvider()])]) : [AwsCredentialsProviderChain(credentialsProviders=[EnvironmentVariableCredentialsProvider(), SystemPropertyCredentialsProvider(), WebIdentityTokenCredentialsProvider(), ProfileCredentialsProvider(profileName=default, profileFile=ProfileFile(sections=[])), ContainerCredentialsProvider()]): Unable to load credentials from any of the providers in the chain AwsCredentialsProviderChain(credentialsProviders=[EnvironmentVariableCredentialsProvider(), SystemPropertyCredentialsProvider(), WebIdentityTokenCredentialsProvider(), ProfileCredentialsProvider(profileName=default, profileFile=ProfileFile(sections=[])), ContainerCredentialsProvider()]) : [EnvironmentVariableCredentialsProvider(): Unable to load credentials from system settings. Access key must be specified either via environment variable (AWS_ACCESS_KEY_ID) or system property (aws.accessKeyId)., SystemPropertyCredentialsProvider(): Unable to load credentials from system settings. Access key must be specified either via environment variable (AWS_ACCESS_KEY_ID) or system property (aws.accessKeyId)., WebIdentityTokenCredentialsProvider(): Not authorized to perform sts:AssumeRoleWithWebIdentity (Service: Sts, Status Code: 403, Request ID: 81e6f31f-2ca4-40f6-XXXXXXX69f9acca), ProfileCredentialsProvider(profileName=default, profileFile=ProfileFile(sections=[])): Profile file contained no credentials for profile 'default': ProfileFile(sections=[]), ContainerCredentialsProvider(): Cannot fetch credentials from container - neither AWS_CONTAINER_CREDENTIALS_FULL_URI or AWS_CONTAINER_CREDENTIALS_RELATIVE_URI environment variables are set.]] at software.amazon.awssdk.core.exception.SdkClientException$BuilderImpl.build(SdkClientException.java:111) at software.amazon.awssdk.auth.credentials.AwsCredentialsProviderChain.resolveCredentials(AwsCredentialsProviderChain.java:130) at software.amazon.msk.auth.iam.internals.MSKCredentialProvider.loadCredentialsWithRetry(MSKCredentialProvider.java:175) at software.amazon.msk.auth.iam.internals.MSKCredentialProvider.resolveCredentials(MSKCredentialProvider.java:162) at software.amazon.msk.auth.iam.IAMClientCallbackHandler.handleCallback(IAMClientCallbackHandler.java:99) at software.amazon.msk.auth.iam.IAMClientCallbackHandler.handle(IAMClientCallbackHandler.java:77) at software.amazon.msk.auth.iam.internals.IAMSaslClient.generateClientMessage(IAMSaslClient.java:139) at software.amazon.msk.auth.iam.internals.IAMSaslClient.evaluateChallenge(IAMSaslClient.java:96) at org.apache.kafka.common.security.authenticator.SaslClientAuthenticator.lambda$createSaslToken$1(SaslClientAuthenticator.java:534)

kafbat / kafka-ui