IAM Role not working

[christrt9]

Issue submitter TODO list

• [X] I've looked up my issue in FAQ
• [X] I've searched for an already existing issues here
• [X] I've tried running master-labeled docker image and the issue still persists there
• [X] I'm running a supported version of the application which is listed here

Describe the bug (actual behavior)

I run kafka-ui in EC2 with IAM Role but is not working

Caused by: com.amazonaws.SdkClientException: Unable to load AWS credentials from any provider in the chain: [EnvironmentVariableCredentialsProvider: Unable to load AWS credentials from environment variables (AWS_ACCESS_KEY_ID (or AWS_ACCESS_KEY) and AWS_SECRET_KEY (or AWS_SECRET_ACCESS_KEY)), SystemPropertiesCredentialsProvider: Unable to load AWS credentials from Java system properties (aws.accessKeyId and aws.secretKey), WebIdentityTokenCredentialsProvider: You must specify a value for roleArn and roleSessionName, com.amazonaws.auth.profile.ProfileCredentialsProvider@4af8948c: profile file cannot be null, com.amazonaws.auth.EC2ContainerCredentialsProviderWrapper@3af8e25a: Unauthorized (Service: null; Status Code: 401; Error Code: null; Request ID: null; Proxy: null)]

Expected behavior

with with role.

Your installation details

EC2 with docker container version: latest and APP version: https://github.com/provectus/kafka-ui/commit/56fa824

I have this value in docker-compose

      KAFKA_CLUSTERS_0_NAME: Developers-dev
      KAFKA_CLUSTERS_0_PROPERTIES_SECURITY_PROTOCOL: SASL_SSL
      KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS: xxxx
      KAFKA_CLUSTERS_0_PROPERTIES_SASL_MECHANISM: AWS_MSK_IAM
      KAFKA_CLUSTERS_0_PROPERTIES_SASL_CLIENT_CALLBACK_HANDLER_CLASS: 'software.amazon.msk.auth.iam.IAMClientCallbackHandler'
      KAFKA_CLUSTERS_0_PROPERTIES_SASL_JAAS_CONFIG: 'software.amazon.msk.auth.iam.IAMLoginModule required awsRoleArn="arn:aws:iam::xxxx:role/xxx" awsRoleSessionName="kafka-ui"  awsStsRegion="xxx";'

Policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "kafka-cluster:DescribeCluster",
                "kafka-cluster:AlterCluster",
                "kafka-cluster:Connect"
            ],
            "Resource": "arn:aws:kafka:xxxx"
        },
        {
            "Sid": "VisualEditor1",
            "Effect": "Allow",
            "Action": [
                "kafka-cluster:DeleteGroup",
                "kafka-cluster:DescribeCluster",
                "kafka-cluster:ReadData",
                "kafka-cluster:DescribeTopicDynamicConfiguration",
                "kafka-cluster:AlterTopicDynamicConfiguration",
                "kafka-cluster:AlterGroup",
                "kafka-cluster:AlterClusterDynamicConfiguration",
                "kafka-cluster:AlterTopic",
                "kafka-cluster:CreateTopic",
                "kafka-cluster:DescribeTopic",
                "kafka-cluster:AlterCluster",
                "kafka-cluster:DescribeGroup",
                "kafka-cluster:DescribeClusterDynamicConfiguration",
                "kafka-cluster:Connect",
                "kafka-cluster:DeleteTopic",
                "kafka-cluster:WriteData"
            ],
            "Resource": "arn:aws:kafka:xxx*"
        },
        {
            "Sid": "VisualEditor2",
            "Effect": "Allow",
            "Action": [
                "kafka-cluster:AlterGroup",
                "kafka-cluster:DescribeGroup"
            ],
            "Resource": "arn:aws:kafka:xxx*"
        }
    ]
}

Steps to reproduce

Try with IAM Role and this config

Screenshots

No response

Logs

No response

Additional context

No response

[github-actions[bot]]

Hello there r00t9! 👋

Thank you and congratulations 🎉 for opening your very first issue in this project! 💖

In case you want to claim this issue, please comment down below! We will try to get back to you as soon as we can. 👀

[kothapet]

The way I did this is that I attached the role (instance profile) to EC2 directly, and not SASL_JAAS_CONFIG. Then test thru CLI that you can access Msk. If this works then you role is correct. Otherwise you issues with role.

Then run the kafka ui docker.. And it works fine. The following is what I added to docker file

#
# For MSK IAM authentication 
#
kafka.clusters.0.properties.security.protocol: SASL_SSL
kafka.clusters.0.properties.sasl.mechanism: AWS_MSK_IAM
kafka.clusters.0.properties.sasl.client.callback.handler.class: software.amazon.msk.auth.iam.IAMClientCallbackHandler
kafka.clusters.0.properties.sasl.jaas.config: software.amazon.msk.auth.iam.IAMLoginModule required;
#
# For MSK TLS 
#
kafka.clusters.0.properties.ssl.truststore.location: /some-folder/kafka.client.truststore.jks
kafka.clusters.0.properties.ssl.truststore.password: changeit

[Janhavi-Shinde]

@kothapet How did you test access to MSK through CLI? Can you pls provide your commands? I'm using ECS and Fargate tasks to run my UI, and using a MSK cluster, but I'm having issues where the UI isn't retriving complete cluster details, and I suspect connectivity issues. Can you please provide some more info on how you tested access to MSK?

[kothapet]

@Janhavi-Shinde follow the documentation on AWS MSK developer guide. https://docs.aws.amazon.com/msk/latest/developerguide/getting-started.html. step 1-4. If you are using IAM make sure setup the roles correctly and check your Security groups. If you are using some other authentication look at the security authentication and authorization sections.

[Janhavi-Shinde]

@kothapet I followed that same guide to create a provisioned cluster (also created a serverless cluster but I'm trying to connect the provisioned one right now). I've checked the IAM role I am using has very liberal access (e.g kafka-cluster:* and is applied to all my clusters), my security group has sufficient access inbound and outbound, and the UI and MSK cluster is within the same VPC and Subnets also.