Is this library compatible with IAM Roles for Service Accounts?

[yortch]

We are leveraging IRSA (IAM Roles for Service Accounts - https://docs.aws.amazon.com/eks/latest/userguide/create-service-account-iam-policy-and-role.html) feature to connect from Red Hat OpenShift cluster, which is effectively a Kubernetes cluster. We have been able to connect to ASK MSK cluster using PLAINTEXT and TLS. We are now attempting to connect to MSK using IAM Access Control. We have configured our role, policy and required properties (as documented here: https://docs.aws.amazon.com/msk/latest/developerguide/iam-access-control.html), however we have been unable to connect and are getting the "access denied" error below. Here is a snippet of the relevant error. We would like to confirm that this library is compatible with "IRSA":

2021-07-22 17:56:45,021 FINE  [com.ama.aut.AWS4Signer] (vert.x-kafka-consumer-thread-0) Generating a new signing key as the signing key not available in the cache for the date 1626912000000
2021-07-22 17:56:45,022 DEBUG [sof.ama.msk.aut.iam.int.IAMSaslClient] (vert.x-kafka-consumer-thread-0) Setting SASL/AWS_MSK_IAM client state to RECEIVE_SERVER_RESPONSE
2021-07-22 17:56:45,022 DEBUG [sof.ama.msk.aut.iam.int.IAMSaslClient] (vert.x-kafka-consumer-thread-0) State RECEIVE_SERVER_RESPONSE at end of evaluating challenge
2021-07-22 17:56:45,022 DEBUG [org.apa.kaf.com.sec.aut.SaslClientAuthenticator] (vert.x-kafka-consumer-thread-0) [Consumer clientId=kafka-consumer-command, groupId=service] Set SASL client state to INTERMEDIATE
2021-07-22 17:56:45,808 DEBUG [org.apa.kaf.com.sec.aut.SaslClientAuthenticator] (kafka-producer-network-thread | kafka-producer-event) [Producer clientId=kafka-producer-event] Set SASL client state to FAILED
2021-07-22 17:56:45,808 INFO  [org.apa.kaf.com.net.Selector] (kafka-producer-network-thread | kafka-producer-event) [Producer clientId=kafka-producer-event] Failed authentication with b-3.demo-clus.52zv2x.c1.kafka.us-east-1.amazonaws.com/10.112.44.165 ([210a6cac-9307-4532-a563-b24b531f1b89]: Access denied)
2021-07-22 17:56:45,809 DEBUG [org.apa.kaf.cli.NetworkClient] (kafka-producer-network-thread | kafka-producer-event) [Producer clientId=kafka-producer-event] Node -1 disconnected.
2021-07-22 17:56:45,809 ERROR [org.apa.kaf.cli.NetworkClient] (kafka-producer-network-thread | kafka-producer-event) [Producer clientId=kafka-producer-event] Connection to node -1 (b-3.demo-clus.52zv2x.c1.kafka.us-east-1.amazonaws.com/10.112.44.165:9098) failed authentication due to: [210a6cac-9307-4532-a563-b24b531f1b89]: Access denied

Thanks in advanced, Jorge

[sayantacC]

@yortch IAM access control should work with IAM Roles for Service Accounts, though I have not tested that scenario myself. Could you help answer these questions:

• Are you using the aws-msk-iam-auth library as a uber jar on you classpath or as a dependency built into your application ?
• Which version of the library are you using ? 1.0 or 1.1.0
• Could you please paste the client configuration ?
• Would it be possible for you to provide or describe the role(s) and policies you are using ?

[yortch]

Thank you for responding. This issue turned out to be an issue caused by the AWS boundary policy being used. Since the kafka-cluster:* permissions were introduced back in May, they were not added in the AWS account's boundary policy. After they were added, we were able to successfully connect to MSK via IAM using IRSA. For reference here are the answers to your questions:

• Dependency versions added (via POM):

  <dependency>
    <groupId>software.amazon.msk</groupId>
    <artifactId>aws-msk-iam-auth</artifactId>
    <version>1.1.0</version>
  </dependency>
  <dependency>
    <groupId>com.amazonaws</groupId>
    <artifactId>aws-java-sdk-sts</artifactId>
    <version>1.12.28</version>
  </dependency>

• Kafka client properties:

  security.protocol=SASL_SSL
  sasl.mechanism=AWS_MSK_IAM
  sasl.jaas.config=software.amazon.msk.auth.iam.IAMLoginModule required;
  sasl.client.callback.handler.class=software.amazon.msk.auth.iam.IAMClientCallbackHandler