Enables developers to use AWS Identity and Access Management (IAM) to connect to their Amazon Managed Streaming for Apache Kafka (Amazon MSK) clusters.
This PR adds support to pass session credentials to an STS role credential provider in addition to static AWS credentials. This allows to receive temporary credentials using STS and then pass them along. This can be useful in role-chaining scenarios, where the MSKCredentialProvider is known to be used for a short amount of time only.
Background
In Imply Polaris, we provide users with the ability to set up authentication to their MSK clusters using a user-provided ARN of an IAM role. Before starting ingestion from that MSK cluster to Druid, we want to give users the ability to "test" their connection and credentials. To do so, we create a short-lived KafkaConsumer which we pass a set of client configuration values (as outlined in this project).
Due to our security architecture, we don't want the service running the above logic to be able to assume arbitrary roles. Instead, the service will first assume another role (in fact, the role that the Druid cluster uses), and only then assumes the user-provided role. To do this role-chaining, we need to be able to pass the short-lived session credentials we received from assuming the initial role to the MSKCredentialProvider.
The complete flow would then be as follows:
Use STS client to assume role A, get temporary credentials
Create a KafkaConsumer with configuration properties including a SASL config containing those temporary credentials. For example, if the credentials come from stsClient.assumeRole(roleRequest).getCredentials(), pseudo-code for setting the SASL config could look like this:
This PR adds support to pass session credentials to an STS role credential provider in addition to static AWS credentials. This allows to receive temporary credentials using STS and then pass them along. This can be useful in role-chaining scenarios, where the MSKCredentialProvider is known to be used for a short amount of time only.
Background
In Imply Polaris, we provide users with the ability to set up authentication to their MSK clusters using a user-provided ARN of an IAM role. Before starting ingestion from that MSK cluster to Druid, we want to give users the ability to "test" their connection and credentials. To do so, we create a short-lived KafkaConsumer which we pass a set of client configuration values (as outlined in this project).
Due to our security architecture, we don't want the service running the above logic to be able to assume arbitrary roles. Instead, the service will first assume another role (in fact, the role that the Druid cluster uses), and only then assumes the user-provided role. To do this role-chaining, we need to be able to pass the short-lived session credentials we received from assuming the initial role to the MSKCredentialProvider.
The complete flow would then be as follows:
Create a KafkaConsumer with configuration properties including a SASL config containing those temporary credentials. For example, if the
credentialscome fromstsClient.assumeRole(roleRequest).getCredentials(), pseudo-code for setting the SASL config could look like this:Resolves #111
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.