Aiven-Open / tiered-storage-for-apache-kafka

RemoteStorageManager for Apache Kafka® Tiered Storage
Apache License 2.0
91 stars 19 forks source link

Add AWS STS to allow using AWS creds from service account #511

Closed AagonP closed 6 months ago

AagonP commented 6 months ago

What is currently missing?

Using

rsm.config.aws.credentials.provider.class=software.amazon.awssdk.auth.credentials.WebIdentityTokenFileCredentialsProvider

deals with reading the environment variables that are injected to EKS pod and assuming the role with STS.

Kafka logs show:

WARN To use web identity tokens, the 'sts' service module must be on the class path. (software.amazon.awssdk.auth.credentials.internal.WebIdentityCredentialsUtils) and authentication failed.

How could this be improved?

Internally, WebIdentityTokenFileCredentialsProvider will call AWS STS so you will need to add it to the dependencies list.

Is this a feature you would work on yourself?

ivanyu commented 6 months ago

Hi @AagonP Could you please try adding the STS JAR to the plugin classpath and see if this works for you?

ivanyu commented 6 months ago

I'm not sure we should include it as a dependency in the distribution, but we at least should document this.

AagonP commented 6 months ago

Hi @ivanyu , yes I did add STS JAR on my own and it's working fine with the configuration I provided above.

ivanyu commented 6 months ago

Probably it makes sense to include STS in the distro after all, considering that this is a popular credentials provider. Made a PR

AagonP commented 6 months ago

Thanks! Cheers