Use/Support the default environment variables for AWS SDKs

[danielloader]

Thanks for making this repository and project! Looks useful. I went to go and run it on EKS and utilise S3 (via a VPC endpoint) and immediately realised my usual ways of passing AWS credentials to pods wouldn't work.

It would be nice if the code could support the default environment variables AWS CLI/SDKs can use without configuration if available in the shell:

https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-envvars.html#envvars-list

As an additional follow on question:

Will this support either IRSA or Pod Identity mappings?

Both need to be able to support a flavour of web token rather than the credentials as environment variables.

Edit: Noticed you're using the minio client library not the AWS SDK so this might all be moot.

[LouisHaftmann]

We could try to detect whether the cache server is running on AWS and auto-configure the storage driver. I personally haven't used AWS so it might require some tinkering to set up a testing environment and get it to work.

In the meantime - have you tried setting the environment variables manually?

[danielloader]

That's worked absolutely fine, if anything I was being a bit needy expecting it to operate like all my other s3 connecting applications in EKS.

[LouisHaftmann]

We could probably add some sort of auto-configuration but it would introduce more complexity and points of failure.

What do you think @DrJume ?

[danielloader]

I'd say if you can't support IRSA or EKS Pod Identity authentication methods the pain of remapping the environment variables for legacy authentication in EKS isn't worth the effort on your team. Thanks for considering it.

The only way this would make sense from a maintenance perspective would be using the aws sdk and s3 client they provide which handles all the authentication options out the box with the default client.

https://docs.aws.amazon.com/sdk-for-javascript/v3/developer-guide/setting-credentials-node.html#credchain see the cascading chain of various methods it checks when there's no explicit configuration.

Edit: one caveat to be aware of https://stackoverflow.com/a/75374148

[LouisHaftmann]

I'll close this issue for now as this is currently not a high priority for us. We will probably come back to this in future releases.

[DrJume]

We could probably add some sort of auto-configuration but it would introduce more complexity and points of failure.

What do you think @DrJume ?

Yeah, we don't have much expertise with AWS so we can't estimate how to implement it easily and how big the benefit would be.