Support default authentication methods of the AWS SDK

[aljoshare]

Is your feature request related to a problem? Please describe.

In order to be able to use cross-account IRSA with chained AssumeRole, I need to possibility to set the AWS profile via AWS_PROFILE environment variable. I can inject the AWS configuration but it will not be picked up even if I set the correct profile name. I assume that the aws_sdk_auth field of the used thanos.io/objstore library is set to false by default and there is currently no way to set it to true.

Describe the solution you'd like

I would love to set the aws_sdk_auth field via configuration. For example:

mimir:
  structuredConfig:
    common:
      storage:
        backend: s3
        s3:
          endpoint: s3.eu-central-1.amazonaws.com
          region: eu-central-1
          aws_sdk_auth: true
    blocks_storage:
      s3:
        bucket_name: s3bucket

Describe alternatives you've considered

Additional context

https://github.com/thanos-io/objstore#s3 https://github.com/thanos-io/objstore/blob/d0c43443ecda0d8cffbd4d215569a0dba9ba39ec/providers/s3/s3.go#L226 https://docs.aws.amazon.com/sdk-for-go/api/aws/session/

[aljoshare]

I added the possibility to enable the aws sdk auth and pass it to the thanos objstore library but the problem still exists. I try to assume a role from a different account via role chaining in the ~/.aws/config and it doesn't work. It never passes the sanity check of the ingester. It works in an aws-cli container with the same IRSA and env configuration. Any ideas?

[aljoshare]

I found the issue: By default, the amazon-eks-pod-identity-webhook adds multiple environment variables which take precedence over the ./aws/config and tell the SDK to use the role to assume the actual role instead of the assumed role for calling the S3 bucket. This annotation did the trick:

annotations:
    # optional: A comma-separated list of initContainers and container names
    #   to skip adding volumes and environment variables
    eks.amazonaws.com/skip-containers: "ingester"

I will create a PR which allows to use the default credential provider chain of the AWS SDK, which supports the use of role chaining in the config file.