AWSCredentials support for AssumeRoleWithWebIdentity mechanism

[jrevels]

I'm trying to use AWS.jl from within a K8s pod that is running with a IAM-role-linked ServiceAccount (see also: https://aws.amazon.com/blogs/opensource/introducing-fine-grained-iam-roles-service-accounts/). Via this mechanism, it's super easy to provision credentials to an individual pod so that commands in the pod are automatically run via the specified role (basically similar to EC2 instance-attached roles, but at a per-pod level). A bunch of the other SDKs already support this out-of-the-box without any required configuration.

Is there a way to for us to use AWS.jl with this mechanism? Naively, it seems like commands fail with permissions errors; inspecting more deeply, it looks like constructing a new AWSCredentials instance skips past the pod IAM role settings and just grabs the underlying node's role (i.e. the underlying EC2 instance's attached role). Reading the AWSCredentials code, it looks like it doesn't automatically read either the AWS_WEB_IDENTITY_TOKEN_FILE environmental variable or the web_identity_token_file setting from ~/.aws/config when searching for credentials.

If AWS.jl adds support for this (and/or if there's a nice workaround we can use), Beacon Biosignals can start using AWS.jl, which would be a big win for us 🙂 We're currently locked to using PyCall w/ boto3 because of this mechanism, which can have annoying interactions with async Julia code.

additional references:

• https://docs.aws.amazon.com/credref/latest/refdocs/setting-global-web_identity_token_file.html
• https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-role.html#cli-configure-role-oidc
• https://github.com/aws/aws-sdk-go/issues/3101 (a similiar issue for the AWS Go SDK)

[SimonDanisch]

It looks like we need to add and properly integrate this credential function:

function credentials_from_webtoken(role_arn, role_session)
    token = read(ENV["AWS_WEB_IDENTITY_TOKEN_FILE"], String)
    result = assume_role_with_web_identity(role_arn, role_session, token)
    role_creds = result["AssumeRoleWithWebIdentityResult"]["Credentials"]
     return AWSCredentials(
        role_creds["AccessKeyId"],
        role_creds["SecretAccessKey"],
        role_creds["SessionToken"];
        expiry=DateTime(rstrip(role_creds["Expiration"], 'Z')),
        renew=()->credentials_from_webtoken(role_arn, role_session)
    )
end

[jrevels]

replacing arguments with appropriate ENV vars:

function credentials_from_webtoken()
    token = read(ENV["AWS_WEB_IDENTITY_TOKEN_FILE"], String)
    result = assume_role_with_web_identity(ENV["AWS_ROLE_ARN"], ENV["AWS_ROLE_SESSION_NAME"], token)
    role_creds = result["AssumeRoleWithWebIdentityResult"]["Credentials"]
     return AWSCredentials(
        role_creds["AccessKeyId"],
        role_creds["SecretAccessKey"],
        role_creds["SessionToken"];
        expiry=DateTime(rstrip(role_creds["Expiration"], 'Z')),
        renew=credentials_from_webtoken
    )
end

[mattBrzezinski]

Yes this is definitely something I can add in, the credential code is based roughly off of the boto3 specs.

I should be able to add this in some time this week/weekend and make a new release.

[SimonDanisch]

Bump :) Anything we can do to help?

[mattBrzezinski]

Sorry I've just been swamped with non-open source work and haven't even had time to look at this yet. This functionality can be added in here, I think the best place would be after the dot_aws_config() function.

If you'd like to make the merge request for it, that'd be lovely (along side a test, some examples can be found here). Otherwise I can get around to this at a later date.

[jrevels]

We can try to devote some resources to this next week :)

[mattBrzezinski]

We can try to devote some resources to this next week :)

Awesome! I might have some time this week, depending on how things go but I don't want to make any promises on it.

[SimonDanisch]

Alright, this actually stack overflows on refresh 😅 MWE:

using AWS, Random, Mocking, Dates
using JSON3, HTTP

AWS.@service STS
global acces_key = randstring(10)
global secret_access_key = randstring(50)
global session_token = randstring(50)
Mocking.activate()
function credentials_from_webtoken()
    role_arn = ENV["AWS_ROLE_ARN"]
    role_session = ENV["AWS_ROLE_SESSION_NAME"]
    token = read(ENV["AWS_WEB_IDENTITY_TOKEN_FILE"], String)
    result = STS.assume_role_with_web_identity(role_arn, role_session, token)
    role_creds = result["AssumeRoleWithWebIdentityResult"]["Credentials"]
    return AWS.AWSCredentials(role_creds["AccessKeyId"], role_creds["SecretAccessKey"],
                              role_creds["SessionToken"];
                              expiry=DateTime(rstrip(role_creds["Expiration"], 'Z')),
                              renew=credentials_from_webtoken)
end

http_request_patch = @patch function AWS._http_request(request)
    creds = Dict("AccessKeyId" => acces_key, "SecretAccessKey" => secret_access_key,
                 "SessionToken" => session_token, "Expiration" => string(now(UTC)))
    result = Dict("AssumeRoleWithWebIdentityResult" => Dict("Credentials" => creds))
    return HTTP.Response(200, ["Content-Type" => "text/json", "charset" => "utf-8"], body=JSON3.write(result))
end

tokenfile = joinpath(@__DIR__, "test-token")
token = string(rand(UInt128))
write(tokenfile, token)

ENV["AWS_ROLE_ARN"] = "arn:aws:iam::371827381723:role/test"
ENV["AWS_ROLE_SESSION_NAME"] = "test"
ENV["AWS_WEB_IDENTITY_TOKEN_FILE"] = tokenfile
ENV["AWS_ACCESS_KEY_ID"] = "test"
ENV["AWS_SECRET_ACCESS_KEY"] = "test"

config = apply(http_request_patch) do
    AWS.global_aws_config(AWS.AWSConfig(credentials_from_webtoken(), "us-east-2", "json"))
end
# this overflows:
AWS.check_credentials(config.credentials)

This is the fix:

function credentials_from_webtoken()
    role_arn = ENV["AWS_ROLE_ARN"]
    role_session = ENV["AWS_ROLE_SESSION_NAME"]
    token = read(ENV["AWS_WEB_IDENTITY_TOKEN_FILE"], String)
    # We need to use the default config explicitly,
    # like we do in the first call to assume_role_with_web_identity,
    # otherwise it will try to use the expired credentials in `renew` and gets into a stackoverflow
    default_config = AWS.AWSConfig()
    result = STS.assume_role_with_web_identity(role_arn, role_session, token; aws_config=default_config)
    role_creds = result["AssumeRoleWithWebIdentityResult"]["Credentials"]
    return AWS.AWSCredentials(role_creds["AccessKeyId"], role_creds["SecretAccessKey"],
                              role_creds["SessionToken"];
                              expiry=DateTime(rstrip(role_creds["Expiration"], 'Z')),
                              renew=credentials_from_webtoken)
end

[SimonDanisch]

Awesome, thank you! :)