profile-credentials-provider does not support assumed roles

[krestivo-kdinfotech]

[foo]
aws_access_key_id=redacted
aws_secret_access_key=redacted
region=us-west-2

I can do babashka pods aws functions using the above profile.

[assumed]
role_arn = arn:aws:iam::671645499998:role/OrganizationAccountAccessRole
source_profile= foo
region=us-west-2

Does not work however.

It gives me:

{:cognitect.anomalies/category :cognitect.anomalies/fault,
 :cognitect.anomalies/message
 "Unable to fetch credentials. See log for more details."}

There is no "log" that I can tell from which I can see more details, but it's clear that either the Babashka AWS pod or the underlying Cognitect AWS API library does not support automatic STS assume role as for example boto3 and the AWS CLI written on top of it does.

I can hack around this by writing my own STS utility functions, but just FYI this will confuse people I'm sure.

[stevebuik]

same for basic-credentials-provider blocked me. I'm one of those confused people.

my workaround is to use jet in shell to assume role and change env vars, then call bb

[jeroenvandijk]

@krestivo-kdinfotech These options you are referring to seem to be relatively new ~/.aws/credentials format (link)

Cognitect AWS API library only supports a very small part of the v1 credentials file

We have decided to support the credential_process (for background see)

With credential_process you can do basically anything you want with credentials (documentation, an example)

[stevebuik]

Thanks for the link but the example is very complex. That will keep me staying with my shell/jet workaround because it's much simpler.

sts assume-role is an AWS security best practice so maybe it's worth considering this for built-in support instead of needing a custom process?

[jeroenvandijk]

@stevebuik Maybe you can describe how you would like to see the sts assume-role to be built-in?

[stevebuik]

happy to. it would take 2 changes:

1. basic-credentials-provider needs a new passthrough arg for aws session token. this is required for assumed roles
2. a new credentials fn that calls assume role and load the results into a basic provider. something like this..

   (defn assumed-role-creds-provider
   "assume an AWS role and return a credentials-provider that will use the new keys to authenticate requests"
   ; https://github.com/babashka/pod-babashka-aws#credentials
   [{:keys [region role-arn]}]
   (let [sts-client (aws/client {:api    :sts
                               :region region})
       {:keys [Credentials] :as response}
       (aws/invoke sts-client
                   {:op      :AssumeRole
                    :request {:RoleArn         role-arn
                              :RoleSessionName "Hotplate-CI-deploy"}})]
   (if-let [k (:AccessKeyId Credentials)]
     (credentials/basic-credentials-provider
       {:access-key-id     k
   ; TODO session token here
        :secret-access-key (:SecretAccessKey Credentials)})
     (throw (ex-info "failed to assume role" response)))))

this would be "built-in" enough. it would allow CI to use assume-role conditions to lock down so only CI servers can escalate privs for deployment.

[stevebuik]

fwiw here's my jet-based workaround...

            - bash <(curl -s https://raw.githubusercontent.com/borkdude/jet/master/install)
            - aws sts assume-role
              --role-arn "REDACTED"
              --role-session-name AWSCLI-Hotplate-CI
              | jet --from json --to edn --keywordize
              > /tmp/creds.edn
            - export AWS_ACCESS_KEY_ID=$(cat /tmp/creds.edn| jet --query ":Credentials :AccessKeyId println")
            - export AWS_SECRET_ACCESS_KEY=$(cat /tmp/creds.edn| jet --query ":Credentials :SecretAccessKey println")
            # must include session for temp creds https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp_use-resources.html
            - export AWS_SESSION_TOKEN=$(cat /tmp/creds.edn| jet --query ":Credentials :SessionToken println")
            - rm /tmp/creds.edn

[jeroenvandijk]

@borkdude Do you think we could add support for the CredentialProvider protocol now that Babashka supports reify and defprotocol?

None of the above described functionality is available in Cognitect's AWS API, however this lib supports this protocol making more customisations possible. If we want to prevent deviating too much away from the source lib, and spending more time on support probably, I think supporting this protocol would help. We could then refer to the source lib instead.

Here is an example how of how the above sts functionality (described by @stevebuik) would work with the protocol: link

[borkdude]

@jeroenvandijk Bb supports protocols and reify on protocols, but we're dealing with serialized objects from bb <-> pod which might complicate things since you can't serialize functions?

Doing this with https://github.com/grzm/awyeah-api might be easier - this is basically AWS API rewritten for babashka, from source.

[jeroenvandijk]

Bb supports protocols and reify on protocols, but we're dealing with serialized objects from bb <-> pod which might complicate things since you can't serialize functions?

Ah yeah, forgot about this part. Sounds like it would indeed be complicated.

Maybe extending basic-credentials-provider with the :session-token field like @stevebuik suggested would give the required flexibility. Maybe better to give it a different name to avoid confusion with Cognitect's lib.

I didn't try https://github.com/grzm/awyeah-api yet, but that might be a good option too.

[borkdude]

Yeah, I think we can support some non-standard things as long as they have a clear name and some docs.

[jeroenvandijk]

I think supporting the protocol is not too complicated after all (see #57). I have some issues in the tests (see CI results of #57).

At the moment I'm not working with AWS so I'd like to look into this later. If anyone wants to investigate, feel free.