boto / boto3

AWS SDK for Python
https://aws.amazon.com/sdk-for-python/
Apache License 2.0
8.95k stars 1.86k forks source link

Proactively refresh credential for assume-role-with-web-identity call #2345

Open Jeffwan opened 4 years ago

Jeffwan commented 4 years ago

Is your feature request related to a problem? Please describe. We are seeing an increased latency when making calls to various AWS API's. The latency only appear on the first call made every hour which make me believe its due to the AssumeRoleWithWebIdentity call that the SDK would be making to get the credentials.

I manually test a few calls aws sts assume-role-with-web-identity It takes 1-3s.

My application is latency sensitive, latency on assume-role-with-web-identity is not acceptable.

Describe the solution you'd like

SDK should managed credential fresh for users. Every assume-role-with-web-identity call will return Expiration and you can set a factor like 0.9, if it's close to expiration date, just refresh the credentails proactively in separate process and update AWS credentials for services calls.

{
    "Credentials": {
        "AccessKeyId": "ASIAVCDxxxxx7IYIK7P",
        "SecretAccessKey": "BSAaiVxxxxxxxxBT",
        "SessionToken": "FwoGZXxxxxxxx",
        "Expiration": "2020-03-12T01:37:28Z"
    },
    "SubjectFromWebIdentityToken": "system:serviceaccount:namespace:default-editor",
    "AssumedRoleUser": {
        "AssumedRoleId": "AROAVCDTNILGPQMFBAVR4:xxxx",
        "Arn": "arn:aws:sts::xxxx:assumed-role/saname/xxxx"
    },
    "Provider": "arn:aws:iam::xxxxx:oidc-provider/oidc.eks.us-west-2.amazonaws.com/id/451DD7xxxxAAD886BC910",
    "Audience": "sts.amazonaws.com"
}
swetashre commented 4 years ago

@Jeffwan - Thank you for your post. Marking this as feature request.

jqmichael commented 4 years ago

@swetashre I think https://github.com/boto/boto3/issues/443 is talking about a different problem where credentials from assume role were not refreshed at all. However, in this particular issue, boto3 does the credential refresh for assume-role-with-web-identity, it's just unfortunately it does so synchronously which slows down application.