Add support for AWS Web Identity Token File authentication

[mrkovalchuk]

Add support for AWS Web Identity Token File authentication.

Related issue: https://github.com/HENNGE/aiodynamo/issues/142

[mrkovalchuk]

For now, it's just a workaround. I need approval on a concept before I write tests, etc. I do manual tests, and it works nicely.

[ojii]

Please make sure to set up pre-commit as this will run various code quality/standardization checks when you make a commit. right now some of them (eg mypy) are failing on this branch.

[ojii]

So, I've just spent some time trying to see if I can figure out k8s/eks and the result is a resounding NO. For me to accept this PR though, I want to see this running in an EKS cluster.

Could you (or someone else who is interested in this being added) please provide a script/sequence of commands to run that create an EKS cluster, run some code in that cluster that shows this code works (eg show the output of AssumeRoleWithWebIdentityCredentials(...).get_key(...)) and then cleans everything up on AWS. The script/sequence of commands will be run in an environment that has admin access to an aws account, so don't worry about that part.

[dimaqq]

Label this help-wanted maybe? 🙄

[MarkusSintonen]

Hi @ojii, @mrkovalchuk! We have been running following implementation successfully for a long time in EKS: https://github.com/HENNGE/aiodynamo/compare/master...MarkusSintonen:support-AssumeRoleWithWebIdentity?expand=1

It has a very good test coverage (I adapted our unit tests to be compatible here) with the necessary parametrization for the credentials. It uses the JSON-format of the STS-API instead of XML. Im happy to open this one as an alternative PR instead of current PR (which doesnt have test coverage, also its missing retrying of certain STS client errors which are documented as retryable, etc.)

Here are (secrets-omitted) output we get in the EKS cluster:

% kubectl -n <namespace> exec -it deployment/<deployment> -- bash
$ python3 -m asyncio
asyncio REPL 3.11.3 (main, May  3 2023, 08:21:46) [GCC 10.2.1 20210110] on linux
Use "await" directly instead of "asyncio.run()".
Type "help", "copyright", "credits" or "license" for more information.
>>> import asyncio
>>> import httpx
>>> import json
>>> from aiodynamo.http.httpx import HTTPX
>>> from extensions.aiodynamo.credentials import AssumeRoleWithWebIdentityEnvironmentCredentials
>>> from aiodynamo.http.types import Request
>>>
>>> # Show get_key output
>>> http = HTTPX(httpx.AsyncClient())
>>> creds = AssumeRoleWithWebIdentityEnvironmentCredentials()
>>> res = await creds.get_key(http)
>>> res.id
'<SECRET_OMITTED>'
>>> res.secret
'<SECRET_OMITTED>'
>>> res.token
'<SECRET_OMITTED>'
>>>
>>> # Show raw request and response
>>> str(creds.url)
'https://sts.eu-central-1.amazonaws.com/?Action=AssumeRoleWithWebIdentity&RoleArn=<SECRET_OMITTED>&RoleSessionName=aiodynamo-session-1684762674&Version=2011-06-15&WebIdentityToken=<SECRET_OMITTED>'
>>> response = await http(Request(method="POST", url=str(creds.url), headers={"Accept": "application/json"}, body=None))
>>> response.status
200
>>> print(json.dumps(json.loads(response.body), indent=2))
{
  "AssumeRoleWithWebIdentityResponse": {
    "AssumeRoleWithWebIdentityResult": {
      "AssumedRoleUser": {
        "Arn": "<SECRET_OMITTED>",
        "AssumedRoleId": "<SECRET_OMITTED>"
      },
      "Audience": "sts.amazonaws.com",
      "Credentials": {
        "AccessKeyId": "<SECRET_OMITTED>",
        "Expiration": 1684766752.0,
        "SecretAccessKey": "<SECRET_OMITTED>",
        "SessionToken": "<SECRET_OMITTED>"
      },
      "PackedPolicySize": null,
      "Provider": "<SECRET_OMITTED>",
      "SourceIdentity": null,
      "SubjectFromWebIdentityToken": "<SECRET_OMITTED>"
    },
    "ResponseMetadata": {
      "RequestId": "<UUID>"
    }
  }
}

[mrkovalchuk]

Hi guys. I'm so sorry, and I want to be honest, I have no interest in that feature anymore. I will close this PR and if someone needs this implementation, fill free to continue. Maybe it will be me. Who knows. Sorry again.

[ls-alexander-muehlbauer]

[...] We have been running following implementation successfully for a long time in EKS: master...MarkusSintonen:support-AssumeRoleWithWebIdentity?expand=1 (compare)

[...] Im happy to open this one as an alternative PR instead of current PR (which doesnt have test coverage, also its missing retrying of certain STS client errors which are documented as retryable, etc.)

Hey @MarkusSintonen, would be actually great to see a PR for your implementation. I think it's something this library can benefit from @ojii.

Let me know if I can be of any help.

[ojii]

Hey @MarkusSintonen, would be actually great to see a PR for your implementation. I think it's something this library can benefit from @ojii.

Let me know if I can be of any help.

My primary issue with all the proposed implementations of this is still that I cannot figure out how to test this in an AWS environment since I cannot figure out EKS. If an implementation of this is to be added to this repo, I want to see clear instruction on how to test this in the actual environment, which are doable by someone who has no understanding of k8s at all.

Alternatively, someone could provide this functionality as a third party package. The credentials system in aiodynamo is intentionally built in a way where you can provide your own mechanism to get credentials.