AWS Secrets Engine HCL Templating

[clushie]

Is your feature request related to a problem? Please describe.

I'm in the process of evaluating Vault to combine the Auth Method JWT/OIDC with AWS Secrets Engine.

My goal is to "dynamically" allow access to AWS Roles based on the JWT "payload".

In other words: I'd like to be able to dynamically retrieve AWS Roles based on templates instead of a static policy_document.

Describe the solution you'd like

Basically in the example of AWS secret documentation a role is getting created. On creation a policy_document is created. I assume it is json

vault write aws/roles/my-role \
    credential_type=iam_user \
    policy_document=-<<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "ec2:*",
      "Resource": "*"
    }
  ]
}
EOF

For my scenario I'd need it to be a HCL template in there.

vault write aws/roles/gitlab-aws-access \
    credential_type=iam_user \
    policy_document=-<<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "ec2:*",
      "Resource": "{{identity.entity.metadata.gitlab_path}}/{{gitlab_branch}}/*"
    }
  ]
}
EOF

which would greatly combine with a role

vault write auth/jwt/role/gitlab-aws-access - <<EOF
{
  "role_type": "jwt",
  "policies": ["gitlab-aws-access"],
  "token_explicit_max_ttl": 600,
  "bound_claims": {
    # claim, # expected claim value
    "namespace_path": "my-gitlab-project",
    "ref_type": "branch"
  },
  "claim_mappings": {
# match CI_JOB_JWT project_id into vault metadata "gitlab_id 
    "project_path": "gitlab_path",
    "ref": "gitlab_branch"
  }

and a policy

vault policy write gitlab-aws-access - <<EOF

path "aws/gitlab-aws-access" {
  capabilities = ["read",  "list"]
}
EOF

Describe alternatives you've considered

I don't know. Maybe I can use AWS STS directly instead of going the stretch through Vault? I believe STS also supports some kind of JWT based authentication, but no idea how I'd allow this.

I'd really appreciate any response, I believe this approach is novel and would enable us to make GitLab and AWS a true family. Would also be nice for not ec2 but eks, lambda-deployments, Amplify, ...

[norpol]

Apparently the bindings of GCP Secret Backend already allows hcl, so hoping this might be added to the AWS backend too?

[jay-dee7]

Even we ran into a similar situation, we need to let users assume a role to an environment for a specific period of time. Instead of adding them into a vault group and then removing them later on, we'd want to use AWS IAM Policy conditional variables (DateGreaterThan,DateLessThan) to automate this process. I see a simple cli would actually do the job, but it'll be a great addition to vault itself if its an inbuilt feature

[clushie]

I'm very happy to heart that @jay-dee7.

I'm kind of trying to learn enough go to contribute this feature, I've also started digging into the vault-code and I'm very much hoping to contribute this eventually. I figured would be really cool to contribute this within Hacktober, though if there is anyone who would be able to give me a little bit of a tutor/guidance that would be super appreciated!

[jay-dee7]

I've never contributed to vault project but I've been doing Go for 2+ years, i'd love to help you on that side if you need any 😊

[Chili-Man]

We're looking for feature like this as well to dynamically create IAM policies based on the identity of the user

[meyerbro]

I don't know if it will help you but in my case I first went to vault and checked the entity aliases that would match myself (the one that JWT got for me) and then check in the metadata of it to see which variables were there.

Once I confirmed the right value was there I discovered that to use it you will probably need to do this: {{identity.entity.aliases.[JWT-ALIAS].metadata.gitlab_path}}

[JWT-ALIAS] will be visible from the Vault UI.

[adrien-f]

👋 Hi everyone!

I've given this a shot and made #16580 to address the templating of the policy, let me know if you can try it out!

[norpol]

Hey @adrien-f, I have troubles expressing how extremely happy you just made me. I don't work at the place anymore where I wanted to integrate it, but I've just setup a sample project in GitLab, compiled your branch, setup a test vault instance, recreated the configuration from the initial issue (I'm clushie btw) and I've successfully uploaded a file to a S3 bucket at a path that didn't exist yet from within a pipeline based on a policy template. Which basically means you can now combine AWS with GitLab pipelines in an incredible secure and awesome way.

You can see the successful push to S3 here: https://gitlab.com/norpol/vault-aws-s3-demo/-/jobs/2943110795

I'm documenting a bit of what I did, perhaps this might be useful for others in regards of inspiration.

# the AWS credentials that vault is going to use to create temporary accounts
AWS_ACCESS_KEY_ID=
AWS_SECRET_ACCESS_KEY=
bucket_name="replace-me"
gitlab_project_path="gitlab-group/gitlab-project"

vault secrets enable aws
vault auth enable jwt

vault write aws/config/root access_key="${AWS_ACCESS_KEY_ID}" secret_key="${AWS_SECRET_ACCESS_KEY}" region=us-east-1
# change this if you're using a self-hosted GitLab instance
vault write auth/jwt/config jwks_url="https://gitlab.com/-/jwks"  bound_issuer="gitlab.com"
# verify that a the jwt auth is now showing up
vault auth list
vault write -f auth/jwt/role/gitlab-aws-access - <<EOF
{
  "role_type": "jwt",
  "policies": ["gitlab-aws-access"],
  "token_explicit_max_ttl": 600,
  "user_claim": "user_email",
  "bound_claims": {
    "project_path": "${gitlab_project_path}"
  },
  "claim_mappings": {
    "project_path": "project_path",
    "ref": "ref",
    "ref_type": "ref_type"
  }
}
EOF

vault policy write gitlab-aws-access - <<EOF
path "aws/creds/gitlab-aws-access" {
  capabilities = ["read",  "list"]
}
EOF

# the accessor is needed for accessing the `claim_mappings` after authentication via JWT
accessor="$(vault auth list | awk '/jwt/ { print $3; exit }')"
vault write aws/roles/gitlab-aws-access credential_type=iam_user policy_document_template=true policy_document=- <<EOF
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:*"
            ],
            "Resource": "arn:aws:s3:::${bucket_name}/{{ identity.entity.aliases.${accessor}.metadata.project_path }}/{{ identity.entity.aliases.${accessor}.metadata.ref_type }}/{{ identity.entity.aliases.${accessor}.metadata.ref }}/*"
        }
    ]
}
EOF

[macmiranda]

Why can't you use the IAM OIDC provider?

[norpol]

@macmiranda I haven't tried it out yet, I'll check out how it compares. Back when I've opened the ticket there was no CI_JOB_JWT_V2 link to the MR description which now allows the IAM OIDC provider to work I believe.

[adrien-f]

@macmiranda the issue is that IAM doesn't resurface all claims Gitlab exposes, the only one you could work with is the sub but not everything is part of it such as the environment and so far, it's not configurable in Gitlab.

You might be able to do some basic claim matching: https://docs.gitlab.com/ee/ci/cloud_services/#configure-a-conditional-role-with-oidc-claims but might not be enough for everyone

See https://gitlab.com/groups/gitlab-org/-/epics/7193#note_851060975 for more explanations:

AWS only supports the amr, aud, azp, id, oaud and sub condition keys for sts:AssumeRoleWithWebIdentity requests, rendering custom claims as important as environment or environment_protected completely useless for authorization purposes. Would it be possible to include some additional values like environment in the sub claim so they can be used to narrow down the trust conditions like GitHub does?

[archoversight]

Why can't you use the IAM OIDC provider?

This requires that the JWK is available publicly on the internet. If you have a Gitlab instance that is behind the firewall and can't expose it publicly it will not work as AWS does not have a way to upload the JWK when you are setting up the federated trust, and the JWK location has to match what is presented in the JWT.