gruntwork-io / terragrunt

Terragrunt is a flexible orchestration tool that allows Infrastructure as Code written in OpenTofu/Terraform to scale.
https://terragrunt.gruntwork.io/
MIT License
7.93k stars 964 forks source link

Managing multiple iam_role configurations with dependencies between them #2043

Open igorinsky opened 2 years ago

igorinsky commented 2 years ago

Hello, this might be tricky to explain, but I will do my best (there is a TL;DR below)

Terragrunt version: 0.29.10 Terraform version: 1.0.1

Imagine legacy terraform approach with two providers, each assuming different IAM role in different AWS account. One provider deploys to account everything except Route53, since it is handled elsewhere.

Existing setup: roleA to deploy everything, except route53 roleB to deploy route53 in another account ( I neither control the role nor the account ) roleC at Gitlab Runner that can assume both roleA and roleB

Original configuration consists of root terragrunt.hcl with remote_state without role_arn/profile specified, but with global iam_role to specify which role should be used for operations. It works well for a single account which stores everything inside. So, default pipeline scenario is Gitlab Runner starts as roleC, switches to roleA (due to iam_role) and everything is good.

I had multiple failed attempts, but generally they can be divided to the following scenarios:

1) Keep root iam_role with roleA + add iam_role with roleB to child module

CI: AccessDenied roleA -> roleB error.

2) Remove root iam_role + add iam_role with roleB to child module

CI: Failed to load state: AccessDenied: Access Denied

3) Keep root iam_role with roleA + add iam_role with roleB to child module + add profile to remote_state

CI: roleA -> roleB AccessDenied error

4) Comment root iam_role with roleA + add iam_role with roleB to child module + add profile to remote_state + force pipeline to run terragrunt command (aws sts get-caller-identity checked) as roleB already assumed

CI: roleB -> roleA AccessDenied error

So, in any case either roleA or roleB should be able to assume each other.

TL;DR Is there any configuration that allows terragrunt to assume multiple iam_roles (https://terragrunt.gruntwork.io/docs/reference/config-blocks-and-attributes/#iam_role) without necessity of assumed roles to assume each other to get dependencies?

Is there any best practice to pass dependencies between modules triggered by different iam_role configs?

tonerdo commented 2 years ago

@igorinsky it wasn't clear from your description but is there a dependency between role A and role B?

igorinsky commented 2 years ago

@igorinsky it wasn't clear from your description but is there a dependency between role A and role B?

There is a moduleA that was applied with roleA (which is default and passed from root terragrunt.hcl) ModuleB requires roleB and dependency from moduleA, which fails because roleB can't assume roleA.

CI system uses roleC, which can assume both roleA and roleB.

igorinsky commented 2 years ago

I commented all the dependencies inside moduleB to make test cases easier.

Got error message, that S3 bucket does not exist (roleB and roleC don't have such access indeed).

So I added profile configuration to root terragrunt.hcl

image

in reality child module gets repo from git, but it doesn't matter much:

image

and new pipeline fails since roleA can't assume roleB again.

=== So, if we don't use any dependencies, we are blocked from using remote iam_role unless it has access to our S3 bucket (this one seems legit). If we are trying to specify which role should be used to access remote_state, this role.. overtakes everything?