gruntwork-io / terragrunt

Terragrunt is a flexible orchestration tool that allows Infrastructure as Code written in OpenTofu/Terraform to scale.
https://terragrunt.gruntwork.io/
MIT License
7.91k stars 966 forks source link

IDEA: How about introducing a plugin architecture/extending terragrunt with custom functions #2369

Open davehewy opened 1 year ago

davehewy commented 1 year ago

I often find myself writing "boilerplate" code to perform a function. Like looking up an account ID or some static configurations. Terragrunt could go deeper and promote DRY by allowing us to define custom functions to wrap up functionality. Preventing boilerplate TG code polluting all configs

Consider this example.

grant_aws_account_ids = [
    lookup(incude.root.locals.yaml_config.account_id_map, " example")
]

--- what if it could be expressed as

grant_aws_account_ids = [
     get_customer_account_id("example")
]

An alternative approach could be using a TF module as a dependency and including some logic there instead.

If you have any thoughts, comment below. Or if you have ideas on how you can already achieve this.

lstn commented 1 year ago

This would be great - would clean up a lot of duplicated code for us in our terragrunt configs, as well as in templates. We have been loading some inputs from YAML files to unclutter module level terragrunt.hcl configs, and not having custom functions makes these harder to read unfortunately. For instance:

in a module's terragrunt.hcl inputs:

groups_config = yamldecode(templatefile("${get_terragrunt_dir()}/config/groups.yaml", {
    pd_map = dependency.<......>.pd_map
    default_group_path = "/my/path"
  })).groups # this could also be a UDF and be cleaner

In config/groups.yaml:

groups:
  foogroup:
    path: ${default_group_path}
    custom_policy_docs: ${jsonencode([for document in [
        "access_warehouse",
        "foo_manage_clusters",
        "foo_manage_data",
        "foo_manage_network",
        "foo_manage_web",
        ] : lookup(pd_map, document, {}).document
      ])}
    aws_policy_names:
      - "AmazonRDSFullAccess" 
      - "AmazonElastiCacheFullAccess" 
      - "AmazonRedshiftFullAccess" 
      - "AmazonSQSFullAccess" 
      - "AmazonECS_FullAccess" 
  bargroup:
    <...>

With UDFs, this would become much cleaner:

groups:
  foogroup:
    path: ${default_group_path}
    custom_policy_docs: ${my_custom_fn(["access_warehouse", "foo_manage_clusters", "foo_manage_data", "foo_manage_network", "foo_manage_web"])}
    aws_policy_names:
      - "AmazonRDSFullAccess" 
      - "AmazonElastiCacheFullAccess" 
      - "AmazonRedshiftFullAccess" 
      - "AmazonSQSFullAccess" 
      - "AmazonECS_FullAccess" 
  bargroup:
    <...>
alikhil commented 1 year ago

Maybe it could be implemented by shared binaries? So developers could write some code and store it near to terragrunt files and terragrunt will compile them into shared binary files and use to extend with some functions.

Here some links that could inspire someone to try: