chanzuckerberg / fogg

Manage Infrastructure as Code with less pain.
https://chanzuckerberg.github.io/fogg/
MIT License
295 stars 20 forks source link

feat: extra_templates allows users to control templates #1046

Closed jakeyheath closed 6 months ago

jakeyheath commented 6 months ago

Summary

This PR adds a feature to fogg.yml that allows users to add custom golang templates to components. Users will have full access to the plan's struct in the template, allowing them to write configuration files of their own without changing the fogg code base. This is handy when we have repeatable stacks for a specific application or environment, but don't necessarily need it to be hardcoded into the fogg codebase. We can think of these a mini fogg extensions.

You can provide a set of files to any component (env, account, global, default,...) and you can also specify if fogg should overwrite a file that already exists with the overwrite option (defaults to false). Setting overwrite to true would allow someone to scaffold and template an infra stack, but still be allowed to manage the component's infrastructure normally without worrying about fogg overwritting the changes.

Here's an example of how to call it:

envs:
  dev:
    eks:
      extra_templates:
      - overwrite: true
        files:
            test.txt: |
              blah1
            variables.tf: | 
              locals {
                owner_roles = [
                  "poweruser",
                  "okta-czi-admin",
                  "tfe-si",
                ]

                env = {{.Env | quote}}
                tags            = var.tags
                vpc_id          = data.terraform_remote_state.cloud-env.outputs.vpc_id
                subnet_ids      = data.terraform_remote_state.cloud-env.outputs.private_subnets

                node_groups = {
                  "arm" = {
                    size = 1
                    architecture = {
                      ami_type       = "AL2_ARM_64"
                      instance_types = ["t4g.xlarge"]
                    }
                  },
                }
                authorized_github_repos = {
                  chanzuckerberg = [var.github_repo]
                }
                addons = {
                  enable_aws_for_fluentbit = false
                  enable_guardduty         = false
                }
              }
      depends_on:
        components:
          - cloud-env
      module_source: git@github.com:chanzuckerberg/shared-infra//terraform/modules/eks-cluster-v2?ref=eks-cluster-v2-v6.9.1
      variables:
        - cluster_name=var.cluster_name
        - cluster_version=var.cluster_version
        - owner_roles
        - node_groups
        - authorized_github_repos
        - addons
      provider_aliases:
        aws.us-east-1: aws.us-east-1
      providers:
        aws:
          additional_regions:
            - us-east-1
        datadog:
          version: 3.20.0
        kubernetes:
          enabled: true
          version: 2.23.0
          cluster_component_name: eks-cluster-v2
          custom_provider: false
        kubectl:
          enabled: true
          version: 1.14.0
        helm:
          enabled: true
          version: 2.11.0
        okta:
          custom_provider: true
          version: 3.41.0
          registry_namespace: okta
          org_name: czi-prod

In addition to the normal stuff fogg creates, it would also produce these files:

terraform/envs/dev/eks/variables.tf

locals {
  owner_roles = [
    "poweruser",
    "okta-czi-admin",
    "tfe-si",
  ]
  env        = "virtual-cell-dev"
  tags       = var.tags
  vpc_id     = data.terraform_remote_state.cloud-env.outputs.vpc_id
  subnet_ids = data.terraform_remote_state.cloud-env.outputs.private_subnets
  node_groups = {
    "arm" = {
      size = 1
      architecture = {
        ami_type       = "AL2_ARM_64"
        instance_types = ["t4g.xlarge"]
      }
    },
  }
  authorized_github_repos = {
    chanzuckerberg = [var.github_repo]
  }
  addons = {
    enable_aws_for_fluentbit = false
    enable_guardduty         = false
  }
}

terraform/envs/dev/eks/test.txt

blah1

Notice how instead of relying on env = var.tags.env, we could code-generate the local variable env with env = {{.Env | quote}}, since the template has access to the full component plan.