Add `stacks.name_template` section to `atmos.yaml`. Add `Go` templating to Atmos stack manifests

[aknysh]

what

• Add stacks.name_template section to atmos.yaml
• Add Go templating to Atmos stack manifests
• Update docs
  • https://pr-560.atmos-docs.ue2.dev.plat.cloudposse.org/cli/configuration/#stacks
  • https://pr-560.atmos-docs.ue2.dev.plat.cloudposse.org/core-concepts/stacks/templating/

why

• Add stacks.name_template section to atmos.yaml - stacks.name_template section allows greater flexibility in defining the naming convention for the top-level Atmos stacks

  • stacks.name_pattern configures the name pattern for the top-level Atmos stacks using the context variables namespace, tenant, environment and stage as the tokens. Depending on the structure of your organization, OUs, accounts and regions, set stacks.name_pattern to the following:

  • name_pattern: {stage} - if you use just one region and a few accounts (stages) in just one organization and one OU. In this case, the top-level Atmos stacks will use just the stage (account) in their names, and to provision the Atmos components in the top-level stacks, you will be executing Atmos commands like atmos terraform apply <component> --stack dev, atmos terraform apply <component> --stack staging and atmos terraform apply <component> --stack prod

  • name_pattern: {environment}-{stage} - if you have multiple regions and accounts (stages) in just one organization and one OU. In this case, the top-level Atmos stacks will use the environment (region) and stage (account) in their names, and to provision the Atmos components in the top-level stacks, you will be executing Atmos commands like atmos terraform apply <component> --stack ue2-dev, atmos terraform apply <component> --stack uw2-staging and atmos terraform apply <component> --stack ue1-prod. Note that the name_pattern can also be defined as {stage}-{environment}, in which case the Atmos commands will look like atmos terraform apply <component> --stack dev-ue2

  • name_pattern: {tenant}-{environment}-{stage} - if you have multiple regions, OUs (tenants) and accounts (stages) in just one organization. In this case, the top-level Atmos stacks will use the tenant, environment (region) and stage (account) in their names, and to provision the Atmos components in the top-level stacks, you will be executing Atmos commands like atmos terraform apply <component> --stack plat-ue2-dev, atmos terraform apply <component> --stack core-uw2-staging and atmos terraform apply <component> --stack plat-ue1-prod, where plat and core are the OUs/tenants in your organization

  • name_pattern: {namespace}-{tenant}-{environment}-{stage} - if you have a multi-org, multi-tenant, multi-account and multi-region architecture. In this case, the top-level Atmos stacks will use the namespace, tenant, environment (region) and stage (account) in their names, and to provision the Atmos components in the top-level stacks, you will be executing Atmos commands like atmos terraform apply <component> --stack org1-plat-ue2-dev, atmos terraform apply <component> --stack org2-core-uw2-staging and atmos terraform apply <component> --stack org2-plat-ue1-prod, where org1 and org2 are the organization names (defined as namespace in the corresponding _defaults.yaml config files for the organizations)

  • stacks.name_template serves the same purpose as stacks.name_pattern (defines the naming convention for the top-level Atmos stacks), but provides much more functionality. Instead of using the predefined context variables as tokens, it uses Go templates. Sprig Functions are supported as well

  • For the Go template tokens, and you can use any Atmos sections (e.g. vars, providers, settings) that the Atmos command atmos describe component <component> -s <stack> generates for a component in a stack.

  • name_template: "{{.vars.tenant}}-{{.vars.environment}}-{{.vars.stage}}" defines the same name pattern for the top-level Atmos stacks as name_pattern: "{tenant}-{environment}-{stage}" does

  • Since stacks.name_template allows using any variables form the vars section (and other sections), you can define your own naming convention for your organization or for different clouds (AWS, Azure, GCP). For example, in the corresponding _defaults.yaml stack manifests, you can use the following variables:

    • org instead of namespace
    • division instead of tenant
    • region instead of environment
    • account instead of stage

    Then define the following stacks.name_template in atmos.yaml:

    stacks:
    name_template: "{{.vars.division}}-{{.vars.account}}-{{.vars.region}}"

    You will be able to execute all Atmos commands using the newly defined naming convention:

    atmos terraform plan <component> -s <division-account-region>
    atmos terraform apply <component> -s <division-account-region>
    atmos describe component <component> -s <division-account-region>

    NOTE: Use either stacks.name_pattern or stacks.name_template to define the naming convention for the top-level Atmos stacks. stacks.name_template has higher priority. If stacks.name_template is specified, stacks.name_pattern will be ignored.

• Add Go templating to Atmos stack manifests

  Atmos supports Go templates in stack manifests. Sprig Functions are supported as well.

  You can use Go templates in the following Atmos section to refer to values in the same or other sections:

  • vars
  • settings
  • env
  • metadata
  • providers
  • overrides
  • backend
  • backend_type

  
  

  NOTE: In the template tokens, you can refer to any value in any section that the Atmos command atmos describe component <component> -s <stack>generates.

  For example, let's say we have the following component configuration using Go templates:

  component:
  terraform:
    vpc:
      settings:
        setting1: 1
        setting2: 2
        setting3: "{{ .vars.var3 }}"
        setting4: "{{ .settings.setting1 }}"
        component: vpc
        backend_type: s3
        region: "us-east-2"
        assume_role: "<role-arn>"
      backend_type: "{{ .settings.backend_type }}"
      metadata:
        component: "{{ .settings.component }}"
      providers:
        aws:
          region: "{{ .settings.region }}"
          assume_role: "{{ .settings.assume_role }}"
      env:
        ENV1: e1
        ENV2: "{{ .settings.setting1 }}-{{ .settings.setting2 }}"
      vars:
        var1: "{{ .settings.setting1 }}"
        var2: "{{ .settings.setting2 }}"
        var3: 3
        # Add the tags to all the resources provisioned by this Atmos component
        tags:
          atmos_component: "{{ .atmos_component }}"
          atmos_stack: "{{ .atmos_stack }}"
          atmos_manifest: "{{ .atmos_stack_file }}"
          region: "{{ .vars.region }}"
          terraform_workspace: "{{ .workspace }}"
          assumed_role: "{{ .providers.aws.assume_role }}"
          description: "{{ .atmos_component }} component provisioned in {{ .atmos_stack }} stack by assuming IAM role {{ .providers.aws.assume_role }}"
          # `provisioned_at` uses the Sprig functions
          # https://masterminds.github.io/sprig/date.html
          # https://pkg.go.dev/time#pkg-constants
          provisioned_at: '{{ dateInZone "2006-01-02T15:04:05Z07:00" (now) "UTC" }}'

  When executing Atmos commands like atmos describe component and atmos terraform plan/apply, Atmos processes all the template tokens in the manifest and generates the final configuration for the component in the stack:

  settings:
  setting1: 1
  setting2: 2
  setting3: 3
  setting4: 1
  component: vpc
  backend_type: s3
  region: us-east-2
  assume_role: <role-arn>
  backend_type: s3
  metadata:
  component: vpc
  providers:
  aws:
    region: us-east-2
    assume_role: <role-arn>
  env:
  ENV1: e1
  ENV2: 1-2
  vars:
  var1: 1
  var2: 2
  var3: 3
  tags:
    assumed_role: <role-arn>
    atmos_component: vpc
    atmos_manifest: orgs/acme/plat/dev/us-east-2
    atmos_stack: plat-ue2-dev
    description: vpc component provisioned in plat-ue2-dev stack by assuming IAM role <role-arn>
    provisioned_at: "2024-03-12T16:18:24Z"
    region: us-east-2
    terraform_workspace: plat-ue2-dev

  
  

  While Go templates in Atmos stack manifests offer great flexibility for various use-cases, one of the obvious use-cases is to add a standard set of tags to all the resources in the infrastructure.

  For example, by adding this configuration to the stacks/orgs/acme/_defaults.yaml Org-level stack manifest:

  terraform:
  vars:
    tags:
      atmos_component: "{{ .atmos_component }}"
      atmos_stack: "{{ .atmos_stack }}"
      atmos_manifest: "{{ .atmos_stack_file }}"
      terraform_workspace: "{{ .workspace }}"
      provisioned_at: '{{ dateInZone "2006-01-02T15:04:05Z07:00" (now) "UTC" }}'

  the tags will be processed and automatically added to all the resources provisioned in the infrastructure.