Modify solution to be able to support multiple environments in same repo

[gwright99]

This solution was originally designed with the idea that customers with multiple environments would run a dedicated installer repo for each environment (i.e. a repo for DEV, and another for PROD).

This approach worked for some customers, but did not align with other implementations where the client team wanted to manage all of their deployments out of a single repo.

We should consider retrofitting the project structure to allow 2+ deployments to be run out of a single repo instance.

[gwright99]

Aggregating ideas for how to do so:

• https://github.com/adriannavarrobetrian/satoshi-terraform
• https://github.com/TusharParasrampuria/three-tier-architecture/tree/main

[gwright99]

Another idea: use Terraform workspaces

This approach will:

• Generate separate state files in the backend.
• Can support separate tfvar files (see below for elaboration).
• Could allow additional customization in the tool.

Implementation Considerations

Although the state files would be separate, the project files are not segregated/differentiated like how git branches can differ. As per this Spacelift article on managing variables in a Terraform workspace, there would likely be multiple tfvars files:

vars_dev.tfvars
vars_test.tfvars
vars_stage.tfvars
vars_prod.tfvars

These files could be specifically used during deployment:

# Example - using a specific tfvars file
$ terraform apply -var-file=vars_dev.tfvars

This could be baked into the Makefile solution we use:

...
apply-dev:
    @terraform apply -var-file=vars_dev.tfvars
apply-test:
    @terraform apply -var-file=vars_test.tfvars
...

# OR resolve workspace dynamically
apply-agnostic:
    @terraform apply -var-file=vars_$(terraform workspace show).tfvars

Given how the project is structured (_exclusive focus on terraform.tfvars and related secrets in SSM, with all project files checked into git_), this approach could work pretty well.

Terraform Resource Configuration

TF Objects could be modified to behave differently based on env (i.e. workspace) or include an environment descriptor in their tags. Examples:

Assign different config based on environment

locals {
  instance_type = terraform.workspace == “prod” ? “t2.large” : “t2.micro”
}

Add Tag with environment identifier

variable "name_tag" {
  type        = string
  description = "Name of the EC2 instance"
  default     = "EC2"
}

resource "aws_instance" "my_vm" {
  ami           = var.ami //Ubuntu AMI
  instance_type = var.instance_type

  tags = {
    Name = format("%s_%s", var.name_tag, terraform.workspace)
  }
}

Caveats

1. As per education materials re: TF Workspaces, introducing workspaces in the Terraform workflow increases the risk of human error.

2. Workspaces are generally meant to be temporary, but this approach would use them as a permanently fixture.

3. This approach may not be the best solution to manage multiple staging environments if organizations want these environments to be strictly separate (rather than a multi-environment-spanning monorepo).

[gwright99]

The more reading I do about Workspaces, the more I see them referred to as a "a bad idea that will blow up in your face". An alternative, more physically-separated solution could use folders to segregate environments:

• Each env gets its own provider and variables.
• Common components live as modules.

/
├── environments/
│   ├── staging/
│   │   ├── provider.tf
│   │   ├── variables.tf
│   │   └── main.tf
│   └── prod/
│       ├── providers.tf
│       ├── variables.tf
│    /   └── app.tf
└── modules/
    ├── ec2/
    ├── vpc/
    │   ├── main.tf
    │   └── variables.tf
    └── application

[adriannavarrobetrian]

Workspaces is definitely not suitable for production. The state for different environments all live in the same state file, it's a recipe for disaster if the state is broken in ANY of the environments.

This is the proper way to do it as per terraform common/best practices. I'm working on a POC google cloud terraform deployment at the moment and I'm following that pattern.

/
├── environments/
│   ├── staging/
│   │   ├── provider.tf
│   │   ├── variables.tf
│   │   └── main.tf
│   └── prod/
│       ├── providers.tf
│       ├── variables.tf
│    /   └── app.tf
└── modules/
    ├── ec2/
    ├── vpc/
    │   ├── main.tf
    │   └── variables.tf
    └── application