search

terraform-google-modules / terraform-example-foundation

Shows how the CFT modules can be composed to build a secure cloud foundation
https://cloud.google.com/architecture/security-foundations
Apache License 2.0
1.2k stars 706 forks source link

Allow deploying more than one instance of the LZ under an organization #1170

Closed mromascanu123 closed 3 months ago

mromascanu123 commented 5 months ago

TL;DR

If a deployment already completed under a folder, a 2'nd deployment under a different folder in the same organization will fail. Because some resources are deployed at the org level and their names are not suffix-randomized as is the case e.g. with project IDs. Deploying multiple instances, one by developer, is necessary for testing. When deploying 1-org in 2'nd deployment getting

Error: Error creating NotificationConfig: googleapi: Error 409: Requested entity already exists

with google_scc_notification_config.scc_notification_config, on scc_notification.tf line 32, in resource "google_scc_notification_config" "scc_notification_config": 32: resource "google_scc_notification_config" "scc_notification_config" {

Error: Error waiting to create TagKey: Error waiting for Creating TagKey: Error code 6, message: generic::ALREADY_EXISTS: A TagKey with short name 'environment' already exists under parent 'organizations/946862951350'

with google_tags_tag_key.tag_keys["environment"], on tags.tf line 56, in resource "google_tags_tag_key" "tag_keys": 56: resource "google_tags_tag_key" "tag_keys" {

Terraform Resources

Resources like "google_access_context_manager_access_policy" or "google_tags_tag_key" have as parent "organizations/${var.parent_id}"
In case Terraform supports as parent only the organization then give an option to randomize the resource names to avoid conflicts

Detailed design

Randomize the names using a suffix. Intent already in 1-org/envs/shared/tags.tf - just randomize values : 
locals {
tags = {
    environment = {
      shortname   = "environment${local.key_suffix}"
      description = "Environment identification"
      values      = ["bootstrap", "production", "non-production", "development"]
    }
...
key_suffix  = var.create_unique_tag_key ? "-${random_string.tag_key_suffix.result}" : ""
}

resource "google_tags_tag_key" "tag_keys" {
  for_each = local.tags

  parent      = "organizations/${local.org_id}"
  short_name  = each.value.shortname
  description = each.value.description
}

Additional information

Similarly randomize resource name in 1-org/envs/shared/scc_notification.tf (right now hardcoded as below and defined at org level) resource "google_scc_notification_config" "scc_notification_config" { config_id = var.scc_notification_name organization = local.org_id

mromascanu123 commented 4 months ago

Here are some org-level resources

1-org/envs/shared/scc_notification.tf

resource "google_scc_notification_config" "scc_notification_config" { config_id = var.scc_notification_name organization = local.org_id ... }

1-org/envs/shared/org_policy.tf :

resource "google_access_context_manager_access_policy" "access_policy" { count = var.create_access_context_manager_access_policy ? 1 : 0 parent = "organizations/${local.org_id}" title = "default policy" }

terraform-google-modules/vpc-service-controls/google/main.tf

resource "google_access_context_manager_access_policy" "access_policy" { provider = google parent = "organizations/${var.parent_id}" title = var.policy_name }

1-org/envs/shared/tags.tf

resource "google_tags_tag_key" "tag_keys" { for_each = local.tags parent = "organizations/${local.org_id}" short_name = each.value.shortname description = each.value.description }

Tags are being referenced via the tfstate 2-environments/modules/env_baseline/remote.tf locals { ... tags = data.terraform_remote_state.org.outputs.tags }

2-environments/modules/env_baseline/folders.tf resource "google_tags_tag_binding" "folder_env" { parent = "//cloudresourcemanager.googleapis.com/${google_folder.env.id}" tagvalue = local.tags["environment${var.env}"]

depends_on = [time_sleep.wait_60_seconds] }

The deployment of org-level resources should be separated from the deployment of other TEF resources living under folders and projects, this to allow multiple instances of TEF to be deployed by developers under same org. In the case of vpc-service-controls the names of access levels are randomized hence there should be no conflicts. And the actual value of the access_context_manager_policy_id is determined programatically

The tags are provisioned and referred to via the remote tfstate in the rest of the code terraform-google-modules/vpc-service-controls/google/main.tf cc: @fmichaelobrien@google.com

mromascanu123 commented 4 months ago

Or rather than separating the deployment of org-level resources detect if already existing e.g. (something very approximately along the lines): data "external" "get_access_context_manager_id" { // the script would essentially do the manual steps documented in the readme program = ["bash","get_access_context_manager_id.sh"] query = { org_id = var.org_id } }

output "access_context_manager_id" { value = data.external.get_access_context_manager_id.result.id }

Then check in the module - if the resource exists set count=0

mromascanu123 commented 4 months ago

For the tags : create_unique_tag_key = true in terraform.tfvars In 1-org/envs/shared/tags.tf locals { ... tags = { environment = { shortname = "environment${local.key_suffix}" ... key_suffix = var.create_unique_tag_key ? "-${random_string.tag_key_suffix.result}" : ""

}

mromascanu123 commented 4 months ago

For the scc_notification In 1-org/envs/shared/terraform.mod.tfvars scc_notification_name = "scc-notify"

In 1-org/envs/shared/scc_notification.tf

resource "google_scc_notification_config" "scc_notification_config" { config_id = var.scc_notification_name ...

In 1-org/envs/shared/outputs.tf output "scc_notification_name" { value = var.scc_notification_name description = "Name of SCC Notification" }

Can change this

fmichaelobrien commented 4 months ago

stale bot timer restart - https://github.com/terraform-google-modules/terraform-example-foundation/blob/master/.github/workflows/stale.yml#L21

eeaton commented 3 months ago

This is feasible today when running the foundation deployer scripts. The global var file set for that script includes the variables that must be unique per instance of LZ, including parent_folder, create_unique_tag_key, etc.