Dependency issue between Perimeter and Bridges on CREATE and UPDATE operations

[steffencircle]

TL;DR

We are using the regular_service_perimterand the bridge_service_perimeter Modules in order to add Project-Numbers to the Perimeter and the respective Bridge.

During the initial CREATE operation, projects first need to be added to the "Regular" Perimeter and after that to the "Bridge".

However when a Project is removed, it first needs to be removed from the "Bridge" and only after that from the "Regular" Perimeter.

I found no way in order to enforce such a behaviour.

Expected behavior

Somehow Terraform needs to perform the update backwards than it did during creation.

Observed behavior

I used the code below to demonstrate the issue. Basically during the initial apply, all projects get created and correctly added to the Perimter and then the Bridge.

Then in the next run i am changing the perimter_projects local values in order to simulate the removal of a Project.

What we then see is that TF first tries to remove the Project from the Perimeter,which results in the following error message:

module.perimeter.google_access_context_manager_service_perimeter.regular_service_perimeter: Modifying... [id=accessPolicies/POLICY_ID/servicePerimeters/vpc_sc_tf_test_perimeter]
╷
│ Error: Error updating ServicePerimeter "accessPolicies/POLICY_ID/servicePerimeters/vpc_sc_tf_test_perimeter": googleapi: Error 400: There is a problem with the Enforced Config configuration. projects/POLICY_IDis in service perimeter bridge(s) 'vpc_sc_tf_test_bridge', but not in a regular service perimeter. Each resource in a service perimeter bridge must be in a regular service perimeter.
│
│ with module.perimeter.google_access_context_manager_service_perimeter.regular_service_perimeter,
│ on .terraform/modules/perimeter/modules/regular_service_perimeter/main.tf line 21, in resource "google_access_context_manager_service_perimeter" "regular_service_perimeter":
│ 21: resource "google_access_context_manager_service_perimeter" "regular_service_perimeter" {
│

Terraform Configuration

locals {
    projects = ["one","two","three","four","five"]
}

module "project" {
  for_each = toset(local.projects) 
  source  = "terraform-google-modules/project-factory/google"
  version = "~> 11.3"

  name                 = "project-${each.key}"
  random_project_id    = "true"

  org_id               = ORG_ID
  folder_id            = FOLDER_ID
  billing_account      = BILLING_ACCOUNT
}

locals {
    perimeter_projects = [ for p in local.projects : module.project[p].project_number ]
    #perimeter_projects = [ for p in local.projects : module.project[p].project_number if p != "one" ]
}

module "perimeter" {
  source              = "terraform-google-modules/vpc-service-controls/google//modules/regular_service_perimeter"
  policy              = POLICY_ID
  perimeter_name      = "vpc_sc_tf_test_perimeter"
  description         = "Perimeter shielding projects"
  resources           = local.perimeter_projects
  restricted_services = ["storage.googleapis.com"]

  shared_resources = {
    all = local.perimeter_projects
  }
}

module "bridge" {
  source         = "terraform-google-modules/vpc-service-controls/google//modules/bridge_service_perimeter"
  policy         = POLICY_ID
  perimeter_name = "vpc_sc_tf_test_bridge"
  description    = "Bridge Projects"

  resources = module.perimeter.shared_resources["all"]

}

Terraform Version

Terraform v1.1.5
on linux_amd64
+ provider registry.terraform.io/hashicorp/google v3.90.1
+ provider registry.terraform.io/hashicorp/google-beta v4.11.0
+ provider registry.terraform.io/hashicorp/null v3.1.0
+ provider registry.terraform.io/hashicorp/random v3.1.0
+ provider registry.terraform.io/hashicorp/time v0.7.2

Additional information

No response

[morgante]

Can you try again with v4? We've changed how projects are handled and I'm hoping it fixed this.

[github-actions[bot]]

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 7 days