ER: Canadian Government Federal and Provincial Clients: Fortinet based hub-spoke Landing Zone

Purpose

Create a GCP Landing Zone for Canadian Public Sector clients - currently 2 of the largest PubSec clients - one Federal and one Provincial
Start: 20240305

Work Items

WI1: document bootstrap project (the project where we run 0-bootstrap and create the 2 seed projects)
WI2: bootstrap automation including super admin IAM roles and bootstrap project service enablements - will fix 0,1 and 2 statefile chaining
WI3: force TF 1.3 to 1.6 only in validation script - no 1.7

High Level Strategy: last update 20240314 with client team

all 4 bring system up in one of the following ways
unchanged = existing repo issues
- output: defaults only
localization - run tf locally with 1.3
- output: local issues only
replace with TF 1.6 (for PBR)
- output: TF 1.3 to 1.6 upgrade problems
architecture update (Fortigate 4 + 2+ VPCs) - get diff between (optionally running) TEF arch and target Arch
scripting changes below (local, tfvars, symlinks)
Integrate Fortinet TF for VMs - disable TEF transitive VMs
FedRamp, NIST-800-53, ITSG-33 compliance https://cloud.google.com/architecture/configure-networks-fedramp-dod-google-cloud

20240314: in parallel plan

lower priority: 0: running the TEF unmodified as CB/CSR - to avoid CB related issues for now
All-devs: 1: Local: Run locally as TF 1.3 (check optional support removed in 1.4) - no build tool - no CB/CSR: start with TF 1.6 (not necessarily 1.7) we know 1.3.0 , check 1.6.0 works out-of-the box - locally with no build tool (no cloud build for now) - keep terraform-google-modules links for now - see https://github.com/terraform-google-modules/terraform-example-foundation/blob/master/1-org/README.md#running-terraform-locally
Andres: 1: run local TF on TF 1.6 - follow see https://github.com/terraform-google-modules/terraform-example-foundation/blob/master/1-org/README.md#running-terraform-locally check if TF 1.6 local build fails - if it does we need a 1.3-1.6 upgrade what are the mods last upgrade reference - what could modifiable in 1.3.0 upgrade under https://github.com/terraform-google-modules/terraform-example-foundation/pull/831/files see 1.7.4 to 1.3.0 downgrade comment
no need to update the Dockerfile from 1.3 - as CB is out of the picture for local deployment
Marian, Youssef 3: localize terraform-google-modules links - either static or dynamic(current)
Marian, Youssef 4: scripts for localized modules (sed removals, version removal and ../local rewrites), fix symlinks - including those that point to a n/a tfvars

Priority: split why vs how

P1 - FG into TEF (req for: zero-trust, pbr over default routing)
- P1.1 - PBR (policy based routing) as part of FG - see https://github.com/terraform-google-modules/terraform-example-foundation/issues/1141
- P1.2 - VMs for transitivity refactor (TG like routing handled by FGs) - see https://github.com/terraform-google-modules/terraform-example-foundation/blob/master/3-networks-hub-and-spoke/modules/transitivity/main.tf#L38
- P1.1.1 - TF 1.6+ upgrade required for PBR (in concert with the ILB) -
- - will need likely a repo wide upgrade to mitigate chained and x-ref state imports between modules
P2 - hardcoding into root script/yaml
- P2.1 add sh script at least for 0-bootstrap
P3 - CB/CSR retrofit for bringing in ADO
- P3.1 - localized deployment - aka "../path" no double google/tf-gcp links (as in PBMM repo)

Future:

cover off armor standard, waf, acm
fix 0-bootrap run locally - expand it like 1, 2,3

refere to symlinks https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/blob/main/environments/bootstrap/bootstrap.sh#L453

Tracking Issue: https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/issues/345 org: olapp branch: https://github.com/CloudLandingZone/terraform-example-foundation Previous TEF run Sept 2023 - https://github.com/terraform-google-modules/terraform-example-foundation/issues/940

Take the existing TEF V4 and adapt the Fortinet terraform example LB sandwich HA cluster below Verified https://github.com/fortinet/fortigate-tutorial-gcp/tree/main/terraform Unverified https://github.com/40net-cloud/fortinet-gcp-solutions/tree/master/FortiGate look at the best one from Fortinet https://github.com/fortinet/fortigate-terraform-deploy/tree/main/gcp/7.4

Architecture

https://github.com/GoogleCloudPlatform/pubsec-declarative-toolkit/wiki/Architecture
Base Landing Zone

Merged with Fortigate LB sandwich cluster - re-peer with above

see existing LZ V2 architecture that we will generally model off as a defacto port from KCC LZ V2
core-landing-zone - https://github.com/GoogleCloudPlatform/pubsec-declarative-toolkit/tree/main/solutions/core-landing-zone
client-setup - https://github.com/GoogleCloudPlatform/pubsec-declarative-toolkit/tree/main/solutions/client-setup
client-landing-zone - https://github.com/GoogleCloudPlatform/pubsec-declarative-toolkit/tree/main/solutions/client-landing-zone
client-project-setup - https://github.com/GoogleCloudPlatform/pubsec-declarative-toolkit/tree/main/solutions/client-project-setup
guardrails - https://github.com/GoogleCloudPlatform/pubsec-declarative-toolkit/tree/main/solutions/guardrails

Work Items

correlate TEF with the work by the TF GenAI blueprint teams's work in https://cloud.google.com/architecture/genai-mlops-blueprint genai foundations fork in https://github.com/GoogleCloudPlatform/terraform-google-enterprise-genai

from 345

TEF V4 is our focus now as the base LZ with Fortigate integration from the above repo https://github.com/terraform-google-modules/terraform-example-foundation

follow previous: https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/issues/243 review previous issues: https://github.com/terraform-google-modules/terraform-example-foundation/issues/940

Org: olapp repo https://github.com/CloudLandingZone/terraform-example-foundation issue https://github.com/terraform-google-modules/terraform-example-foundation/issues/1133

michael@cloudshell:~$ ls
FGVM8VTM24000185.lic  FGVM8VTM24000186.lic  fortigate-terraform-olapp  fortinet-gcp-solutions-olapp  gcloud-ola  kcc-olapp  README-cloudshell.txt
michael@cloudshell:~$ mkdir tef-olapp
michael@cloudshell:~$ cd tef-olapp/
michael@cloudshell:~/tef-olapp$ mkdir github
michael@cloudshell:~/tef-olapp$ cd github/
michael@cloudshell:~/tef-olapp/github$ mkdir _CloudLandingZone-main
michael@cloudshell:~/tef-olapp/github$ git clone https://github.com/CloudLandingZone/terraform-example-foundation.git
Cloning into 'terraform-example-foundation'...

Prep

org is relatively clean with minimal IAM permissions
billing quota is 25 (asked for 20 in addition to default 5)
project quota is default - adding 40 to get 60 total https://github.com/GoogleCloudPlatform/pubsec-declarative-toolkit/wiki/Onboarding#quota-increase

0-bootstrap

raised - will see if we can interleave the cloud-setup groups and the TEF groups later https://github.com/terraform-google-modules/terraform-example-foundation/issues/1135

following https://github.com/terraform-google-modules/terraform-example-foundation/tree/master/0-bootstrap#prerequisites

where is group_org_admins in the tfvars file - do CB first https://github.com/terraform-google-modules/terraform-example-foundation/blob/master/0-bootstrap/terraform.example.tfvars#L44 https://github.com/terraform-google-modules/terraform-example-foundation/issues/1136

copy tfvars

https://github.com/terraform-google-modules/terraform-example-foundation/blob/master/0-bootstrap/README.md#deploying-with-cloud-build

ichael@cloudshell:~/tef-olapp/github$ cd terraform-example-foundation/0-bootstrap/
michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap$ mv terraform.example.tfvars terraform.tfvars
michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap$

replace get org via https://github.com/GoogleCloudPlatform/pubsec-declarative-toolkit/blob/gh766-script/solutions/setup.sh#L101

michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap$ gcloud config set project tef-olapp
BOOT_PROJECT_ID=tef-olapp
ORG_ID=$(gcloud projects get-ancestors $BOOT_PROJECT_ID --format='get(id)' | tail -1)
michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ echo $ORG_ID
63025...

michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ BILLING_FORMAT="--format=value(billingAccountName)"
BILLING_ID=$(gcloud billing projects describe $BOOT_PROJECT_ID $BILLING_FORMAT | sed 's/.*\///')
michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ echo $BILLING_ID
012...

org_id = "REPLACE_ME" # format "000000000000"
billing_account = "REPLACE_ME" # format "000000-000000-000000"
group_org_admins = "REPLACE_ME"
group_billing_admins = "REPLACE_ME"
# group_org_admins = "gcp-organization-admins@example.com"
# group_billing_admins = "gcp-billing-admins@example.com"

default_region = "us-central1"
parent_folder = "01234567890"

#  Optional - for enabling the automatic groups creation, uncoment the groups
#  variable and update the values with the desired group names
# groups = {
#   create_groups = true,
#   billing_project = "billing-project",
#   required_groups = {
#     group_org_admins           = "group_org_admins_local_test@example.com"
#     group_billing_admins       = "group_billing_admins_local_test@example.com"
#     billing_data_users         = "billing_data_users_local_test@example.com"
#     audit_data_users           = "audit_data_users_local_test@example.com"
#     monitoring_workspace_users = "monitoring_workspace_users_local_test@example.com"
#   },
#   optional_groups = {
#     gcp_platform_viewer      = "gcp_platform_viewer_local_test@example.com"
#     gcp_security_reviewer    = "gcp_security_reviewer_local_test@example.com"
#     gcp_network_viewer       = "gcp_network_viewer_local_test@example.com"
#     gcp_scc_admin            = "gcp_scc_admin_local_test@example.com"
#     gcp_global_secrets_admin = "gcp_global_secrets_admin_local_test@example.com"
#     gcp_audit_viewer         = "gcp_audit_viewer_local_test@example.com"
#   }
# }

to
org_id = "63...53" # format "000000000000"
billing_account = "012...B" # format "000000-000000-000000"
group_org_admins = "gcp-organization-admins@o..p"
group_billing_admins = "gcp-billing-admins@ob..p"
 parent_folder = "10...6"

using the groups left over from the cloud setup for now

gcp-organization-admins
gcp-billing-admins

adding

audit_data_users
monitoring_workspace_users
billing_data_users

validating


michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ ../scripts/validate-requirements.sh -o 63...53 -b 01...B -u mi..pp
Validating required utility tools...
Validating Terraform installation...
Validating Google Cloud SDK installation...
Validating Git installation...
  git default branch must be configured as main.
  See the instructions at https://github.com/terraform-google-modules/terraform-example-foundation/blob/master/docs/TROUBLESHOOTING.md#default-branch-setting .
Validating local gcloud configuration...
Validating roles assignment for current end user credential...
  The User must have the Organization Roles resourcemanager.folderCreator, resourcemanager.organizationAdmin and roles/orgpolicy.policyAdmin.
Validating 0-bootstrap configuration...
.......................................
Validation failed!
Errors found:
  git default branch must be configured as main.
  There are missing organization level roles on the Credential.

michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ git status
On branch master
Your branch is up to date with 'origin/master'.

Changes not staged for commit:
  (use "git add/rm <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
        deleted:    terraform.example.tfvars

no changes added to commit (use "git add" and/or "git commit -a")

raised - there is no main branch only master - adjusting script and moving on https://github.com/terraform-google-modules/terraform-example-foundation/issues/1137

in https://github.com/terraform-google-modules/terraform-example-foundation/blob/master/0-bootstrap/README.md#deploying-with-cloud-build

adjust https://github.com/terraform-google-modules/terraform-example-foundation/blob/master/scripts/validate-requirements.sh#L127

    if ! git config init.defaultBranch | grep "main" >/dev/null ; then
        echo "  git default branch must be configured as main."
        echo "  See the instructions at https://github.com/terraform-google-modules/terraform-example-foundation/blob/master/docs/TROUBLESHOOTING.md#default-branch-setting ."
        ERRORS+=$'  git default branch must be configured as main.\n'
    fi

Branch instructions are not accurate https://github.com/terraform-google-modules/terraform-example-foundation/blob/master/docs/TROUBLESHOOTING.md#default-branch-setting

there is no main branch so we should use master - what if we are doing development on a ghNNN-name branch?

michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ git config init.defaultBranch
michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ git config --global init.defaultBranch master
michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ git config --global init.defaultBranch main
michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ git status
On branch master
Your branch is up to date with 'origin/master'.

Changes not staged for commit:
  (use "git add/rm <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
        deleted:    terraform.example.tfvars

no changes added to commit (use "git add" and/or "git commit -a")
michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ git config --global init.defaultBranch master

roles

Billing Account Administrator
Compute Shared VPC Admin
Folder Admin
Folder Creator
Organization Administrator
Organization Policy Administrator
Project Billing Manager
Project Creator
Project Deleter
Project IAM Admin
Service Account Token Creator

creating main branch for now instead of hoping "master" wont cause an issue later in cloud build

michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ git checkout -b main
Switched to a new branch 'main'
michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ git config init.defaultBranch
master
michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ git config --global init.defaultBranch main
michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ git config init.defaultBranch
main

michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ ../scripts/validate-requirements.sh -o 63,,,
Validating required utility tools...
Validating Terraform installation...
Validating Google Cloud SDK installation...
Validating Git installation...
Validating local gcloud configuration...
Validating roles assignment for current end user credential...
Validating 0-bootstrap configuration...
.......................................
Validation successful!
No errors found.

Terraform version 1.7.2

little worried about pre 1.3 references - this would suggest OPTIONAL deprecation issues

michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ terraform --version
Terraform v1.7.2

0 - bootstrap - terraform init

michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ terraform init

Initializing the backend...
Initializing modules...
Downloading registry.terraform.io/terraform-google-modules/gcloud/google 3.4.0 for bootstrap_csr_repo...
- bootstrap_csr_repo in .terraform/modules/bootstrap_csr_repo
- bootstrap_projects_remove_editor in modules/parent-iam-remove-role
Downloading registry.terraform.io/terraform-google-modules/gcloud/google 3.4.0 for build_terraform_image...
- build_terraform_image in .terraform/modules/build_terraform_image
- cicd_project_iam_member in modules/parent-iam-member
Downloading registry.terraform.io/terraform-google-modules/cloud-storage/google 5.0.0 for gcp_projects_state_bucket...
- gcp_projects_state_bucket in .terraform/modules/gcp_projects_state_bucket/modules/simple_bucket
Downloading registry.terraform.io/terraform-google-modules/group/google 0.6.1 for optional_group...
- optional_group in .terraform/modules/optional_group
- org_iam_member in modules/parent-iam-member
- parent_iam_member in modules/parent-iam-member
Downloading registry.terraform.io/terraform-google-modules/group/google 0.6.1 for required_group...
- required_group in .terraform/modules/required_group
Downloading registry.terraform.io/terraform-google-modules/bootstrap/google 7.0.0 for seed_bootstrap...
- seed_bootstrap in .terraform/modules/seed_bootstrap
Downloading registry.terraform.io/terraform-google-modules/org-policy/google 5.3.0 for seed_bootstrap.enable_cross_project_service_account_usage...
- seed_bootstrap.enable_cross_project_service_account_usage in .terraform/modules/seed_bootstrap.enable_cross_project_service_account_usage
Downloading registry.terraform.io/terraform-google-modules/kms/google 2.3.0 for seed_bootstrap.kms...
- seed_bootstrap.kms in .terraform/modules/seed_bootstrap.kms
Downloading registry.terraform.io/terraform-google-modules/project-factory/google 14.5.0 for seed_bootstrap.seed_project...
- seed_bootstrap.seed_project in .terraform/modules/seed_bootstrap.seed_project
- seed_bootstrap.seed_project.budget in .terraform/modules/seed_bootstrap.seed_project/modules/budget
- seed_bootstrap.seed_project.essential_contacts in .terraform/modules/seed_bootstrap.seed_project/modules/essential_contacts
- seed_bootstrap.seed_project.gsuite_group in .terraform/modules/seed_bootstrap.seed_project/modules/gsuite_group
- seed_bootstrap.seed_project.project-factory in .terraform/modules/seed_bootstrap.seed_project/modules/core_project_factory
- seed_bootstrap.seed_project.project-factory.project_services in .terraform/modules/seed_bootstrap.seed_project/modules/project_services
- seed_bootstrap.seed_project.quotas in .terraform/modules/seed_bootstrap.seed_project/modules/quota_manager
- seed_bootstrap.seed_project.shared_vpc_access in .terraform/modules/seed_bootstrap.seed_project/modules/shared_vpc_access
- seed_project_iam_member in modules/parent-iam-member
Downloading registry.terraform.io/terraform-google-modules/bootstrap/google 7.0.0 for tf_cloud_builder...
- tf_cloud_builder in .terraform/modules/tf_cloud_builder/modules/tf_cloudbuild_builder
Downloading registry.terraform.io/terraform-google-modules/cloud-storage/google 5.0.0 for tf_cloud_builder.bucket...
- tf_cloud_builder.bucket in .terraform/modules/tf_cloud_builder.bucket/modules/simple_bucket
- tf_private_pool in modules/cb-private-pool
Downloading registry.terraform.io/terraform-google-modules/network/google 9.0.0 for tf_private_pool.firewall_rules...
- tf_private_pool.firewall_rules in .terraform/modules/tf_private_pool.firewall_rules/modules/firewall-rules
Downloading registry.terraform.io/terraform-google-modules/network/google 9.0.0 for tf_private_pool.peered_network...
- tf_private_pool.peered_network in .terraform/modules/tf_private_pool.peered_network
- tf_private_pool.peered_network.firewall_rules in .terraform/modules/tf_private_pool.peered_network/modules/firewall-rules
- tf_private_pool.peered_network.routes in .terraform/modules/tf_private_pool.peered_network/modules/routes
- tf_private_pool.peered_network.subnets in .terraform/modules/tf_private_pool.peered_network/modules/subnets
- tf_private_pool.peered_network.vpc in .terraform/modules/tf_private_pool.peered_network/modules/vpc
Downloading registry.terraform.io/terraform-google-modules/vpn/google 4.0.0 for tf_private_pool.vpn_ha_cb_to_onprem...
- tf_private_pool.vpn_ha_cb_to_onprem in .terraform/modules/tf_private_pool.vpn_ha_cb_to_onprem/modules/vpn_ha
Downloading registry.terraform.io/terraform-google-modules/bootstrap/google 7.0.0 for tf_source...
- tf_source in .terraform/modules/tf_source/modules/tf_cloudbuild_source
Downloading registry.terraform.io/terraform-google-modules/cloud-storage/google 5.0.0 for tf_source.cloudbuild_bucket...
- tf_source.cloudbuild_bucket in .terraform/modules/tf_source.cloudbuild_bucket/modules/simple_bucket
Downloading registry.terraform.io/terraform-google-modules/project-factory/google 14.5.0 for tf_source.cloudbuild_project...
- tf_source.cloudbuild_project in .terraform/modules/tf_source.cloudbuild_project
- tf_source.cloudbuild_project.budget in .terraform/modules/tf_source.cloudbuild_project/modules/budget
- tf_source.cloudbuild_project.essential_contacts in .terraform/modules/tf_source.cloudbuild_project/modules/essential_contacts
- tf_source.cloudbuild_project.gsuite_group in .terraform/modules/tf_source.cloudbuild_project/modules/gsuite_group
- tf_source.cloudbuild_project.project-factory in .terraform/modules/tf_source.cloudbuild_project/modules/core_project_factory
- tf_source.cloudbuild_project.project-factory.project_services in .terraform/modules/tf_source.cloudbuild_project/modules/project_services
- tf_source.cloudbuild_project.quotas in .terraform/modules/tf_source.cloudbuild_project/modules/quota_manager
- tf_source.cloudbuild_project.shared_vpc_access in .terraform/modules/tf_source.cloudbuild_project/modules/shared_vpc_access
Downloading registry.terraform.io/terraform-google-modules/bootstrap/google 7.0.0 for tf_workspace...
- tf_workspace in .terraform/modules/tf_workspace/modules/tf_cloudbuild_workspace
Downloading registry.terraform.io/terraform-google-modules/cloud-storage/google 5.0.0 for tf_workspace.artifacts_bucket...
- tf_workspace.artifacts_bucket in .terraform/modules/tf_workspace.artifacts_bucket/modules/simple_bucket
Downloading registry.terraform.io/terraform-google-modules/cloud-storage/google 5.0.0 for tf_workspace.log_bucket...
- tf_workspace.log_bucket in .terraform/modules/tf_workspace.log_bucket/modules/simple_bucket
Downloading registry.terraform.io/terraform-google-modules/cloud-storage/google 5.0.0 for tf_workspace.state_bucket...
- tf_workspace.state_bucket in .terraform/modules/tf_workspace.state_bucket/modules/simple_bucket

Initializing provider plugins...
- Finding hashicorp/null versions matching ">= 2.1.0"...
- Finding hashicorp/external versions matching ">= 2.2.2"...
- Finding hashicorp/google versions matching ">= 3.33.0, >= 3.43.0, >= 3.50.0, >= 3.53.0, >= 3.64.0, >= 3.67.0, >= 3.77.0, >= 3.83.0, >= 4.17.0, >= 4.25.0, >= 4.28.0, != 4.31.0, >= 4.46.0, >= 4.64.0, >= 5.7.0, < 6.0.0"...
- Finding hashicorp/random versions matching ">= 2.1.0, >= 2.2.0, >= 3.1.0, ~> 3.4"...
- Finding hashicorp/time versions matching ">= 0.5.0"...
- Finding hashicorp/google-beta versions matching ">= 3.43.0, >= 3.50.0, >= 3.64.0, >= 3.67.0, >= 3.77.0, >= 4.11.0, >= 4.17.0, >= 4.28.0, != 4.31.0, >= 4.64.0, >= 5.7.0, < 6.0.0"...
- Installing hashicorp/null v3.2.2...
- Installed hashicorp/null v3.2.2 (signed by HashiCorp)
- Installing hashicorp/external v2.3.3...
- Installed hashicorp/external v2.3.3 (signed by HashiCorp)
- Installing hashicorp/google v5.19.0...
- Installed hashicorp/google v5.19.0 (signed by HashiCorp)
- Installing hashicorp/random v3.6.0...
- Installed hashicorp/random v3.6.0 (signed by HashiCorp)
- Installing hashicorp/time v0.10.0...
- Installed hashicorp/time v0.10.0 (signed by HashiCorp)
- Installing hashicorp/google-beta v5.19.0...
- Installed hashicorp/google-beta v5.19.0 (signed by HashiCorp)

Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

Terraform plan - 260

michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ terraform plan -input=false -out bootstrap.tfplan
terraform plan -input=false -out bootstrap.tfplan

20240306: 1036

Terraform vet - need a local shell

michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ export VET_PROJECT_ID=tef-olapp
michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ terraform show -json bootstrap.tfplan > bootstrap.json
michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ gcloud beta terraform vet bootstrap.json --policy-library="../policy-library" --project ${VET_PROJECT_ID}
Pausing command execution:

This command requires the `terraform-tools` component to be installed. Would you like to install the `terraform-tools` component to continue command execution? (Y/n)?  

ERROR: (gcloud.beta.terraform.vet) 
You cannot perform this action because the Google Cloud CLI component manager 
is disabled for this installation. You can run the following command 
to achieve the same result for this installation: 

sudo apt-get install google-cloud-sdk-terraform-tools

michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ sudo apt-get install google-cloud-sdk-terraform-tools
********************************************************************************
You are running apt-get inside of Cloud Shell. Note that your Cloud Shell  
machine is ephemeral and no system-wide change will persist beyond session end. 

To suppress this warning, create an empty ~/.cloudshell/no-apt-get-warning file.
The command will automatically proceed in 5 seconds or on any key. 

Visit https://cloud.google.com/shell/help for more information.                 
********************************************************************************
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following package was automatically installed and is no longer required:
  libpcre2-posix2
Use 'sudo apt autoremove' to remove it.
The following NEW packages will be installed:
  google-cloud-sdk-terraform-tools
0 upgraded, 1 newly installed, 0 to remove and 7 not upgraded.
Need to get 24.9 MB of archives.
After this operation, 120 MB of additional disk space will be used.
Get:1 https://packages.cloud.google.com/apt cloud-sdk-bullseye/main amd64 google-cloud-sdk-terraform-tools amd64 462.0.1-0 [24.9 MB]
Fetched 24.9 MB in 2s (12.8 MB/s)                            
debconf: delaying package configuration, since apt-utils is not installed
Selecting previously unselected package google-cloud-sdk-terraform-tools.
(Reading database ... 151687 files and directories currently installed.)
Preparing to unpack .../google-cloud-sdk-terraform-tools_462.0.1-0_amd64.deb ...
Unpacking google-cloud-sdk-terraform-tools (462.0.1-0) ...
Setting up google-cloud-sdk-terraform-tools (462.0.1-0) ...

michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ gcloud beta terraform vet bootstrap.json --policy-library="../policy-library" --project ${VET_PROJECT_ID}
Validating resources...done.

Terraform apply

terraform apply bootstrap.tfplan

Get billing project quotas before running apply https://github.com/GoogleCloudPlatform/pubsec-declarative-toolkit/wiki/Onboarding#quota-increase see https://support.google.com/code/contact/billing_quota_increase https://support.google.com/code/contact/project_quota_increase

20240306:1224 running

terraform apply

michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ gcloud services enable cloudresourcemanager.googleapis.com
Operation "operations/acat.p2-153288813308-adc4acf2-18f5-4617-bd64-7d5df77820f6" finished successfully.
michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ gcloud services enable cloudidentity.googleapis.com
Operation "operations/acat.p2-153288813308-796324ee-c8f6-45f6-9c6b-79c27589f037" finished successfully.
michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ gcloud services list | grep NAME
NAME: cloudidentity.googleapis.com
NAME: cloudresourcemanager.googleapis.com
michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ gcloud services enable cloudapis.googleapis.com
Operation "operations/acat.p2-153288813308-0b7d17c4-8781-4af3-9e61-ccececbb4119" finished successfully.
michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ gcloud services enable servicemanagement.googleapis.com
michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ gcloud services enable serviceusage.googleapis.com
michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ gcloud services enable storage-api.googleapis.com
michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ gcloud services enable storage.googleapis.com
michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ gcloud services list | grep NAME
NAME: analyticshub.googleapis.com
NAME: bigquery.googleapis.com
NAME: bigqueryconnection.googleapis.com
NAME: bigquerydatapolicy.googleapis.com
NAME: bigquerymigration.googleapis.com
NAME: bigqueryreservation.googleapis.com
NAME: bigquerystorage.googleapis.com
NAME: cloudapis.googleapis.com
NAME: cloudidentity.googleapis.com
NAME: cloudresourcemanager.googleapis.com
NAME: cloudtrace.googleapis.com
NAME: dataform.googleapis.com
NAME: dataplex.googleapis.com
NAME: datastore.googleapis.com
NAME: logging.googleapis.com
NAME: monitoring.googleapis.com
NAME: servicemanagement.googleapis.com
NAME: serviceusage.googleapis.com
NAME: sql-component.googleapis.com
NAME: storage-api.googleapis.com
NAME: storage-component.googleapis.com
NAME: storage.googleapis.com

check roles
Billing Account Administrator
Compute Shared VPC Admin
Folder Admin
Folder Creator
Organization Administrator
Organization Policy Administrator
Project Billing Manager
Project Creator
Project Deleter
Project IAM Admin
Service Account Token Creator
Service Usage Consumer

check https://github.com/terraform-google-modules/terraform-example-foundation/issues/965

michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ terraform apply bootstrap.tfplan

module.seed_bootstrap.random_id.suffix: Creating...
module.seed_bootstrap.module.seed_project.module.project-factory.random_id.random_project_id_suffix: Creating...
module.tf_private_pool.random_string.suffix: Creating...
random_string.suffix: Creating...
module.seed_bootstrap.module.seed_project.module.project-factory.random_id.random_project_id_suffix: Creation complete after 0s [id=Mco]
module.seed_bootstrap.random_id.suffix: Creation complete after 0s [id=zKQ]
random_string.suffix: Creation complete after 0s [id=wm4z]
module.tf_private_pool.random_string.suffix: Creation complete after 0s [id=4ika]
google_folder.bootstrap: Creating...
module.seed_bootstrap.google_organization_iam_binding.billing_creator: Creating...
module.seed_bootstrap.google_folder_iam_member.org_admin_serviceusage_consumer[0]: Creating...
module.seed_bootstrap.google_folder_iam_member.org_admin_service_account_user[0]: Creating...
module.seed_bootstrap.google_organization_iam_member.org_admins_group["roles/billing.user"]: Creating...
module.seed_bootstrap.google_organization_iam_member.org_billing_admin: Creating...
module.seed_bootstrap.google_folder_iam_member.tmp_project_creator[0]: Creating...
module.seed_bootstrap.google_organization_iam_member.org_admins_group["roles/resourcemanager.organizationAdmin"]: Creating...
module.seed_bootstrap.google_organization_iam_binding.billing_creator: Creation complete after 5s [id=630259462753/roles/billing.creator]
module.seed_bootstrap.google_folder_iam_member.org_admin_service_account_user[0]: Creation complete after 9s [id=folders/1078109772786/roles/iam.serviceAccountUser/group:gcp-organization-admins@obrienlabs.app]
google_folder.bootstrap: Still creating... [10s elapsed]
module.seed_bootstrap.google_folder_iam_member.org_admin_serviceusage_consumer[0]: Still creating... [10s elapsed]
module.seed_bootstrap.google_organization_iam_member.org_billing_admin: Still creating... [10s elapsed]
module.seed_bootstrap.google_organization_iam_member.org_admins_group["roles/billing.user"]: Still creating... [10s elapsed]
module.seed_bootstrap.google_folder_iam_member.tmp_project_creator[0]: Still creating... [10s elapsed]
module.seed_bootstrap.google_organization_iam_member.org_admins_group["roles/resourcemanager.organizationAdmin"]: Still creating... [10s elapsed]
google_folder.bootstrap: Creation complete after 12s [id=folders/865611452734]
module.seed_bootstrap.google_folder_iam_member.org_admin_serviceusage_consumer[0]: Creation complete after 14s [id=folders/1078109772786/roles/serviceusage.serviceUsageConsumer/group:gcp-organization-admins@obrienlabs.app]
module.seed_bootstrap.google_folder_iam_member.tmp_project_creator[0]: Creation complete after 14s [id=folders/1078109772786/roles/resourcemanager.projectCreator/group:gcp-organization-admins@obrienlabs.app]
module.seed_bootstrap.module.seed_project.module.project-factory.google_project.main: Creating...
module.seed_bootstrap.google_organization_iam_member.org_billing_admin: Creation complete after 18s [id=630259462753/roles/billing.admin/group:gcp-billing-admins@obrienlabs.app]
module.seed_bootstrap.google_organization_iam_member.org_admins_group["roles/billing.user"]: Creation complete after 19s [id=630259462753/roles/billing.user/group:gcp-organization-admins@obrienlabs.app]
module.seed_bootstrap.google_organization_iam_member.org_admins_group["roles/resourcemanager.organizationAdmin"]: Creation complete after 19s [id=630259462753/roles/resourcemanager.organizationAdmin/group:gcp-organization-admins@obrienlabs.app]
╷
│ Error: failed pre-requisites: failed to check permissions on billing account "billingAccounts/012EDD-5AD5ED-ECFF0B": googleapi: Error 403: Cloud Billing API has not been used in project tef-olapp before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/cloudbilling.googleapis.com/overview?project=tef-olapp then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.
│ Details:
│ [
│   {
│     "@type": "type.googleapis.com/google.rpc.Help",
│     "links": [
│       {
│         "description": "Google developers console API activation",
│         "url": "https://console.developers.google.com/apis/api/cloudbilling.googleapis.com/overview?project=tef-olapp"
│       }
│     ]
│   },
│   {
│     "@type": "type.googleapis.com/google.rpc.ErrorInfo",
│     "domain": "googleapis.com",
│     "metadata": {
│       "consumer": "projects/tef-olapp",
│       "service": "cloudbilling.googleapis.com"
│     },
│     "reason": "SERVICE_DISABLED"
│   }
│ ]
│ , accessNotConfigured
│ 
│   with module.seed_bootstrap.module.seed_project.module.project-factory.google_project.main,
│   on .terraform/modules/seed_bootstrap.seed_project/modules/core_project_factory/main.tf line 73, in resource "google_project" "main":
│   73: resource "google_project" "main" {
│ 
╵
michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$

billing is enabled but not the api

enabling billing api
michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ gcloud services enable cloudbilling.googleapis.com
Operation "operations/acat.p2-153288813308-9c2dddaa-7b1d-4ac0-bd9c-3fe344d1e782" finished successfully.

raised https://github.com/terraform-google-modules/terraform-example-foundation/issues/1139

1232 terraform init

terraform plan -input=false -out bootstrap.tfplan
Plan: 248 to add, 0 to change, 0 to destroy.

terraform apply

michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ terraform apply bootstrap.tfplan
module.seed_bootstrap.module.seed_project.module.project-factory.google_project.main: Creating...
module.seed_bootstrap.module.seed_project.module.project-factory.google_project.main: Still creating... [10s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.google_project.main: Still creating... [20s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.google_project.main: Still creating... [30s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.google_project.main: Still creating... [40s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.google_project.main: Still creating... [50s elapsed]

module.seed_bootstrap.module.seed_project.module.project-factory.google_project.main: Still creating... [1m0s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.google_project.main: Still creating... [1m10s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.google_project.main: Still creating... [1m20s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.google_project.main: Still creating... [1m30s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.google_project.main: Still creating... [1m40s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.google_project.main: Still creating... [1m50s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.google_project.main: Still creating... [2m0s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.google_project.main: Still creating... [2m10s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.google_project.main: Still creating... [2m20s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.google_project.main: Still creating... [2m30s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.google_project.main: Still creating... [2m40s elapsed]

module.seed_bootstrap.module.seed_project.module.project-factory.google_project.main: Still creating... [2m50s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.google_project.main: Still creating... [3m0s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.google_project.main: Still creating... [3m10s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.google_project.main: Still creating... [3m20s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.google_project.main: Still creating... [3m30s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.google_project.main: Creation complete after 3m34s [id=projects/prj-b-seed-31ca]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["monitoring.googleapis.com"]: Creating...
module.seed_bootstrap.module.seed_project.module.project-factory.google_resource_manager_lien.lien[0]: Creating...
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["accesscontextmanager.googleapis.com"]: Creating...
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["iam.googleapis.com"]: Creating...
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["cloudasset.googleapis.com"]: Creating...
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["cloudbilling.googleapis.com"]: Creating...
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["serviceusage.googleapis.com"]: Creating...
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["logging.googleapis.com"]: Creating...
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["essentialcontacts.googleapis.com"]: Creating...
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["storage-api.googleapis.com"]: Creating...
module.seed_bootstrap.module.seed_project.module.project-factory.google_resource_manager_lien.lien[0]: Creation complete after 1s [id=p830013448499-ldf597632-f200-4bf9-8345-c7388b366ed8]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["compute.googleapis.com"]: Creating...
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["compute.googleapis.com"]: Creation complete after 3s [id=prj-b-seed-31ca/compute.googleapis.com]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["securitycenter.googleapis.com"]: Creating...

module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["iam.googleapis.com"]: Still creating... [10s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["accesscontextmanager.googleapis.com"]: Still creating... [10s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["cloudasset.googleapis.com"]: Still creating... [10s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["cloudbilling.googleapis.com"]: Still creating... [10s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["serviceusage.googleapis.com"]: Still creating... [10s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["essentialcontacts.googleapis.com"]: Still creating... [10s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["logging.googleapis.com"]: Still creating... [10s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["storage-api.googleapis.com"]: Still creating... [10s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["securitycenter.googleapis.com"]: Still creating... [10s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["serviceusage.googleapis.com"]: Still creating... [20s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["cloudbilling.googleapis.com"]: Still creating... [20s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["cloudasset.googleapis.com"]: Still creating... [20s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["essentialcontacts.googleapis.com"]: Still creating... [20s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["logging.googleapis.com"]: Still creating... [20s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["storage-api.googleapis.com"]: Still creating... [20s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["cloudbilling.googleapis.com"]: Creation complete after 21s [id=prj-b-seed-31ca/cloudbilling.googleapis.com]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["logging.googleapis.com"]: Creation complete after 21s [id=prj-b-seed-31ca/logging.googleapis.com]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["serviceusage.googleapis.com"]: Creation complete after 21s [id=prj-b-seed-31ca/serviceusage.googleapis.com]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["monitoring.googleapis.com"]: Creation complete after 21s [id=prj-b-seed-31ca/monitoring.googleapis.com]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["cloudkms.googleapis.com"]: Creating...
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["storage-api.googleapis.com"]: Creation complete after 21s [id=prj-b-seed-31ca/storage-api.googleapis.com]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["iam.googleapis.com"]: Creation complete after 21s [id=prj-b-seed-31ca/iam.googleapis.com]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["accesscontextmanager.googleapis.com"]: Creation complete after 21s [id=prj-b-seed-31ca/accesscontextmanager.googleapis.com]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["essentialcontacts.googleapis.com"]: Creation complete after 21s [id=prj-b-seed-31ca/essentialcontacts.googleapis.com]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["cloudasset.googleapis.com"]: Creation complete after 21s [id=prj-b-seed-31ca/cloudasset.googleapis.com]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["pubsub.googleapis.com"]: Creating...
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["billingbudgets.googleapis.com"]: Creating...
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["assuredworkloads.googleapis.com"]: Creating...
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["iamcredentials.googleapis.com"]: Creating...
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["bigquery.googleapis.com"]: Creating...
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["admin.googleapis.com"]: Creating...
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["cloudbuild.googleapis.com"]: Creating...
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["cloudresourcemanager.googleapis.com"]: Creating...
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["securitycenter.googleapis.com"]: Still creating... [20s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["securitycenter.googleapis.com"]: Creation complete after 20s [id=prj-b-seed-31ca/securitycenter.googleapis.com]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["iamcredentials.googleapis.com"]: Creation complete after 3s [id=prj-b-seed-31ca/iamcredentials.googleapis.com]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["servicenetworking.googleapis.com"]: Creating...
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["appengine.googleapis.com"]: Creating...

s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["pubsub.googleapis.com"]: Still creating... [10s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["billingbudgets.googleapis.com"]: Still creating... [10s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["assuredworkloads.googleapis.com"]: Still creating... [10s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["bigquery.googleapis.com"]: Still creating... [10s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["admin.googleapis.com"]: Still creating... [10s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["cloudbuild.googleapis.com"]: Still creating... [10s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["cloudresourcemanager.googleapis.com"]: Still creating... [10s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["appengine.googleapis.com"]: Still creating... [10s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["servicenetworking.googleapis.com"]: Still creating... [10s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["cloudkms.googleapis.com"]: Still creating... [20s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["assuredworkloads.googleapis.com"]: Still creating... [20s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["pubsub.googleapis.com"]: Still creating... [20s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["billingbudgets.googleapis.com"]: Still creating... [20s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["bigquery.googleapis.com"]: Still creating... [20s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["cloudbuild.googleapis.com"]: Still creating... [20s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["admin.googleapis.com"]: Still creating... [20s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["cloudresourcemanager.googleapis.com"]: Still creating... [20s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["servicenetworking.googleapis.com"]: Creation complete after 19s [id=prj-b-seed-31ca/servicenetworking.googleapis.com]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["assuredworkloads.googleapis.com"]: Creation complete after 22s [id=prj-b-seed-31ca/assuredworkloads.googleapis.com]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["appengine.googleapis.com"]: Creation complete after 19s [id=prj-b-seed-31ca/appengine.googleapis.com]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["admin.googleapis.com"]: Creation complete after 22s [id=prj-b-seed-31ca/admin.googleapis.com]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["cloudbuild.googleapis.com"]: Creation complete after 22s [id=prj-b-seed-31ca/cloudbuild.googleapis.com]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["cloudresourcemanager.googleapis.com"]: Creation complete after 22s [id=prj-b-seed-31ca/cloudresourcemanager.googleapis.com]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["bigquery.googleapis.com"]: Creation complete after 22s [id=prj-b-seed-31ca/bigquery.googleapis.com]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["cloudkms.googleapis.com"]: Creation complete after 22s [id=prj-b-seed-31ca/cloudkms.googleapis.com]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["billingbudgets.googleapis.com"]: Creation complete after 22s [id=prj-b-seed-31ca/billingbudgets.googleapis.com]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["pubsub.googleapis.com"]: Creation complete after 22s [id=prj-b-seed-31ca/pubsub.googleapis.com]
module.seed_bootstrap.module.seed_project.module.project-factory.google_project_default_service_accounts.default_service_accounts[0]: Creating...
module.seed_bootstrap.data.google_storage_project_service_account.gcs_account: Reading...
module.seed_bootstrap.module.enable_cross_project_service_account_usage.google_project_organization_policy.project_policy_boolean[0]: Creating...
module.seed_bootstrap.module.kms[0].google_kms_key_ring.key_ring: Creating...
google_service_account.terraform-env-sa["bootstrap"]: Creating...
google_service_account.terraform-env-sa["proj"]: Creating...
google_service_account.terraform-env-sa["org"]: Creating...
google_service_account.terraform-env-sa["env"]: Creating...
google_service_account.terraform-env-sa["net"]: Creating...
module.seed_bootstrap.data.google_storage_project_service_account.gcs_account: Read complete after 1s [id=service-830013448499@gs-project-accounts.iam.gserviceaccount.com]
module.seed_bootstrap.module.enable_cross_project_service_account_usage.google_project_organization_policy.project_policy_boolean[0]: Creation complete after 1s [id=prj-b-seed-31ca:constraints/iam.disableCrossProjectServiceAccountUsage]
╷
│ Error: Error creating service account: googleapi: Error 403: Identity and Access Management (IAM) API has not been used in project tef-olapp before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/iam.googleapis.com/overview?project=tef-olapp then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.
│ Details:
│ [
│   {
│     "@type": "type.googleapis.com/google.rpc.Help",
│     "links": [
│       {
│         "description": "Google developers console API activation",
│         "url": "https://console.developers.google.com/apis/api/iam.googleapis.com/overview?project=tef-olapp"
│       }
│     ]
│   },
│   {
│     "@type": "type.googleapis.com/google.rpc.ErrorInfo",
│     "domain": "googleapis.com",
│     "metadata": {
│       "consumer": "projects/tef-olapp",
│       "service": "iam.googleapis.com"
│     },
│     "reason": "SERVICE_DISABLED"
│   }
│ ]
│ , accessNotConfigured
│ 
│   with google_service_account.terraform-env-sa["env"],
│   on sa.tf line 140, in resource "google_service_account" "terraform-env-sa":
│  140: resource "google_service_account" "terraform-env-sa" {
│ 
╵
╷
│ Error: Error creating service account: googleapi: Error 403: Identity and Access Management (IAM) API has not been used in project tef-olapp before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/iam.googleapis.com/overview?project=tef-olapp then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.
│ Details:
│ [
│   {
│     "@type": "type.googleapis.com/google.rpc.Help",
│     "links": [
│       {
│         "description": "Google developers console API activation",
│         "url": "https://console.developers.google.com/apis/api/iam.googleapis.com/overview?project=tef-olapp"
│       }
│     ]
│   },
│   {
│     "@type": "type.googleapis.com/google.rpc.ErrorInfo",
│     "domain": "googleapis.com",
│     "metadata": {
│       "consumer": "projects/tef-olapp",
│       "service": "iam.googleapis.com"
│     },
│     "reason": "SERVICE_DISABLED"
│   }
│ ]
│ , accessNotConfigured
│ 
│   with google_service_account.terraform-env-sa["proj"],
│   on sa.tf line 140, in resource "google_service_account" "terraform-env-sa":
│  140: resource "google_service_account" "terraform-env-sa" {
│ 
╵
╷
│ Error: Error creating service account: googleapi: Error 403: Identity and Access Management (IAM) API has not been used in project tef-olapp before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/iam.googleapis.com/overview?project=tef-olapp then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.
│ Details:
│ [
│   {
│     "@type": "type.googleapis.com/google.rpc.Help",
│     "links": [
│       {
│         "description": "Google developers console API activation",
│         "url": "https://console.developers.google.com/apis/api/iam.googleapis.com/overview?project=tef-olapp"
│       }
│     ]
│   },
│   {
│     "@type": "type.googleapis.com/google.rpc.ErrorInfo",
│     "domain": "googleapis.com",
│     "metadata": {
│       "consumer": "projects/tef-olapp",
│       "service": "iam.googleapis.com"
│     },
│     "reason": "SERVICE_DISABLED"
│   }
│ ]
│ , accessNotConfigured
│ 
│   with google_service_account.terraform-env-sa["bootstrap"],
│   on sa.tf line 140, in resource "google_service_account" "terraform-env-sa":
│  140: resource "google_service_account" "terraform-env-sa" {
│ 
╵
╷
│ Error: Error creating service account: googleapi: Error 403: Identity and Access Management (IAM) API has not been used in project tef-olapp before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/iam.googleapis.com/overview?project=tef-olapp then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.
│ Details:
│ [
│   {
│     "@type": "type.googleapis.com/google.rpc.Help",
│     "links": [
│       {
│         "description": "Google developers console API activation",
│         "url": "https://console.developers.google.com/apis/api/iam.googleapis.com/overview?project=tef-olapp"
│       }
│     ]
│   },
│   {
│     "@type": "type.googleapis.com/google.rpc.ErrorInfo",
│     "domain": "googleapis.com",
│     "metadata": {
│       "consumer": "projects/tef-olapp",
│       "service": "iam.googleapis.com"
│     },
│     "reason": "SERVICE_DISABLED"
│   }
│ ]
│ , accessNotConfigured
│ 
│   with google_service_account.terraform-env-sa["org"],
│   on sa.tf line 140, in resource "google_service_account" "terraform-env-sa":
│  140: resource "google_service_account" "terraform-env-sa" {
│ 
╵
╷
│ Error: Error creating service account: googleapi: Error 403: Identity and Access Management (IAM) API has not been used in project tef-olapp before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/iam.googleapis.com/overview?project=tef-olapp then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.
│ Details:
│ [
│   {
│     "@type": "type.googleapis.com/google.rpc.Help",
│     "links": [
│       {
│         "description": "Google developers console API activation",
│         "url": "https://console.developers.google.com/apis/api/iam.googleapis.com/overview?project=tef-olapp"
│       }
│     ]
│   },
│   {
│     "@type": "type.googleapis.com/google.rpc.ErrorInfo",
│     "domain": "googleapis.com",
│     "metadata": {
│       "consumer": "projects/tef-olapp",
│       "service": "iam.googleapis.com"
│     },
│     "reason": "SERVICE_DISABLED"
│   }
│ ]
│ , accessNotConfigured
│ 
│   with google_service_account.terraform-env-sa["net"],
│   on sa.tf line 140, in resource "google_service_account" "terraform-env-sa":
│  140: resource "google_service_account" "terraform-env-sa" {
│ 
╵
╷
│ Error: Error creating KeyRing: googleapi: Error 403: Cloud Key Management Service (KMS) API has not been used in project tef-olapp before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/cloudkms.googleapis.com/overview?project=tef-olapp then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.
│ Details:
│ [
│   {
│     "@type": "type.googleapis.com/google.rpc.Help",
│     "links": [
│       {
│         "description": "Google developers console API activation",
│         "url": "https://console.developers.google.com/apis/api/cloudkms.googleapis.com/overview?project=tef-olapp"
│       }
│     ]
│   },
│   {
│     "@type": "type.googleapis.com/google.rpc.ErrorInfo",
│     "domain": "googleapis.com",
│     "metadata": {
│       "consumer": "projects/tef-olapp",
│       "service": "cloudkms.googleapis.com"
│     },
│     "reason": "SERVICE_DISABLED"
│   }
│ ]
│ 
│   with module.seed_bootstrap.module.kms[0].google_kms_key_ring.key_ring,
│   on .terraform/modules/seed_bootstrap.kms/main.tf line 21, in resource "google_kms_key_ring" "key_ring":
│   21: resource "google_kms_key_ring" "key_ring" {
│ 
╵
╷
│ Error: error listing service accounts on project prj-b-seed-31ca: failed to list service accounts on project "prj-b-seed-31ca": googleapi: Error 403: Identity and Access Management (IAM) API has not been used in project tef-olapp before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/iam.googleapis.com/overview?project=tef-olapp then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.
│ Details:
│ [
│   {
│     "@type": "type.googleapis.com/google.rpc.Help",
│     "links": [
│       {
│         "description": "Google developers console API activation",
│         "url": "https://console.developers.google.com/apis/api/iam.googleapis.com/overview?project=tef-olapp"
│       }
│     ]
│   },
│   {
│     "@type": "type.googleapis.com/google.rpc.ErrorInfo",
│     "domain": "googleapis.com",
│     "metadata": {
│       "consumer": "projects/tef-olapp",
│       "service": "iam.googleapis.com"
│     },
│     "reason": "SERVICE_DISABLED"
│   }
│ ]
│ , accessNotConfigured
│ 
│   with module.seed_bootstrap.module.seed_project.module.project-factory.google_project_default_service_accounts.default_service_accounts[0],
│   on .terraform/modules/seed_bootstrap.seed_project/modules/core_project_factory/main.tf line 134, in resource "google_project_default_service_accounts" "default_service_accounts":
│  134: resource "google_project_default_service_accounts" "default_service_accounts" {
│ 
╵

1237 need iam api -

michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ gcloud services enable iam.googleapis.com
Operation "operations/acat.p2-153288813308-7f675593-6ea2-4bcc-ac0c-09b4d227de62" finished successfully.

wait 5 min - then retry apply raised https://github.com/terraform-google-modules/terraform-example-foundation/issues/1140

init, plan, 
Plan: 223 to add, 0 to change, 0 to destroy.

apply
1251

michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ terraform apply bootstrap.tfplan
module.seed_bootstrap.module.seed_project.module.project-factory.google_project_default_service_accounts.default_service_accounts[0]: Creating...
google_service_account.terraform-env-sa["bootstrap"]: Creating...
module.seed_bootstrap.module.kms[0].google_kms_key_ring.key_ring: Creating...
google_service_account.terraform-env-sa["proj"]: Creating...
google_service_account.terraform-env-sa["org"]: Creating...
google_service_account.terraform-env-sa["net"]: Creating...
google_service_account.terraform-env-sa["env"]: Creating...
module.seed_bootstrap.module.seed_project.module.project-factory.google_project_default_service_accounts.default_service_accounts[0]: Creation complete after 1s [id=projects/prj-b-seed-31ca]
google_service_account.terraform-env-sa["proj"]: Creation complete after 1s [id=projects/prj-b-seed-31ca/serviceAccounts/sa-terraform-proj@prj-b-seed-31ca.iam.gserviceaccount.com]
google_service_account.terraform-env-sa["org"]: Creation complete after 2s [id=projects/prj-b-seed-31ca/serviceAccounts/sa-terraform-org@prj-b-seed-31ca.iam.gserviceaccount.com]
google_service_account.terraform-env-sa["bootstrap"]: Creation complete after 2s [id=projects/prj-b-seed-31ca/serviceAccounts/sa-terraform-bootstrap@prj-b-seed-31ca.iam.gserviceaccount.com]
google_service_account.terraform-env-sa["env"]: Creation complete after 2s [id=projects/prj-b-seed-31ca/serviceAccounts/sa-terraform-env@prj-b-seed-31ca.iam.gserviceaccount.com]
google_service_account.terraform-env-sa["net"]: Creation complete after 2s [id=projects/prj-b-seed-31ca/serviceAccounts/sa-terraform-net@prj-b-seed-31ca.iam.gserviceaccount.com]
module.parent_iam_member["proj"].google_folder_iam_member.folder_parent_iam["roles/compute.xpnAdmin"]: Creating...
google_billing_account_iam_member.tf_billing_user["proj"]: Creating...
module.parent_iam_member["proj"].google_folder_iam_member.folder_parent_iam["roles/resourcemanager.folderAdmin"]: Creating...
google_billing_account_iam_member.tf_billing_user["net"]: Creating...
module.parent_iam_member["net"].google_folder_iam_member.folder_parent_iam["roles/resourcemanager.folderViewer"]: Creating...
google_billing_account_iam_member.tf_billing_user["env"]: Creating...
module.parent_iam_member["org"].google_folder_iam_member.folder_parent_iam["roles/resourcemanager.folderAdmin"]: Creating...
module.seed_bootstrap.google_folder_iam_binding.project_creator[0]: Creating...
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/logging.configWriter"]: Creating...
google_billing_account_iam_member.tf_billing_user["org"]: Creating...
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/logging.configWriter"]: Creation complete after 5s [id=630259462753/roles/logging.configWriter/serviceAccount:sa-terraform-org@prj-b-seed-31ca.iam.gserviceaccount.com]
module.org_iam_member["net"].google_organization_iam_member.org_parent_iam["roles/compute.xpnAdmin"]: Creating...
module.parent_iam_member["proj"].google_folder_iam_member.folder_parent_iam["roles/compute.xpnAdmin"]: Creation complete after 5s [id=folders/1078109772786/roles/compute.xpnAdmin/serviceAccount:sa-terraform-proj@prj-b-seed-31ca.iam.gserviceaccount.com]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/securitycenter.sourcesEditor"]: Creating...
google_billing_account_iam_member.tf_billing_user["org"]: Creation complete after 9s [id=012EDD-5AD5ED-ECFF0B/roles/billing.user/serviceAccount:sa-terraform-org@prj-b-seed-31ca.iam.gserviceaccount.com]
module.org_iam_member["env"].google_organization_iam_member.org_parent_iam["roles/browser"]: Creating...
module.org_iam_member["net"].google_organization_iam_member.org_parent_iam["roles/compute.xpnAdmin"]: Creation complete after 4s [id=630259462753/roles/compute.xpnAdmin/serviceAccount:sa-terraform-net@prj-b-seed-31ca.iam.gserviceaccount.com]
module.parent_iam_member["proj"].google_folder_iam_member.folder_parent_iam["roles/artifactregistry.admin"]: Creating...
google_billing_account_iam_member.tf_billing_user["env"]: Still creating... [10s elapsed]
module.parent_iam_member["org"].google_folder_iam_member.folder_parent_iam["roles/resourcemanager.folderAdmin"]: Still creating... [10s elapsed]
module.parent_iam_member["net"].google_folder_iam_member.folder_parent_iam["roles/resourcemanager.folderViewer"]: Still creating... [10s elapsed]
google_billing_account_iam_member.tf_billing_user["proj"]: Still creating... [10s elapsed]
module.seed_bootstrap.google_folder_iam_binding.project_creator[0]: Still creating... [10s elapsed]
google_billing_account_iam_member.tf_billing_user["net"]: Still creating... [10s elapsed]
module.parent_iam_member["proj"].google_folder_iam_member.folder_parent_iam["roles/resourcemanager.folderAdmin"]: Still creating... [10s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/securitycenter.sourcesEditor"]: Still creating... [10s elapsed]
google_billing_account_iam_member.tf_billing_user["proj"]: Creation complete after 18s [id=012EDD-5AD5ED-ECFF0B/roles/billing.user/serviceAccount:sa-terraform-proj@prj-b-seed-31ca.iam.gserviceaccount.com]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/essentialcontacts.admin"]: Creating...
google_billing_account_iam_member.tf_billing_user["net"]: Creation complete after 18s [id=012EDD-5AD5ED-ECFF0B/roles/billing.user/serviceAccount:sa-terraform-net@prj-b-seed-31ca.iam.gserviceaccount.com]
module.parent_iam_member["bootstrap"].google_folder_iam_member.folder_parent_iam["roles/resourcemanager.folderAdmin"]: Creating...
google_billing_account_iam_member.tf_billing_user["env"]: Creation complete after 18s [id=012EDD-5AD5ED-ECFF0B/roles/billing.user/serviceAccount:sa-terraform-env@prj-b-seed-31ca.iam.gserviceaccount.com]
module.org_iam_member["env"].google_organization_iam_member.org_parent_iam["roles/browser"]: Creation complete after 9s [id=630259462753/roles/browser/serviceAccount:sa-terraform-env@prj-b-seed-31ca.iam.gserviceaccount.com]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/securitycenter.notificationConfigEditor"]: Creating...
module.parent_iam_member["net"].google_folder_iam_member.folder_parent_iam["roles/dns.admin"]: Creating...
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/securitycenter.sourcesEditor"]: Creation complete after 13s [id=630259462753/roles/securitycenter.sourcesEditor/serviceAccount:sa-terraform-org@prj-b-seed-31ca.iam.gserviceaccount.com]
module.org_iam_member["bootstrap"].google_organization_iam_member.org_parent_iam["roles/browser"]: Creating...
module.parent_iam_member["proj"].google_folder_iam_member.folder_parent_iam["roles/artifactregistry.admin"]: Still creating... [10s elapsed]
module.parent_iam_member["net"].google_folder_iam_member.folder_parent_iam["roles/resourcemanager.folderViewer"]: Still creating... [20s elapsed]
module.parent_iam_member["org"].google_folder_iam_member.folder_parent_iam["roles/resourcemanager.folderAdmin"]: Still creating... [20s elapsed]
module.seed_bootstrap.google_folder_iam_binding.project_creator[0]: Still creating... [20s elapsed]
module.parent_iam_member["proj"].google_folder_iam_member.folder_parent_iam["roles/resourcemanager.folderAdmin"]: Still creating... [20s elapsed]
module.parent_iam_member["org"].google_folder_iam_member.folder_parent_iam["roles/resourcemanager.folderAdmin"]: Creation complete after 27s [id=folders/1078109772786/roles/resourcemanager.folderAdmin/serviceAccount:sa-terraform-org@prj-b-seed-31ca.iam.gserviceaccount.com]
module.parent_iam_member["env"].google_folder_iam_member.folder_parent_iam["roles/resourcemanager.folderAdmin"]: Creating...
module.parent_iam_member["net"].google_folder_iam_member.folder_parent_iam["roles/resourcemanager.folderViewer"]: Creation complete after 27s [id=folders/1078109772786/roles/resourcemanager.folderViewer/serviceAccount:sa-terraform-net@prj-b-seed-31ca.iam.gserviceaccount.com]
module.org_iam_member["env"].google_organization_iam_member.org_parent_iam["roles/assuredworkloads.admin"]: Creating...
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/essentialcontacts.admin"]: Still creating... [10s elapsed]
module.parent_iam_member["bootstrap"].google_folder_iam_member.folder_parent_iam["roles/resourcemanager.folderAdmin"]: Still creating... [10s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/securitycenter.notificationConfigEditor"]: Still creating... [10s elapsed]
module.parent_iam_member["net"].google_folder_iam_member.folder_parent_iam["roles/dns.admin"]: Still creating... [10s elapsed]
module.org_iam_member["bootstrap"].google_organization_iam_member.org_parent_iam["roles/browser"]: Still creating... [10s elapsed]
module.parent_iam_member["proj"].google_folder_iam_member.folder_parent_iam["roles/artifactregistry.admin"]: Still creating... [20s elapsed]
module.seed_bootstrap.google_folder_iam_binding.project_creator[0]: Still creating... [30s elapsed]
module.parent_iam_member["proj"].google_folder_iam_member.folder_parent_iam["roles/resourcemanager.folderAdmin"]: Still creating... [30s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/essentialcontacts.admin"]: Creation complete after 13s [id=630259462753/roles/essentialcontacts.admin/serviceAccount:sa-terraform-org@prj-b-seed-31ca.iam.gserviceaccount.com]
module.parent_iam_member["net"].google_folder_iam_member.folder_parent_iam["roles/compute.orgSecurityResourceAdmin"]: Creating...
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/securitycenter.notificationConfigEditor"]: Creation complete after 13s [id=630259462753/roles/securitycenter.notificationConfigEditor/serviceAccount:sa-terraform-org@prj-b-seed-31ca.iam.gserviceaccount.com]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.tagAdmin"]: Creating...
module.seed_bootstrap.google_folder_iam_binding.project_creator[0]: Creation complete after 36s [id=folders/1078109772786/roles/resourcemanager.projectCreator]
module.parent_iam_member["proj"].google_folder_iam_member.folder_parent_iam["roles/compute.networkAdmin"]: Creating...
module.org_iam_member["bootstrap"].google_organization_iam_member.org_parent_iam["roles/browser"]: Creation complete after 18s [id=630259462753/roles/browser/serviceAccount:sa-terraform-bootstrap@prj-b-seed-31ca.iam.gserviceaccount.com]
module.parent_iam_member["net"].google_folder_iam_member.folder_parent_iam["roles/compute.networkAdmin"]: Creating...
module.parent_iam_member["proj"].google_folder_iam_member.folder_parent_iam["roles/resourcemanager.folderAdmin"]: Creation complete after 36s [id=folders/1078109772786/roles/resourcemanager.folderAdmin/serviceAccount:sa-terraform-proj@prj-b-seed-31ca.iam.gserviceaccount.com]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.organizationAdmin"]: Creating...
module.parent_iam_member["proj"].google_folder_iam_member.folder_parent_iam["roles/artifactregistry.admin"]: Creation complete after 27s [id=folders/1078109772786/roles/artifactregistry.admin/serviceAccount:sa-terraform-proj@prj-b-seed-31ca.iam.gserviceaccount.com]
module.parent_iam_member["net"].google_folder_iam_member.folder_parent_iam["roles/compute.securityAdmin"]: Creating...
module.parent_iam_member["env"].google_folder_iam_member.folder_parent_iam["roles/resourcemanager.folderAdmin"]: Still creating... [10s elapsed]
module.org_iam_member["env"].google_organization_iam_member.org_parent_iam["roles/assuredworkloads.admin"]: Still creating... [10s elapsed]
module.parent_iam_member["bootstrap"].google_folder_iam_member.folder_parent_iam["roles/resourcemanager.folderAdmin"]: Still creating... [20s elapsed]
module.parent_iam_member["net"].google_folder_iam_member.folder_parent_iam["roles/dns.admin"]: Still creating... [20s elapsed]
module.org_iam_member["env"].google_organization_iam_member.org_parent_iam["roles/assuredworkloads.admin"]: Creation complete after 13s [id=630259462753/roles/assuredworkloads.admin/serviceAccount:sa-terraform-env@prj-b-seed-31ca.iam.gserviceaccount.com]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.tagUser"]: Creating...
module.parent_iam_member["bootstrap"].google_folder_iam_member.folder_parent_iam["roles/resourcemanager.folderAdmin"]: Creation complete after 23s [id=folders/1078109772786/roles/resourcemanager.folderAdmin/serviceAccount:sa-terraform-bootstrap@prj-b-seed-31ca.iam.gserviceaccount.com]
module.org_iam_member["net"].google_organization_iam_member.org_parent_iam["roles/browser"]: Creating...
module.parent_iam_member["net"].google_folder_iam_member.folder_parent_iam["roles/compute.orgSecurityResourceAdmin"]: Still creating... [10s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.tagAdmin"]: Still creating... [10s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.tagAdmin"]: Creation complete after 14s [id=630259462753/roles/resourcemanager.tagAdmin/serviceAccount:sa-terraform-org@prj-b-seed-31ca.iam.gserviceaccount.com]

dule.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.tagAdmin"]: Still creating... [10s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.tagAdmin"]: Creation complete after 14s [id=630259462753/roles/resourcemanager.tagAdmin/serviceAccount:sa-terraform-org@prj-b-seed-31ca.iam.gserviceaccount.com]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/accesscontextmanager.policyAdmin"]: Creating...
module.parent_iam_member["net"].google_folder_iam_member.folder_parent_iam["roles/dns.admin"]: Creation complete after 27s [id=folders/1078109772786/roles/dns.admin/serviceAccount:sa-terraform-net@prj-b-seed-31ca.iam.gserviceaccount.com]
module.org_iam_member["proj"].google_organization_iam_member.org_parent_iam["roles/browser"]: Creating...
module.parent_iam_member["proj"].google_folder_iam_member.folder_parent_iam["roles/compute.networkAdmin"]: Still creating... [10s elapsed]
module.parent_iam_member["net"].google_folder_iam_member.folder_parent_iam["roles/compute.networkAdmin"]: Still creating... [10s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.organizationAdmin"]: Still creating... [10s elapsed]
module.parent_iam_member["net"].google_folder_iam_member.folder_parent_iam["roles/compute.securityAdmin"]: Still creating... [10s elapsed]
module.parent_iam_member["env"].google_folder_iam_member.folder_parent_iam["roles/resourcemanager.folderAdmin"]: Still creating... [20s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.tagUser"]: Still creating... [10s elapsed]
module.org_iam_member["net"].google_organization_iam_member.org_parent_iam["roles/browser"]: Still creating... [10s elapsed]
module.parent_iam_member["net"].google_folder_iam_member.folder_parent_iam["roles/compute.orgSecurityResourceAdmin"]: Still creating... [20s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.organizationAdmin"]: Creation complete after 17s [id=630259462753/roles/resourcemanager.organizationAdmin/serviceAccount:sa-terraform-org@prj-b-seed-31ca.iam.gserviceaccount.com]
module.org_iam_member["proj"].google_organization_iam_member.org_parent_iam["roles/accesscontextmanager.policyAdmin"]: Creating...
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/accesscontextmanager.policyAdmin"]: Still creating... [10s elapsed]
module.org_iam_member["proj"].google_organization_iam_member.org_parent_iam["roles/browser"]: Still creating... [10s elapsed]
module.parent_iam_member["proj"].google_folder_iam_member.folder_parent_iam["roles/compute.networkAdmin"]: Still creating... [20s elapsed]
module.parent_iam_member["net"].google_folder_iam_member.folder_parent_iam["roles/compute.networkAdmin"]: Still creating... [20s elapsed]
module.parent_iam_member["net"].google_folder_iam_member.folder_parent_iam["roles/compute.securityAdmin"]: Still creating... [20s elapsed]
module.parent_iam_member["env"].google_folder_iam_member.folder_parent_iam["roles/resourcemanager.folderAdmin"]: Still creating... [30s elapsed]
module.parent_iam_member["env"].google_folder_iam_member.folder_parent_iam["roles/resourcemanager.folderAdmin"]: Creation complete after 31s [id=folders/1078109772786/roles/resourcemanager.folderAdmin/serviceAccount:sa-terraform-env@prj-b-seed-31ca.iam.gserviceaccount.com]
module.org_iam_member["bootstrap"].google_organization_iam_member.org_parent_iam["roles/serviceusage.serviceUsageConsumer"]: Creating...
module.parent_iam_member["net"].google_folder_iam_member.folder_parent_iam["roles/compute.orgSecurityResourceAdmin"]: Creation complete after 27s [id=folders/1078109772786/roles/compute.orgSecurityResourceAdmin/serviceAccount:sa-terraform-net@prj-b-seed-31ca.iam.gserviceaccount.com]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/cloudasset.owner"]: Creating...
module.parent_iam_member["proj"].google_folder_iam_member.folder_parent_iam["roles/compute.networkAdmin"]: Creation complete after 23s [id=folders/1078109772786/roles/compute.networkAdmin/serviceAccount:sa-terraform-proj@prj-b-seed-31ca.iam.gserviceaccount.com]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/browser"]: Creating...
module.parent_iam_member["net"].google_folder_iam_member.folder_parent_iam["roles/compute.networkAdmin"]: Creation complete after 23s [id=folders/1078109772786/roles/compute.networkAdmin/serviceAccount:sa-terraform-net@prj-b-seed-31ca.iam.gserviceaccount.com]
module.org_iam_member["proj"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.organizationAdmin"]: Creating...
module.parent_iam_member["net"].google_folder_iam_member.folder_parent_iam["roles/compute.securityAdmin"]: Creation complete after 23s [id=folders/1078109772786/roles/compute.securityAdmin/serviceAccount:sa-terraform-net@prj-b-seed-31ca.iam.gserviceaccount.com]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.organizationViewer"]: Creating...
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.tagUser"]: Still creating... [20s elapsed]
module.org_iam_member["net"].google_organization_iam_member.org_parent_iam["roles/browser"]: Still creating... [20s elapsed]

odule.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.tagUser"]: Still creating... [20s elapsed]
module.org_iam_member["net"].google_organization_iam_member.org_parent_iam["roles/browser"]: Still creating... [20s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.tagUser"]: Creation complete after 22s [id=630259462753/roles/resourcemanager.tagUser/serviceAccount:sa-terraform-org@prj-b-seed-31ca.iam.gserviceaccount.com]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/orgpolicy.policyAdmin"]: Creating...
module.org_iam_member["net"].google_organization_iam_member.org_parent_iam["roles/browser"]: Creation complete after 22s [id=630259462753/roles/browser/serviceAccount:sa-terraform-net@prj-b-seed-31ca.iam.gserviceaccount.com]
module.org_iam_member["env"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.tagUser"]: Creating...
module.org_iam_member["proj"].google_organization_iam_member.org_parent_iam["roles/accesscontextmanager.policyAdmin"]: Still creating... [10s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/accesscontextmanager.policyAdmin"]: Still creating... [20s elapsed]
module.org_iam_member["proj"].google_organization_iam_member.org_parent_iam["roles/browser"]: Still creating... [20s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/accesscontextmanager.policyAdmin"]: Creation complete after 22s [id=630259462753/roles/accesscontextmanager.policyAdmin/serviceAccount:sa-terraform-org@prj-b-seed-31ca.iam.gserviceaccount.com]
module.org_iam_member["proj"].google_organization_iam_member.org_parent_iam["roles/serviceusage.serviceUsageConsumer"]: Creating...
module.org_iam_member["bootstrap"].google_organization_iam_member.org_parent_iam["roles/serviceusage.serviceUsageConsumer"]: Still creating... [10s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/cloudasset.owner"]: Still creating... [10s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/browser"]: Still creating... [10s elapsed]
module.org_iam_member["proj"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.organizationAdmin"]: Still creating... [10s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.organizationViewer"]: Still creating... [10s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/orgpolicy.policyAdmin"]: Still creating... [10s elapsed]
module.org_iam_member["env"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.tagUser"]: Still creating... [10s elapsed]
module.org_iam_member["proj"].google_organization_iam_member.org_parent_iam["roles/accesscontextmanager.policyAdmin"]: Still creating... [20s elapsed]
module.org_iam_member["proj"].google_organization_iam_member.org_parent_iam["roles/browser"]: Still creating... [30s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/cloudasset.owner"]: Still creating... [20s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/browser"]: Still creating... [20s elapsed]
module.org_iam_member["proj"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.organizationAdmin"]: Still creating... [20s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.organizationViewer"]: Still creating... [20s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/orgpolicy.policyAdmin"]: Still creating... [20s elapsed]
module.org_iam_member["env"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.tagUser"]: Still creating... [20s elapsed]
module.org_iam_member["proj"].google_organization_iam_member.org_parent_iam["roles/accesscontextmanager.policyAdmin"]: Still creating... [30s elapsed]
module.org_iam_member["proj"].google_organization_iam_member.org_parent_iam["roles/browser"]: Still creating... [40s elapsed]
module.org_iam_member["proj"].google_organization_iam_member.org_parent_iam["roles/serviceusage.serviceUsageConsumer"]: Still creating... [20s elapsed]
module.org_iam_member["bootstrap"].google_organization_iam_member.org_parent_iam["roles/serviceusage.serviceUsageConsumer"]: Still creating... [30s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/cloudasset.owner"]: Still creating... [30s elapsed]
module.org_iam_member["proj"].google_organization_iam_member.org_parent_iam["roles/browser"]: Creation complete after 44s [id=630259462753/roles/browser/serviceAccount:sa-terraform-proj@prj-b-seed-31ca.iam.gserviceaccount.com]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/browser"]: Still creating... [30s elapsed]
google_billing_account_iam_member.tf_billing_user["bootstrap"]: Creating...
module.org_iam_member["proj"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.organizationAdmin"]: Still creating... [30s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.organizationViewer"]: Still creating... [30s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/orgpolicy.policyAdmin"]: Still creating... [30s elapsed]
module.org_iam_member["env"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.tagUser"]: Still creating... [30s elapsed]
google_billing_account_iam_member.tf_billing_user["bootstrap"]: Creation complete after 4s [id=012EDD-5AD5ED-ECFF0B/roles/billing.user/serviceAccount:sa-terraform-bootstrap@prj-b-seed-31ca.iam.gserviceaccount.com]
module.org_iam_member["net"].google_organization_iam_member.org_parent_iam["roles/accesscontextmanager.policyAdmin"]: Creating...
module.org_iam_member["proj"].google_organization_iam_member.org_parent_iam["roles/accesscontextmanager.policyAdmin"]: Still creating... [40s elapsed]

ntextmanager.policyAdmin/serviceAccount:sa-terraform-proj@prj-b-seed-31ca.iam.gserviceaccount.com]
module.org_iam_member["bootstrap"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.organizationAdmin"]: Creating...
module.org_iam_member["proj"].google_organization_iam_member.org_parent_iam["roles/serviceusage.serviceUsageConsumer"]: Still creating... [30s elapsed]
module.org_iam_member["bootstrap"].google_organization_iam_member.org_parent_iam["roles/serviceusage.serviceUsageConsumer"]: Still creating... [40s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/cloudasset.owner"]: Still creating... [40s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/browser"]: Still creating... [40s elapsed]
module.org_iam_member["proj"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.organizationAdmin"]: Still creating... [40s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.organizationViewer"]: Still creating... [40s elapsed]
module.org_iam_member["bootstrap"].google_organization_iam_member.org_parent_iam["roles/serviceusage.serviceUsageConsumer"]: Creation complete after 43s [id=630259462753/roles/serviceusage.serviceUsageConsumer/serviceAccount:sa-terraform-bootstrap@prj-b-seed-31ca.iam.gserviceaccount.com]
module.parent_iam_member["net"].google_folder_iam_member.folder_parent_iam["roles/compute.orgSecurityPolicyAdmin"]: Creating...
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/cloudasset.owner"]: Creation complete after 43s [id=630259462753/roles/cloudasset.owner/serviceAccount:sa-terraform-org@prj-b-seed-31ca.iam.gserviceaccount.com]
module.org_iam_member["bootstrap"].google_organization_iam_member.org_parent_iam["roles/accesscontextmanager.policyAdmin"]: Creating...
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/browser"]: Creation complete after 42s [id=630259462753/roles/browser/serviceAccount:sa-terraform-org@prj-b-seed-31ca.iam.gserviceaccount.com]
module.seed_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/storage.admin"]: Creating...
module.org_iam_member["proj"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.organizationAdmin"]: Creation complete after 43s [id=630259462753/roles/resourcemanager.organizationAdmin/serviceAccount:sa-terraform-proj@prj-b-seed-31ca.iam.gserviceaccount.com]
module.seed_project_iam_member["env"].google_project_iam_member.project_parent_iam["roles/storage.objectAdmin"]: Creating...
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.organizationViewer"]: Creation complete after 43s [id=630259462753/roles/resourcemanager.organizationViewer/serviceAccount:sa-terraform-org@prj-b-seed-31ca.iam.gserviceaccount.com]
module.seed_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/resourcemanager.projectDeleter"]: Creating...
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/orgpolicy.policyAdmin"]: Creation complete after 40s [id=630259462753/roles/orgpolicy.policyAdmin/serviceAccount:sa-terraform-org@prj-b-seed-31ca.iam.gserviceaccount.com]
module.seed_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/cloudkms.admin"]: Creating...
module.org_iam_member["env"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.tagUser"]: Still creating... [40s elapsed]
module.org_iam_member["net"].google_organization_iam_member.org_parent_iam["roles/accesscontextmanager.policyAdmin"]: Still creating... [10s elapsed]
module.parent_iam_member["net"].google_folder_iam_member.folder_parent_iam["roles/compute.orgSecurityPolicyAdmin"]: Creation complete after 5s [id=folders/1078109772786/roles/compute.orgSecurityPolicyAdmin/serviceAccount:sa-terraform-net@prj-b-seed-31ca.iam.gserviceaccount.com]
module.seed_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/iam.serviceAccountAdmin"]: Creating...
module.org_iam_member["env"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.tagUser"]: Creation complete after 43s [id=630259462753/roles/resourcemanager.tagUser/serviceAccount:sa-terraform-env@prj-b-seed-31ca.iam.gserviceaccount.com]
module.seed_project_iam_member["proj"].google_project_iam_member.project_parent_iam["roles/storage.objectAdmin"]: Creating...
module.org_iam_member["bootstrap"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.organizationAdmin"]: Still creating... [10s elapsed]
module.org_iam_member["proj"].google_organization_iam_member.org_parent_iam["roles/serviceusage.serviceUsageConsumer"]: Still creating... [40s elapsed]

odule.org_iam_member["proj"].google_organization_iam_member.org_parent_iam["roles/serviceusage.serviceUsageConsumer"]: Still creating... [40s elapsed]
module.seed_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/cloudkms.admin"]: Creation complete after 7s [id=prj-b-seed-31ca/roles/cloudkms.admin/serviceAccount:sa-terraform-bootstrap@prj-b-seed-31ca.iam.gserviceaccount.com]
module.seed_project_iam_member["net"].google_project_iam_member.project_parent_iam["roles/storage.objectAdmin"]: Creating...
module.org_iam_member["proj"].google_organization_iam_member.org_parent_iam["roles/serviceusage.serviceUsageConsumer"]: Creation complete after 44s [id=630259462753/roles/serviceusage.serviceUsageConsumer/serviceAccount:sa-terraform-proj@prj-b-seed-31ca.iam.gserviceaccount.com]
module.seed_project_iam_member["org"].google_project_iam_member.project_parent_iam["roles/storage.objectAdmin"]: Creating...
module.org_iam_member["bootstrap"].google_organization_iam_member.org_parent_iam["roles/accesscontextmanager.policyAdmin"]: Still creating... [10s elapsed]
module.seed_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/storage.admin"]: Still creating... [10s elapsed]
module.seed_project_iam_member["env"].google_project_iam_member.project_parent_iam["roles/storage.objectAdmin"]: Still creating... [10s elapsed]
module.seed_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/resourcemanager.projectDeleter"]: Still creating... [10s elapsed]
module.org_iam_member["net"].google_organization_iam_member.org_parent_iam["roles/accesscontextmanager.policyAdmin"]: Still creating... [20s elapsed]
module.seed_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/storage.admin"]: Creation complete after 13s [id=prj-b-seed-31ca/roles/storage.admin/serviceAccount:sa-terraform-bootstrap@prj-b-seed-31ca.iam.gserviceaccount.com]
google_billing_account_iam_member.billing_admin_user["net"]: Creating...
module.seed_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/resourcemanager.projectDeleter"]: Creation complete after 12s [id=prj-b-seed-31ca/roles/resourcemanager.projectDeleter/serviceAccount:sa-terraform-bootstrap@prj-b-seed-31ca.iam.gserviceaccount.com]
google_billing_account_iam_member.billing_admin_user["proj"]: Creating...
module.seed_project_iam_member["env"].google_project_iam_member.project_parent_iam["roles/storage.objectAdmin"]: Creation complete after 12s [id=prj-b-seed-31ca/roles/storage.objectAdmin/serviceAccount:sa-terraform-env@prj-b-seed-31ca.iam.gserviceaccount.com]
google_billing_account_iam_member.billing_admin_user["org"]: Creating...
module.org_iam_member["net"].google_organization_iam_member.org_parent_iam["roles/accesscontextmanager.policyAdmin"]: Creation complete after 22s [id=630259462753/roles/accesscontextmanager.policyAdmin/serviceAccount:sa-terraform-net@prj-b-seed-31ca.iam.gserviceaccount.com]
google_billing_account_iam_member.billing_admin_user["bootstrap"]: Creating...
module.org_iam_member["bootstrap"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.organizationAdmin"]: Creation complete after 18s [id=630259462753/roles/resourcemanager.organizationAdmin/serviceAccount:sa-terraform-bootstrap@prj-b-seed-31ca.iam.gserviceaccount.com]
google_billing_account_iam_member.billing_admin_user["env"]: Creating...
module.org_iam_member["bootstrap"].google_organization_iam_member.org_parent_iam["roles/accesscontextmanager.policyAdmin"]: Creation complete after 15s [id=630259462753/roles/accesscontextmanager.policyAdmin/serviceAccount:sa-terraform-bootstrap@prj-b-seed-31ca.iam.gserviceaccount.com]
module.seed_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/iam.serviceAccountAdmin"]: Still creating... [10s elapsed]
module.seed_project_iam_member["proj"].google_project_iam_member.project_parent_iam["roles/storage.objectAdmin"]: Still creating... [10s elapsed]
google_billing_account_iam_member.billing_admin_user["net"]: Creation complete after 4s [id=012EDD-5AD5ED-ECFF0B/roles/billing.admin/serviceAccount:sa-terraform-net@prj-b-seed-31ca.iam.gserviceaccount.com]
module.seed_project_iam_member["proj"].google_project_iam_member.project_parent_iam["roles/storage.objectAdmin"]: Creation complete after 12s [id=prj-b-seed-31ca/roles/storage.objectAdmin/serviceAccount:sa-terraform-proj@prj-b-seed-31ca.iam.gserviceaccount.com]
module.seed_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/iam.serviceAccountAdmin"]: Creation complete after 12s [id=prj-b-seed-31ca/roles/iam.serviceAccountAdmin/serviceAccount:sa-terraform-bootstrap@prj-b-seed-31ca.iam.gserviceaccount.com]
module.seed_project_iam_member["org"].google_project_iam_member.project_parent_iam["roles/storage.objectAdmin"]: Creation complete after 7s [id=prj-b-seed-31ca/roles/storage.objectAdmin/serviceAccount:sa-terraform-org@prj-b-seed-31ca.iam.gserviceaccount.com]
module.seed_project_iam_member["net"].google_project_iam_member.project_parent_iam["roles/storage.objectAdmin"]: Creation complete after 10s [id=prj-b-seed-31ca/roles/storage.objectAdmin/serviceAccount:sa-terraform-net@prj-b-seed-31ca.iam.gserviceaccount.com]

google_billing_account_iam_member.billing_admin_user["proj"]: Still creating... [10s elapsed]
google_billing_account_iam_member.billing_admin_user["org"]: Still creating... [10s elapsed]
google_billing_account_iam_member.billing_admin_user["bootstrap"]: Still creating... [10s elapsed]
google_billing_account_iam_member.billing_admin_user["env"]: Still creating... [10s elapsed]
google_billing_account_iam_member.billing_admin_user["proj"]: Still creating... [20s elapsed]
google_billing_account_iam_member.billing_admin_user["org"]: Still creating... [20s elapsed]
google_billing_account_iam_member.billing_admin_user["proj"]: Creation complete after 21s [id=012EDD-5AD5ED-ECFF0B/roles/billing.admin/serviceAccount:sa-terraform-proj@prj-b-seed-31ca.iam.gserviceaccount.com]
google_billing_account_iam_member.billing_admin_user["org"]: Creation complete after 21s [id=012EDD-5AD5ED-ECFF0B/roles/billing.admin/serviceAccount:sa-terraform-org@prj-b-seed-31ca.iam.gserviceaccount.com]
google_billing_account_iam_member.billing_admin_user["bootstrap"]: Creation complete after 20s [id=012EDD-5AD5ED-ECFF0B/roles/billing.admin/serviceAccount:sa-terraform-bootstrap@prj-b-seed-31ca.iam.gserviceaccount.com]
google_billing_account_iam_member.billing_admin_user["env"]: Creation complete after 20s [id=012EDD-5AD5ED-ECFF0B/roles/billing.admin/serviceAccount:sa-terraform-env@prj-b-seed-31ca.iam.gserviceaccount.com]
╷
│ Error: Error creating KeyRing: googleapi: Error 403: Cloud Key Management Service (KMS) API has not been used in project tef-olapp before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/cloudkms.googleapis.com/overview?project=tef-olapp then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.
│ Details:
│ [
│   {
│     "@type": "type.googleapis.com/google.rpc.Help",
│     "links": [
│       {
│         "description": "Google developers console API activation",
│         "url": "https://console.developers.google.com/apis/api/cloudkms.googleapis.com/overview?project=tef-olapp"
│       }
│     ]
│   },
│   {
│     "@type": "type.googleapis.com/google.rpc.ErrorInfo",
│     "domain": "googleapis.com",
│     "metadata": {
│       "consumer": "projects/tef-olapp",
│       "service": "cloudkms.googleapis.com"
│     },
│     "reason": "SERVICE_DISABLED"
│   }
│ ]
│ 
│   with module.seed_bootstrap.module.kms[0].google_kms_key_ring.key_ring,
│   on .terraform/modules/seed_bootstrap.kms/main.tf line 21, in resource "google_kms_key_ring" "key_ring":
│   21: resource "google_kms_key_ring" "key_ring" {
│

need cloudkms

20240307:0800

For Terraform 1.3.7 upgrade https://github.com/terraform-google-modules/terraform-example-foundation/issues/1141 https://github.com/terraform-google-modules/terraform-example-foundation/issues/1142

michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ gcloud services enable cloudkms.googleapis.com
Operation "operations/acat.p2-153288813308-f346fb9f-e5a4-4ced-ba6a-d5b82c442f68" finished successfully.

0720 rerun terraform init/plan/apply

Plan: 159 to add, 0 to change, 0 to destroy.

Changes to Outputs:
  + cloud_build_peered_network_id                     = (known after apply)
  + cloud_build_private_worker_pool_id                = (known after apply)
  + cloud_build_worker_range_id                       = (known after apply)
  + cloud_builder_artifact_repo                       = (known after apply)
  + csr_repos                                         = {
      + gcp-bootstrap    = {
          + id      = (known after apply)
          + name    = "gcp-bootstrap"
          + project = "prj-b-cicd-wm4z"
          + url     = (known after apply)
        }
      + gcp-environments = {
          + id      = (known after apply)
          + name    = "gcp-environments"
          + project = "prj-b-cicd-wm4z"
          + url     = (known after apply)
        }
      + gcp-networks     = {
          + id      = (known after apply)
          + name    = "gcp-networks"
          + project = "prj-b-cicd-wm4z"
          + url     = (known after apply)
        }
      + gcp-org          = {
          + id      = (known after apply)
          + name    = "gcp-org"
          + project = "prj-b-cicd-wm4z"
          + url     = (known after apply)
        }
      + gcp-policies     = {
          + id      = (known after apply)
          + name    = "gcp-policies"
          + project = "prj-b-cicd-wm4z"
          + url     = (known after apply)
        }
      + gcp-projects     = {
          + id      = (known after apply)
          + name    = "gcp-projects"
          + project = "prj-b-cicd-wm4z"
          + url     = (known after apply)
        }
      + tf-cloudbuilder  = {
          + id      = (known after apply)
          + name    = "tf-cloudbuilder"
          + project = "prj-b-cicd-wm4z"
          + url     = (known after apply)
        }
    }
  + gcs_bucket_cloudbuild_artifacts                   = {
      + bootstrap = (known after apply)
      + env       = (known after apply)
      + net       = (known after apply)
      + org       = (known after apply)
      + proj      = (known after apply)
    }
  + gcs_bucket_cloudbuild_logs                        = {
      + bootstrap = (known after apply)
      + env       = (known after apply)
      + net       = (known after apply)
      + org       = (known after apply)
      + proj      = (known after apply)
    }

down to 159

expecting more service enablement issues

The list is in https://github.com/terraform-google-modules/terraform-example-foundation/blob/master/0-bootstrap/main.tf#L78

activate_apis = [
    "serviceusage.googleapis.com",
    "servicenetworking.googleapis.com",
    "cloudkms.googleapis.com",
    "compute.googleapis.com",
    "logging.googleapis.com",
    "bigquery.googleapis.com",
    "cloudresourcemanager.googleapis.com",
    "cloudbilling.googleapis.com",
    "cloudbuild.googleapis.com",
    "iam.googleapis.com",
    "admin.googleapis.com",
    "appengine.googleapis.com",
    "storage-api.googleapis.com",
    "monitoring.googleapis.com",
    "pubsub.googleapis.com",
    "securitycenter.googleapis.com",
    "accesscontextmanager.googleapis.com",
    "billingbudgets.googleapis.com",
    "essentialcontacts.googleapis.com",
    "assuredworkloads.googleapis.com",
    "cloudasset.googleapis.com"
  ]

terraform apply

0726

michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ terraform apply bootstrap.tfplan

module.seed_bootstrap.module.kms[0].google_kms_key_ring.key_ring: Creating...
module.seed_bootstrap.module.kms[0].google_kms_key_ring.key_ring: Creation complete after 0s [id=projects/prj-b-seed-31ca/locations/us-central1/keyRings/prj-keyring]
module.seed_bootstrap.module.kms[0].google_kms_crypto_key.key[0]: Creating...
module.seed_bootstrap.module.kms[0].google_kms_crypto_key.key[0]: Creation complete after 1s [id=projects/prj-b-seed-31ca/locations/us-central1/keyRings/prj-keyring/cryptoKeys/prj-key]
module.seed_bootstrap.module.kms[0].google_kms_crypto_key_iam_binding.decrypters[0]: Creating...
module.seed_bootstrap.module.kms[0].google_kms_crypto_key_iam_binding.encrypters[0]: Creating...
module.seed_bootstrap.module.kms[0].google_kms_crypto_key_iam_binding.encrypters[0]: Creation complete after 8s [id=projects/prj-b-seed-31ca/locations/us-central1/keyRings/prj-keyring/cryptoKeys/prj-key/roles/cloudkms.cryptoKeyEncrypter]
module.seed_bootstrap.module.kms[0].google_kms_crypto_key_iam_binding.decrypters[0]: Creation complete after 8s [id=projects/prj-b-seed-31ca/locations/us-central1/keyRings/prj-keyring/cryptoKeys/prj-key/roles/cloudkms.cryptoKeyDecrypter]
module.seed_bootstrap.google_storage_bucket.org_terraform_state: Creating...
module.seed_bootstrap.google_storage_bucket.org_terraform_state: Creation complete after 2s [id=bkt-prj-b-seed-tfstate-cca4]
module.seed_bootstrap.google_storage_bucket_iam_member.orgadmins_state_iam[0]: Creating...
module.gcp_projects_state_bucket.google_storage_bucket.bucket: Creating...
module.gcp_projects_state_bucket.google_storage_bucket.bucket: Creation complete after 1s [id=bkt-prj-b-seed-31ca-gcp-projects-tfstate]
module.seed_bootstrap.google_storage_bucket_iam_member.orgadmins_state_iam[0]: Creation complete after 4s [id=b/bkt-prj-b-seed-tfstate-cca4/roles/storage.admin/group:gcp-organization-admins@obrienlabs.app]
module.tf_source.module.cloudbuild_project.module.project-factory.random_id.random_project_id_suffix: Creating...
module.tf_source.module.cloudbuild_project.module.project-factory.random_id.random_project_id_suffix: Creation complete after 0s [id=wBU]
module.tf_source.module.cloudbuild_project.module.project-factory.google_project.main: Creating...

odule.tf_source.module.cloudbuild_project.module.project-factory.google_project.main: Still creating... [10s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.google_project.main: Still creating... [20s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.google_project.main: Still creating... [30s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.google_project.main: Still creating... [40s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.google_project.main: Still creating... [50s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.google_project.main: Still creating... [1m0s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.google_project.main: Still creating... [1m10s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.google_project.main: Still creating... [1m20s elapsed]

module.tf_source.module.cloudbuild_project.module.project-factory.google_project.main: Still creating... [1m30s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.google_project.main: Still creating... [1m40s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.google_project.main: Still creating... [1m50s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.google_project.main: Still creating... [2m0s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.google_project.main: Still creating... [2m10s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.google_project.main: Still creating... [2m20s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.google_project.main: Still creating... [2m30s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.google_project.main: Still creating... [2m40s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.google_project.main: Still creating... [2m50s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.google_project.main: Still creating... [3m0s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.google_project.main: Still creating... [3m10s elapsed]

module.tf_source.module.cloudbuild_project.module.project-factory.google_project.main: Still creating... [3m20s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.google_project.main: Still creating... [3m30s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.google_project.main: Creation complete after 3m34s [id=projects/prj-b-cicd-wm4z]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["storage-api.googleapis.com"]: Creating...
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["compute.googleapis.com"]: Creating...
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["dns.googleapis.com"]: Creating...
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["servicenetworking.googleapis.com"]: Creating...
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["billingbudgets.googleapis.com"]: Creating...
module.tf_source.module.cloudbuild_project.module.project-factory.google_service_account.default_service_account[0]: Creating...
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["cloudbuild.googleapis.com"]: Creating...
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["artifactregistry.googleapis.com"]: Creating...
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["sourcerepo.googleapis.com"]: Creating...
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["cloudresourcemanager.googleapis.com"]: Creating...
module.tf_source.module.cloudbuild_project.module.project-factory.google_service_account.default_service_account[0]: Creation complete after 1s [id=projects/prj-b-cicd-wm4z/serviceAccounts/project-service-account@prj-b-cicd-wm4z.iam.gserviceaccount.com]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["bigquery.googleapis.com"]: Creating...
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["compute.googleapis.com"]: Creation complete after 3s [id=prj-b-cicd-wm4z/compute.googleapis.com]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["serviceusage.googleapis.com"]: Creating...

module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["servicenetworking.googleapis.com"]: Still creating... [10s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["billingbudgets.googleapis.com"]: Still creating... [10s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["dns.googleapis.com"]: Still creating... [10s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["artifactregistry.googleapis.com"]: Still creating... [10s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["cloudbuild.googleapis.com"]: Still creating... [10s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["cloudresourcemanager.googleapis.com"]: Still creating... [10s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["sourcerepo.googleapis.com"]: Still creating... [10s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["bigquery.googleapis.com"]: Still creating... [10s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["serviceusage.googleapis.com"]: Still creating... [10s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["dns.googleapis.com"]: Still creating... [20s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["billingbudgets.googleapis.com"]: Still creating... [20s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["artifactregistry.googleapis.com"]: Still creating... [20s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["servicenetworking.googleapis.com"]: Still creating... [20s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["storage-api.googleapis.com"]: Still creating... [20s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["cloudbuild.googleapis.com"]: Still creating... [20s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["cloudresourcemanager.googleapis.com"]: Still creating... [20s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["sourcerepo.googleapis.com"]: Still creating... [20s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["bigquery.googleapis.com"]: Still creating... [20s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["sourcerepo.googleapis.com"]: Creation complete after 22s [id=prj-b-cicd-wm4z/sourcerepo.googleapis.com]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["storage-api.googleapis.com"]: Creation complete after 22s [id=prj-b-cicd-wm4z/storage-api.googleapis.com]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["serviceusage.googleapis.com"]: Creation complete after 19s [id=prj-b-cicd-wm4z/serviceusage.googleapis.com]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["logging.googleapis.com"]: Creating...
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["billingbudgets.googleapis.com"]: Creation complete after 22s [id=prj-b-cicd-wm4z/billingbudgets.googleapis.com]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["artifactregistry.googleapis.com"]: Creation complete after 22s [id=prj-b-cicd-wm4z/artifactregistry.googleapis.com]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["servicenetworking.googleapis.com"]: Creation complete after 22s [id=prj-b-cicd-wm4z/servicenetworking.googleapis.com]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["cloudbilling.googleapis.com"]: Creating...
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["workflows.googleapis.com"]: Creating...
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["cloudscheduler.googleapis.com"]: Creating...
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["cloudbuild.googleapis.com"]: Creation complete after 22s [id=prj-b-cicd-wm4z/cloudbuild.googleapis.com]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["dns.googleapis.com"]: Creation complete after 22s [id=prj-b-cicd-wm4z/dns.googleapis.com]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["cloudresourcemanager.googleapis.com"]: Creation complete after 22s [id=prj-b-cicd-wm4z/cloudresourcemanager.googleapis.com]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["bigquery.googleapis.com"]: Creation complete after 21s [id=prj-b-cicd-wm4z/bigquery.googleapis.com]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["admin.googleapis.com"]: Creating...
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["appengine.googleapis.com"]: Creating...
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["iam.googleapis.com"]: Creating...
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["logging.googleapis.com"]: Creation complete after 3s [id=prj-b-cicd-wm4z/logging.googleapis.com]

module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["cloudbilling.googleapis.com"]: Still creating... [10s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["cloudscheduler.googleapis.com"]: Still creating... [10s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["workflows.googleapis.com"]: Still creating... [10s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["appengine.googleapis.com"]: Still creating... [10s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["admin.googleapis.com"]: Still creating... [10s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["iam.googleapis.com"]: Still creating... [10s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["cloudbilling.googleapis.com"]: Still creating... [20s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["cloudscheduler.googleapis.com"]: Still creating... [20s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["workflows.googleapis.com"]: Still creating... [20s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["appengine.googleapis.com"]: Still creating... [20s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["admin.googleapis.com"]: Still creating... [20s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["iam.googleapis.com"]: Still creating... [20s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["appengine.googleapis.com"]: Creation complete after 22s [id=prj-b-cicd-wm4z/appengine.googleapis.com]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["admin.googleapis.com"]: Creation complete after 22s [id=prj-b-cicd-wm4z/admin.googleapis.com]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["cloudbilling.googleapis.com"]: Creation complete after 22s [id=prj-b-cicd-wm4z/cloudbilling.googleapis.com]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["workflows.googleapis.com"]: Creation complete after 22s [id=prj-b-cicd-wm4z/workflows.googleapis.com]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["iam.googleapis.com"]: Creation complete after 22s [id=prj-b-cicd-wm4z/iam.googleapis.com]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["cloudscheduler.googleapis.com"]: Creation complete after 22s [id=prj-b-cicd-wm4z/cloudscheduler.googleapis.com]
module.tf_source.module.cloudbuild_project.module.project-factory.google_project_default_service_accounts.default_service_accounts[0]: Creating...
module.tf_source.google_sourcerepo_repository.gcp_repo["gcp-environments"]: Creating...
module.tf_source.google_sourcerepo_repository.gcp_repo["gcp-policies"]: Creating...
module.tf_source.google_sourcerepo_repository.gcp_repo["gcp-org"]: Creating...
module.tf_source.google_project_iam_member.org_admins_cloudbuild_viewer: Creating...
module.tf_source.google_sourcerepo_repository.gcp_repo["gcp-bootstrap"]: Creating...
module.tf_source.google_sourcerepo_repository.gcp_repo["gcp-projects"]: Creating...
module.tf_source.google_sourcerepo_repository.gcp_repo["gcp-networks"]: Creating...
module.tf_source.google_project_iam_member.org_admins_source_repo_admin[0]: Creating...
module.tf_source.google_sourcerepo_repository.gcp_repo["tf-cloudbuilder"]: Creating...
module.tf_source.module.cloudbuild_project.module.project-factory.google_project_default_service_accounts.default_service_accounts[0]: Creation complete after 0s [id=projects/prj-b-cicd-wm4z]
module.tf_source.google_project_iam_member.org_admins_cloudbuild_editor: Creating...
module.tf_source.google_sourcerepo_repository.gcp_repo["tf-cloudbuilder"]: Creation complete after 1s [id=projects/prj-b-cicd-wm4z/repos/tf-cloudbuilder]
module.tf_source.google_sourcerepo_repository.gcp_repo["gcp-bootstrap"]: Creation complete after 1s [id=projects/prj-b-cicd-wm4z/repos/gcp-bootstrap]
module.tf_source.google_sourcerepo_repository.gcp_repo["gcp-projects"]: Creation complete after 1s [id=projects/prj-b-cicd-wm4z/repos/gcp-projects]
module.tf_source.google_sourcerepo_repository.gcp_repo["gcp-policies"]: Creation complete after 1s [id=projects/prj-b-cicd-wm4z/repos/gcp-policies]
module.tf_source.google_sourcerepo_repository.gcp_repo["gcp-org"]: Creation complete after 1s [id=projects/prj-b-cicd-wm4z/repos/gcp-org]
module.tf_source.module.cloudbuild_bucket.google_storage_bucket.bucket: Creating...
module.tf_source.google_sourcerepo_repository.gcp_repo["gcp-networks"]: Creation complete after 2s [id=projects/prj-b-cicd-wm4z/repos/gcp-networks]
module.tf_source.module.cloudbuild_bucket.google_storage_bucket.bucket: Creation complete after 1s [id=prj-b-cicd-wm4z_cloudbuild]
module.tf_source.google_sourcerepo_repository.gcp_repo["gcp-environments"]: Creation complete after 2s [id=projects/prj-b-cicd-wm4z/repos/gcp-environments]
module.tf_source.google_storage_bucket_iam_member.cloudbuild_iam: Creating...

module.tf_source.google_storage_bucket_iam_member.cloudbuild_iam: Creation complete after 5s [id=b/prj-b-cicd-wm4z_cloudbuild/roles/storage.admin/serviceAccount:1083787941178@cloudbuild.gserviceaccount.com]
module.tf_source.google_project_iam_member.org_admins_cloudbuild_editor: Creation complete after 8s [id=prj-b-cicd-wm4z/roles/cloudbuild.builds.editor/group:gcp-organization-admins@obrienlabs.app]
module.tf_source.google_project_iam_member.org_admins_cloudbuild_viewer: Creation complete after 8s [id=prj-b-cicd-wm4z/roles/viewer/group:gcp-organization-admins@obrienlabs.app]
module.tf_source.google_project_iam_member.org_admins_source_repo_admin[0]: Creation complete after 8s [id=prj-b-cicd-wm4z/roles/source.admin/group:gcp-organization-admins@obrienlabs.app]
module.bootstrap_csr_repo.null_resource.run_destroy_command[0]: Creating...
module.tf_private_pool.module.peered_network[0].module.vpc.google_compute_network.network: Creating...
google_sourcerepo_repository_iam_member.member["proj"]: Creating...
google_sourcerepo_repository_iam_member.member["bootstrap"]: Creating...
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/storage.admin"]: Creating...
google_sourcerepo_repository_iam_member.member["env"]: Creating...
module.tf_cloud_builder.google_service_account.workflow_sa[0]: Creating...
google_sourcerepo_repository_iam_member.member["net"]: Creating...
google_sourcerepo_repository_iam_member.member["org"]: Creating...
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/artifactregistry.admin"]: Creating...
module.bootstrap_csr_repo.null_resource.run_destroy_command[0]: Creation complete after 0s [id=6607708089699954645]
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/compute.networkAdmin"]: Creating...
module.tf_cloud_builder.google_service_account.workflow_sa[0]: Creation complete after 1s [id=projects/prj-b-cicd-wm4z/serviceAccounts/terraform-runner-workflow-sa@prj-b-cicd-wm4z.iam.gserviceaccount.com]
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/cloudbuild.workerPoolOwner"]: Creating...
google_sourcerepo_repository_iam_member.member["bootstrap"]: Creation complete after 4s [id=projects/prj-b-cicd-wm4z/repos/gcp-policies/roles/viewer/serviceAccount:sa-terraform-bootstrap@prj-b-seed-31ca.iam.gserviceaccount.com]
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/iam.serviceAccountAdmin"]: Creating...
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/cloudbuild.workerPoolOwner"]: Creation complete after 7s [id=prj-b-cicd-wm4z/roles/cloudbuild.workerPoolOwner/serviceAccount:sa-terraform-bootstrap@prj-b-seed-31ca.iam.gserviceaccount.com]
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/source.admin"]: Creating...
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/storage.admin"]: Creation complete after 8s [id=prj-b-cicd-wm4z/roles/storage.admin/serviceAccount:sa-terraform-bootstrap@prj-b-seed-31ca.iam.gserviceaccount.com]
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/dns.admin"]: Creating...

te.networkAdmin/serviceAccount:sa-terraform-bootstrap@prj-b-seed-31ca.iam.gserviceaccount.com]
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/workflows.admin"]: Creating...
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/artifactregistry.admin"]: Creation complete after 12s [id=prj-b-cicd-wm4z/roles/artifactregistry.admin/serviceAccount:sa-terraform-bootstrap@prj-b-seed-31ca.iam.gserviceaccount.com]
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/cloudbuild.builds.editor"]: Creating...
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/iam.serviceAccountAdmin"]: Still creating... [10s elapsed]
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/dns.admin"]: Creation complete after 9s [id=prj-b-cicd-wm4z/roles/dns.admin/serviceAccount:sa-terraform-bootstrap@prj-b-seed-31ca.iam.gserviceaccount.com]
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/iam.workloadIdentityPoolAdmin"]: Creating...
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/source.admin"]: Creation complete after 9s [id=prj-b-cicd-wm4z/roles/source.admin/serviceAccount:sa-terraform-bootstrap@prj-b-seed-31ca.iam.gserviceaccount.com]
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/resourcemanager.projectDeleter"]: Creating...
google_sourcerepo_repository_iam_member.member["proj"]: Creation complete after 20s [id=projects/prj-b-cicd-wm4z/repos/gcp-policies/roles/viewer/serviceAccount:sa-terraform-proj@prj-b-seed-31ca.iam.gserviceaccount.com]
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/cloudscheduler.admin"]: Creating...
google_sourcerepo_repository_iam_member.member["env"]: Creation complete after 20s [id=projects/prj-b-cicd-wm4z/repos/gcp-policies/roles/viewer/serviceAccount:sa-terraform-env@prj-b-seed-31ca.iam.gserviceaccount.com]
module.bootstrap_csr_repo.null_resource.run_command[0]: Creating...
module.bootstrap_csr_repo.null_resource.run_command[0]: Provisioning with 'local-exec'...
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): Executing: ["/bin/sh" "-c" "PATH=/google-cloud-sdk/bin:$PATH\n./scripts/push-to-repo.sh prj-b-cicd-wm4z tf-cloudbuilder ./Dockerfile\n"]
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + '[' 3 -lt 3 ']'
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + CSR_PROJECT_ID=prj-b-cicd-wm4z
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + CSR_NAME=tf-cloudbuilder
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + DOCKERFILE_PATH=./Dockerfile
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): ++ mktemp -d
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + tmp_dir=/tmp/tmp.iGWG1EfS69
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + gcloud source repos clone tf-cloudbuilder /tmp/tmp.iGWG1EfS69 --project prj-b-cicd-wm4z
google_sourcerepo_repository_iam_member.member["net"]: Creation complete after 20s [id=projects/prj-b-cicd-wm4z/repos/gcp-policies/roles/viewer/serviceAccount:sa-terraform-net@prj-b-seed-31ca.iam.gserviceaccount.com]
module.tf_private_pool.module.peered_network[0].module.vpc.google_compute_network.network: Still creating... [20s elapsed]
google_sourcerepo_repository_iam_member.member["org"]: Still creating... [20s elapsed]
google_sourcerepo_repository_iam_member.member["org"]: Creation complete after 20s [id=projects/prj-b-cicd-wm4z/repos/gcp-policies/roles/viewer/serviceAccount:sa-terraform-org@prj-b-seed-31ca.iam.gserviceaccount.com]
module.tf_cloud_builder.google_service_account.cb_sa[0]: Creating...
module.tf_cloud_builder.google_artifact_registry_repository.tf-image-repo: Creating...
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): Cloning into '/tmp/tmp.iGWG1EfS69'...
module.tf_cloud_builder.google_service_account.cb_sa[0]: Creation complete after 1s [id=projects/prj-b-cicd-wm4z/serviceAccounts/tf-cb-builder-sa@prj-b-cicd-wm4z.iam.gserviceaccount.com]
module.tf_cloud_builder.google_project_iam_member.trigger_builds: Creating...
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/iam.serviceAccountAdmin"]: Creation complete after 18s [id=prj-b-cicd-wm4z/roles/iam.serviceAccountAdmin/serviceAccount:sa-terraform-bootstrap@prj-b-seed-31ca.iam.gserviceaccount.com]
module.tf_cloud_builder.google_project_iam_member.invoke_workflow_scheduler: Creating...
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/workflows.admin"]: Still creating... [10s elapsed]
module.tf_private_pool.module.peered_network[0].module.vpc.google_compute_network.network: Creation complete after 22s [id=projects/prj-b-cicd-wm4z/global/networks/vpc-b-cbpools]
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/cloudbuild.builds.editor"]: Still creating... [10s elapsed]
module.tf_cloud_builder.google_workflows_workflow.builder: Creating...
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): warning: You appear to have cloned an empty repository.
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): Project [prj-b-cicd-wm4z] repository [tf-cloudbuilder] was cloned to [/tmp/tmp.iGWG1EfS69].
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + cp ./Dockerfile /tmp/tmp.iGWG1EfS69
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + pushd /tmp/tmp.iGWG1EfS69
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): /tmp/tmp.iGWG1EfS69 ~/tef-olapp/github/terraform-example-foundation/0-bootstrap
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + git config credential.helper gcloud.sh
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + git config init.defaultBranch main
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + git config user.email terraform-robot@example.com
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + git config user.name 'TF Robot'
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + git checkout main
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): error: pathspec 'main' did not match any file(s) known to git
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + git checkout -b main
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): Switched to a new branch 'main'
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + git add Dockerfile
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + git commit -m 'Initialize tf dockerfile repo'
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): [main (root-commit) 55aa00d] Initialize tf dockerfile repo
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec):  1 file changed, 39 insertions(+)
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec):  create mode 100644 Dockerfile
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + git push origin main -f

te.networkAdmin/serviceAccount:sa-terraform-bootstrap@prj-b-seed-31ca.iam.gserviceaccount.com]
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/workflows.admin"]: Creating...
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/artifactregistry.admin"]: Creation complete after 12s [id=prj-b-cicd-wm4z/roles/artifactregistry.admin/serviceAccount:sa-terraform-bootstrap@prj-b-seed-31ca.iam.gserviceaccount.com]
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/cloudbuild.builds.editor"]: Creating...
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/iam.serviceAccountAdmin"]: Still creating... [10s elapsed]
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/dns.admin"]: Creation complete after 9s [id=prj-b-cicd-wm4z/roles/dns.admin/serviceAccount:sa-terraform-bootstrap@prj-b-seed-31ca.iam.gserviceaccount.com]
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/iam.workloadIdentityPoolAdmin"]: Creating...
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/source.admin"]: Creation complete after 9s [id=prj-b-cicd-wm4z/roles/source.admin/serviceAccount:sa-terraform-bootstrap@prj-b-seed-31ca.iam.gserviceaccount.com]
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/resourcemanager.projectDeleter"]: Creating...
google_sourcerepo_repository_iam_member.member["proj"]: Creation complete after 20s [id=projects/prj-b-cicd-wm4z/repos/gcp-policies/roles/viewer/serviceAccount:sa-terraform-proj@prj-b-seed-31ca.iam.gserviceaccount.com]
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/cloudscheduler.admin"]: Creating...
google_sourcerepo_repository_iam_member.member["env"]: Creation complete after 20s [id=projects/prj-b-cicd-wm4z/repos/gcp-policies/roles/viewer/serviceAccount:sa-terraform-env@prj-b-seed-31ca.iam.gserviceaccount.com]
module.bootstrap_csr_repo.null_resource.run_command[0]: Creating...
module.bootstrap_csr_repo.null_resource.run_command[0]: Provisioning with 'local-exec'...
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): Executing: ["/bin/sh" "-c" "PATH=/google-cloud-sdk/bin:$PATH\n./scripts/push-to-repo.sh prj-b-cicd-wm4z tf-cloudbuilder ./Dockerfile\n"]
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + '[' 3 -lt 3 ']'
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + CSR_PROJECT_ID=prj-b-cicd-wm4z
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + CSR_NAME=tf-cloudbuilder
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + DOCKERFILE_PATH=./Dockerfile
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): ++ mktemp -d
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + tmp_dir=/tmp/tmp.iGWG1EfS69
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + gcloud source repos clone tf-cloudbuilder /tmp/tmp.iGWG1EfS69 --project prj-b-cicd-wm4z
google_sourcerepo_repository_iam_member.member["net"]: Creation complete after 20s [id=projects/prj-b-cicd-wm4z/repos/gcp-policies/roles/viewer/serviceAccount:sa-terraform-net@prj-b-seed-31ca.iam.gserviceaccount.com]
module.tf_private_pool.module.peered_network[0].module.vpc.google_compute_network.network: Still creating... [20s elapsed]
google_sourcerepo_repository_iam_member.member["org"]: Still creating... [20s elapsed]
google_sourcerepo_repository_iam_member.member["org"]: Creation complete after 20s [id=projects/prj-b-cicd-wm4z/repos/gcp-policies/roles/viewer/serviceAccount:sa-terraform-org@prj-b-seed-31ca.iam.gserviceaccount.com]
module.tf_cloud_builder.google_service_account.cb_sa[0]: Creating...
module.tf_cloud_builder.google_artifact_registry_repository.tf-image-repo: Creating...
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): Cloning into '/tmp/tmp.iGWG1EfS69'...
module.tf_cloud_builder.google_service_account.cb_sa[0]: Creation complete after 1s [id=projects/prj-b-cicd-wm4z/serviceAccounts/tf-cb-builder-sa@prj-b-cicd-wm4z.iam.gserviceaccount.com]
module.tf_cloud_builder.google_project_iam_member.trigger_builds: Creating...
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/iam.serviceAccountAdmin"]: Creation complete after 18s [id=prj-b-cicd-wm4z/roles/iam.serviceAccountAdmin/serviceAccount:sa-terraform-bootstrap@prj-b-seed-31ca.iam.gserviceaccount.com]
module.tf_cloud_builder.google_project_iam_member.invoke_workflow_scheduler: Creating...
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/workflows.admin"]: Still creating... [10s elapsed]
module.tf_private_pool.module.peered_network[0].module.vpc.google_compute_network.network: Creation complete after 22s [id=projects/prj-b-cicd-wm4z/global/networks/vpc-b-cbpools]
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/cloudbuild.builds.editor"]: Still creating... [10s elapsed]
module.tf_cloud_builder.google_workflows_workflow.builder: Creating...
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): warning: You appear to have cloned an empty repository.
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): Project [prj-b-cicd-wm4z] repository [tf-cloudbuilder] was cloned to [/tmp/tmp.iGWG1EfS69].
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + cp ./Dockerfile /tmp/tmp.iGWG1EfS69
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + pushd /tmp/tmp.iGWG1EfS69
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): /tmp/tmp.iGWG1EfS69 ~/tef-olapp/github/terraform-example-foundation/0-bootstrap
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + git config credential.helper gcloud.sh
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + git config init.defaultBranch main
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + git config user.email terraform-robot@example.com
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + git config user.name 'TF Robot'
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + git checkout main
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): error: pathspec 'main' did not match any file(s) known to git
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + git checkout -b main
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): Switched to a new branch 'main'
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + git add Dockerfile
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + git commit -m 'Initialize tf dockerfile repo'
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): [main (root-commit) 55aa00d] Initialize tf dockerfile repo
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec):  1 file changed, 39 insertions(+)
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec):  create mode 100644 Dockerfile
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + git push origin main -f

e.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/workflows.admin"]: Creation complete after 16s [id=prj-b-cicd-wm4z/roles/workflows.admin/serviceAccount:sa-terraform-bootstrap@prj-b-seed-31ca.iam.gserviceaccount.com]
module.tf_cloud_builder.google_project_iam_member.logs_writer: Creating...
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): remote: Waiting for private key checker: 1/1 objects left
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): To https://source.developers.google.com/p/prj-b-cicd-wm4z/r/tf-cloudbuilder
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec):  * [new branch]      main -> main
module.bootstrap_csr_repo.null_resource.run_command[0]: Creation complete after 8s [id=2083100521623893606]
module.tf_private_pool.module.peered_network[0].module.subnets.google_compute_subnetwork.subnetwork["us-central1/sb-b-cbpools-us-central1"]: Creating...
module.tf_cloud_builder.google_cloud_scheduler_job.trigger_workflow: Creation complete after 1s [id=projects/prj-b-cicd-wm4z/locations/us-central1/jobs/trigger-terraform-runner-workflow]
module.tf_private_pool.google_dns_policy.default_policy[0]: Creating...
module.tf_private_pool.google_dns_policy.default_policy[0]: Creation complete after 1s [id=projects/prj-b-cicd-wm4z/policies/dp-b-cbpools-default-policy]
module.tf_cloud_builder.google_storage_bucket_iam_member.member: Creating...
module.tf_cloud_builder.google_artifact_registry_repository.tf-image-repo: Still creating... [10s elapsed]
module.tf_cloud_builder.google_service_account_iam_member.use_cb_sa: Creation complete after 4s [id=projects/prj-b-cicd-wm4z/serviceAccounts/tf-cb-builder-sa@prj-b-cicd-wm4z.iam.gserviceaccount.com/roles/iam.serviceAccountUser/serviceAccount:terraform-runner-workflow-sa@prj-b-cicd-wm4z.iam.gserviceaccount.com]
module.tf_private_pool.google_compute_global_address.worker_pool_range[0]: Creating...
module.tf_cloud_builder.google_project_iam_member.trigger_builds: Still creating... [10s elapsed]
module.tf_cloud_builder.google_artifact_registry_repository.tf-image-repo: Creation complete after 11s [id=projects/prj-b-cicd-wm4z/locations/us-central1/repositories/tf-runners]
module.tf_cloud_builder.google_artifact_registry_repository_iam_member.workflow_list: Creating...
module.tf_cloud_builder.google_sourcerepo_repository_iam_member.member[0]: Creation complete after 4s [id=projects/prj-b-cicd-wm4z/repos/tf-cloudbuilder/roles/viewer/serviceAccount:tf-cb-builder-sa@prj-b-cicd-wm4z.iam.gserviceaccount.com]
module.tf_cloud_builder.google_artifact_registry_repository_iam_member.push_images: Creating...
module.tf_cloud_builder.google_project_iam_member.invoke_workflow_scheduler: Still creating... [10s elapsed]
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/cloudbuild.builds.editor"]: Creation complete after 19s [id=prj-b-cicd-wm4z/roles/cloudbuild.builds.editor/serviceAccount:sa-terraform-bootstrap@prj-b-seed-31ca.iam.gserviceaccount.com]
google_artifact_registry_repository_iam_member.terraform_sa_artifact_registry_reader["bootstrap"]: Creating...
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/iam.workloadIdentityPoolAdmin"]: Creation complete after 15s [id=prj-b-cicd-wm4z/roles/iam.workloadIdentityPoolAdmin/serviceAccount:sa-terraform-bootstrap@prj-b-seed-31ca.iam.gserviceaccount.com]
google_artifact_registry_repository_iam_member.terraform_sa_artifact_registry_reader["env"]: Creating...
module.tf_cloud_builder.google_storage_bucket_iam_member.member: Creation complete after 4s [id=b/bkt-prj-b-cicd-wm4z-tf-cloudbuilder-build-logs/roles/storage.admin/serviceAccount:tf-cb-builder-sa@prj-b-cicd-wm4z.iam.gserviceaccount.com]
google_artifact_registry_repository_iam_member.terraform_sa_artifact_registry_reader["net"]: Creating...
module.tf_cloud_builder.google_project_iam_member.trigger_builds: Creation complete after 16s [id=prj-b-cicd-wm4z/roles/cloudbuild.builds.editor/serviceAccount:terraform-runner-workflow-sa@prj-b-cicd-wm4z.iam.gserviceaccount.com]
google_artifact_registry_repository_iam_member.terraform_sa_artifact_registry_reader["proj"]: Creating...
module.tf_cloud_builder.google_project_iam_member.invoke_workflow_scheduler: Creation complete after 15s [id=prj-b-cicd-wm4z/roles/workflows.invoker/serviceAccount:terraform-runner-workflow-sa@prj-b-cicd-wm4z.iam.gserviceaccount.com]
google_artifact_registry_repository_iam_member.terraform_sa_artifact_registry_reader["org"]: Creating...
module.tf_cloud_builder.google_project_iam_member.logs_writer: Creation complete after 9s [id=prj-b-cicd-wm4z/roles/logging.logWriter/serviceAccount:tf-cb-builder-sa@prj-b-cicd-wm4z.iam.gserviceaccount.com]
module.bootstrap_projects_remove_editor["cicd"].google_project_iam_binding.iam_remove["roles/editor"]: Creating...
module.tf_private_pool.module.peered_network[0].module.subnets.google_compute_subnetwork.subnetwork["us-central1/sb-b-cbpools-us-central1"]: Still creating... [10s elapsed]
module.tf_private_pool.module.peered_network[0].module.subnets.google_compute_subnetwork.subnetwork["us-central1/sb-b-cbpools-us-central1"]: Creation complete after 12s [id=projects/prj-b-cicd-wm4z/regions/us-central1/subnetworks/sb-b-cbpools-us-central1]
module.bootstrap_projects_remove_editor["seed"].google_project_iam_binding.iam_remove["roles/editor"]: Creating...
module.tf_private_pool.google_compute_global_address.worker_pool_range[0]: Still creating... [10s elapsed]
module.tf_cloud_builder.google_artifact_registry_repository_iam_member.push_images: Creation complete after 9s [id=projects/prj-b-cicd-wm4z/locations/us-central1/repositories/tf-runners/roles/artifactregistry.writer/serviceAccount:tf-cb-builder-sa@prj-b-cicd-wm4z.iam.gserviceaccount.com]
module.tf_cloud_builder.google_artifact_registry_repository_iam_member.workflow_list: Creation complete after 10s [id=projects/prj-b-cicd-wm4z/locations/us-central1/repositories/tf-runners/roles/artifactregistry.reader/serviceAccount:terraform-runner-workflow-sa@prj-b-cicd-wm4z.iam.gserviceaccount.com]
module.tf_private_pool.google_compute_global_address.worker_pool_range[0]: Creation complete after 11s [id=projects/prj-b-cicd-wm4z/global/addresses/ga-b-cbpools-worker-pool-range]
module.tf_private_pool.google_service_networking_connection.worker_pool_conn[0]: Creating...
module.tf_private_pool.module.firewall_rules[0].google_compute_firewall.rules["fw-b-cbpools-100-i-a-all-all-all-service-networking"]: Creating...
google_artifact_registry_repository_iam_member.terraform_sa_artifact_registry_reader["env"]: Creation complete after 10s [id=proj

oogle_artifact_registry_repository_iam_member.terraform_sa_artifact_registry_reader["env"]: Creation complete after 10s [id=projects/prj-b-cicd-wm4z/locations/us-central1/repositories/tf-runners/roles/artifactregistry.reader/serviceAccount:sa-terraform-env@prj-b-seed-31ca.iam.gserviceaccount.com]
google_artifact_registry_repository_iam_member.terraform_sa_artifact_registry_reader["bootstrap"]: Still creating... [10s elapsed]
module.bootstrap_projects_remove_editor["cicd"].google_project_iam_binding.iam_remove["roles/editor"]: Creation complete after 7s [id=prj-b-cicd-wm4z/roles/editor]
google_artifact_registry_repository_iam_member.terraform_sa_artifact_registry_reader["net"]: Still creating... [10s elapsed]
google_artifact_registry_repository_iam_member.terraform_sa_artifact_registry_reader["proj"]: Still creating... [10s elapsed]
google_artifact_registry_repository_iam_member.terraform_sa_artifact_registry_reader["org"]: Still creating... [10s elapsed]
module.bootstrap_projects_remove_editor["seed"].google_project_iam_binding.iam_remove["roles/editor"]: Creation complete after 7s [id=prj-b-seed-31ca/roles/editor]
module.tf_private_pool.module.firewall_rules[0].google_compute_firewall.rules["fw-b-cbpools-100-i-a-all-all-all-service-networking"]: Still creating... [10s elapsed]
google_artifact_registry_repository_iam_member.terraform_sa_artifact_registry_reader["bootstrap"]: Still creating... [20s elapsed]
module.tf_private_pool.module.firewall_rules[0].google_compute_firewall.rules["fw-b-cbpools-100-i-a-all-all-all-service-networking"]: Creation complete after 11s [id=projects/prj-b-cicd-wm4z/global/firewalls/fw-b-cbpools-100-i-a-all-all-all-service-networking]
google_artifact_registry_repository_iam_member.terraform_sa_artifact_registry_reader["net"]: Still creating... [20s elapsed]
google_artifact_registry_repository_iam_member.terraform_sa_artifact_registry_reader["bootstrap"]: Creation complete after 25s [id=projects/prj-b-cicd-wm4z/locations/us-central1/repositories/tf-runners/roles/artifactregistry.reader/serviceAccount:sa-terraform-bootstrap@prj-b-seed-31ca.iam.gserviceaccount.com]
google_artifact_registry_repository_iam_member.terraform_sa_artifact_registry_reader["proj"]: Still creating... [20s elapsed]
google_artifact_registry_repository_iam_member.terraform_sa_artifact_registry_reader["org"]: Still creating... [20s elapsed]
google_artifact_registry_repository_iam_member.terraform_sa_artifact_registry_reader["net"]: Creation complete after 23s [id=projects/prj-b-cicd-wm4z/locations/us-central1/repositories/tf-runners/roles/artifactregistry.reader/serviceAccount:sa-terraform-net@prj-b-seed-31ca.iam.gserviceaccount.com]
google_artifact_registry_repository_iam_member.terraform_sa_artifact_registry_reader["proj"]: Creation complete after 20s [id=projects/prj-b-cicd-wm4z/locations/us-central1/repositories/tf-runners/roles/artifactregistry.reader/serviceAccount:sa-terraform-proj@prj-b-seed-31ca.iam.gserviceaccount.com]
google_artifact_registry_repository_iam_member.terraform_sa_artifact_registry_reader["org"]: Creation complete after 21s [id=projects/prj-b-cicd-wm4z/locations/us-central1/repositories/tf-runners/roles/artifactregistry.reader/serviceAccount:sa-terraform-org@prj-b-seed-31ca.iam.gserviceaccount.com]
╷
│ Error: Error waiting for Create Service Networking Connection: error while retrieving operation: googleapi: Error 403: Service Networking API has not been used in project tef-olapp before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/servicenetworking.googleapis.com/overview?project=tef-olapp then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.
│ Details:
│ [
│   {
│     "@type": "type.googleapis.com/google.rpc.Help",
│     "links": [
│       {
│         "description": "Google developers console API activation",
│         "url": "https://console.developers.google.com/apis/api/servicenetworking.googleapis.com/overview?project=tef-olapp"
│       }
│     ]
│   },
│   {
│     "@type": "type.googleapis.com/google.rpc.ErrorInfo",
│     "domain": "googleapis.com",
│     "metadata": {
│       "consumer": "projects/tef-olapp",
│       "service": "servicenetworking.googleapis.com"
│     },
│     "reason": "SERVICE_DISABLED"
│   }
│ ]
│ , accessNotConfigured
│ 
│   with module.tf_private_pool.google_service_networking_connection.worker_pool_conn[0],
│   on modules/cb-private-pool/network.tf line 72, in resource "google_service_networking_connection" "worker_pool_conn":
│   72: resource "google_service_networking_connection" "worker_pool_conn" {
│

0732

current list

michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ gcloud services list | grep NAME
NAME: analyticshub.googleapis.com
NAME: bigquery.googleapis.com
NAME: bigqueryconnection.googleapis.com
NAME: bigquerydatapolicy.googleapis.com
NAME: bigquerymigration.googleapis.com
NAME: bigqueryreservation.googleapis.com
NAME: bigquerystorage.googleapis.com
NAME: cloudapis.googleapis.com
NAME: cloudbilling.googleapis.com
NAME: cloudidentity.googleapis.com
NAME: cloudkms.googleapis.com
NAME: cloudresourcemanager.googleapis.com
NAME: cloudtrace.googleapis.com
NAME: dataform.googleapis.com
NAME: dataplex.googleapis.com
NAME: datastore.googleapis.com
NAME: iam.googleapis.com
NAME: iamcredentials.googleapis.com
NAME: logging.googleapis.com
NAME: monitoring.googleapis.com
NAME: servicemanagement.googleapis.com
NAME: servicenetworking.googleapis.com
NAME: serviceusage.googleapis.com
NAME: sql-component.googleapis.com
NAME: storage-api.googleapis.com
NAME: storage-component.googleapis.com
NAME: storage.googleapis.com

I am going to enable all services below The list is in https://github.com/terraform-google-modules/terraform-example-foundation/blob/master/0-bootstrap/main.tf#L78

activate_apis = [
    "serviceusage.googleapis.com",
    "servicenetworking.googleapis.com",
    "cloudkms.googleapis.com",
    "compute.googleapis.com",
    "logging.googleapis.com",
    "bigquery.googleapis.com",
    "cloudresourcemanager.googleapis.com",
    "cloudbilling.googleapis.com",
    "cloudbuild.googleapis.com",
    "iam.googleapis.com",
    "admin.googleapis.com",
    "appengine.googleapis.com",
    "storage-api.googleapis.com",
    "monitoring.googleapis.com",
    "pubsub.googleapis.com",
    "securitycenter.googleapis.com",
    "accesscontextmanager.googleapis.com",
    "billingbudgets.googleapis.com",
    "essentialcontacts.googleapis.com",
    "assuredworkloads.googleapis.com",
    "cloudasset.googleapis.com"
  ]

enabling - even though most of these are for CB project

cloudbuild.googleapis.com
appengine.googleapis.com
pubsub.googleapis.com
securitycenter.googleapis.com
accesscontextmanager.googleapis.com
billingbudgets.googleapis.com
essentialcontacts.googleapis.com
assuredworkloads.googleapis.com
cloudasset.googleapis.com

https://github.com/terraform-google-modules/terraform-example-foundation/issues/1143

more service enablements

michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ gcloud services enable cloudbuild.googleapis.com
Operation "operations/acf.p2-153288813308-9511143e-75a0-473a-b019-63c3fd280ff7" finished successfully.
michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ gcloud services enable appengine.googleapis.com
Operation "operations/acat.p2-153288813308-787a46f6-f539-4fa5-8f60-b7ca079e6baf" finished successfully.
michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ gcloud services enable pubsub.googleapis.com
michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ gcloud services enable securitycenter.googleapis.com
Operation "operations/acat.p2-153288813308-2e9e4ed9-3423-4a58-9709-70c31d1623c3" finished successfully.
michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ gcloud services enable accesscontextmanager.googleapis.com
Operation "operations/acat.p2-153288813308-8bcee864-8cb1-45ab-9cbc-d10d889e75c3" finished successfully.
michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ gcloud services enable billingbudgets.googleapis.com
Operation "operations/acat.p2-153288813308-3bfd1b6b-068a-434a-b2db-42841928c4dc" finished successfully.
michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ gcloud services enable essentialcontacts.googleapis.com
Operation "operations/acat.p2-153288813308-e747eb4d-6c9f-48fd-8791-b96b3b4b205a" finished successfully.
michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ gcloud services enable assuredworkloads.googleapis.com
Operation "operations/acat.p2-153288813308-a0f14a88-ae37-4d11-8ca7-e500adf89572" finished successfully.
michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ gcloud services enable cloudasset.googleapis.com
Operation "operations/acat.p2-153288813308-3a94bdb9-ca59-4b9f-8146-9150d57eb568" finished successfully.
michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$

review https://github.com/terraform-google-modules/terraform-example-foundation/blob/master/0-bootstrap/sa.tf#L34

// Roles required to manage resources in the Seed project
  granular_sa_seed_project = {
    "bootstrap" = [
      "roles/storage.admin",
      "roles/iam.serviceAccountAdmin",
      "roles/resourcemanager.projectDeleter",
      "roles/cloudkms.admin",
    ],
    "org" = [
      "roles/storage.objectAdmin",
    ],
    "env" = [
      "roles/storage.objectAdmin"
    ],
    "net" = [
      "roles/storage.objectAdmin",
    ],
    "proj" = [
      "roles/storage.objectAdmin",
    ],
  }

  // Roles required to manage resources in the CI/CD project
  granular_sa_cicd_project = {
    "bootstrap" = [
      "roles/storage.admin",
      "roles/compute.networkAdmin",
      "roles/cloudbuild.builds.editor",
      "roles/cloudbuild.workerPoolOwner",
      "roles/artifactregistry.admin",
      "roles/source.admin",
      "roles/iam.serviceAccountAdmin",
      "roles/workflows.admin",
      "roles/cloudscheduler.admin",
      "roles/resourcemanager.projectDeleter",
      "roles/dns.admin",
      "roles/iam.workloadIdentityPoolAdmin",
    ],
  }

Terraform apply 0-bootstrap

Plan: 73 to add, 0 to change, 0 to destroy.

Changes to Outputs:
  + cloud_build_private_worker_pool_id                = (known after apply)
  + gcs_bucket_cloudbuild_artifacts                   = {
      + bootstrap = (known after apply)
      + env       = (known after apply)
      + net       = (known after apply)
      + org       = (known after apply)
      + proj      = (known after apply)
    }
  + gcs_bucket_cloudbuild_logs                        = {
      + bootstrap = (known after apply)
      + env       = (known after apply)
      + net       = (known after apply)
      + org       = (known after apply)
      + proj      = (known after apply)
    }

check terraform cloud version (still modules) jetbrains intellij 2023 (up from 2021) - find references

use $terraform-google-modules = ../modules.. IntelliJ IDEA 2023.3.4 available terraform 233 marketplace plugin

0915

ichael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ terraform apply bootstrap.tfplan
module.tf_private_pool.google_service_networking_connection.worker_pool_conn[0]: Creating...

module.tf_private_pool.google_service_networking_connection.worker_pool_conn[0]: Still creating... [10s elapsed]
module.tf_private_pool.google_service_networking_connection.worker_pool_conn[0]: Still creating... [20s elapsed]
module.tf_private_pool.google_service_networking_connection.worker_pool_conn[0]: Creation complete after 21s [id=projects%2Fprj-b-cicd-wm4z%2Fglobal%2Fnetworks%2Fvpc-b-cbpools:servicenetworking.googleapis.com]
module.tf_private_pool.google_compute_network_peering_routes_config.peering_routes[0]: Creating...
module.tf_private_pool.google_cloudbuild_worker_pool.private_pool: Creating...
module.tf_private_pool.google_compute_network_peering_routes_config.peering_routes[0]: Still creating... [10s elapsed]
module.tf_private_pool.google_cloudbuild_worker_pool.private_pool: Still creating... [10s elapsed]
module.tf_private_pool.google_compute_network_peering_routes_config.peering_routes[0]: Creation complete after 11s [id=projects/prj-b-cicd-wm4z/global/networks/vpc-b-cbpools/networkPeerings/servicenetworking-googleapis-com]
module.tf_private_pool.google_cloudbuild_worker_pool.private_pool: Still creating... [20s elapsed]

https://ccticei@dev.azure.com/ccticei/Migration/_git/TEF-GCP-LZ-HS

odule.build_terraform_image.null_resource.run_command[0] (local-exec):     timeout: 1200s
module.build_terraform_image.null_resource.run_command[0] (local-exec): name: operations/build/prj-b-cicd-wm4z/YTRmODk0MTEtYWNiZi00NDZkLTgwMTAtMThmOWFmNjhiOTAx
module.build_terraform_image.null_resource.run_command[0]: Creation complete after 3s [id=6137778600788507520]

Apply complete! Resources: 73 added, 0 changed, 0 destroyed.

Outputs:

bootstrap_step_terraform_service_account_email = "sa-terraform-bootstrap@prj-b-seed-31ca.iam.gserviceaccount.com"
cloud_build_peered_network_id = "projects/prj-b-cicd-wm4z/global/networks/vpc-b-cbpools"
cloud_build_private_worker_pool_id = "projects/prj-b-cicd-wm4z/locations/us-central1/workerPools/private-pool-4ika"
cloud_build_worker_peered_ip_range = "192.168.0.0/24"
cloud_build_worker_range_id = "projects/prj-b-cicd-wm4z/global/addresses/ga-b-cbpools-worker-pool-range"
cloud_builder_artifact_repo = "projects/prj-b-cicd-wm4z/locations/us-central1/repositories/tf-runners"
cloudbuild_project_id = "prj-b-cicd-wm4z"
common_config = {
  "billing_account" = "012EDD-5AD5ED-ECFF0B"
  "bootstrap_folder_name" = "folders/865611452734"
  "default_region" = "us-central1"
  "folder_prefix" = "fldr"
  "org_id" = "630259462753"
  "parent_folder" = "1078109772786"
  "parent_id" = "folders/1078109772786"
  "project_prefix" = "prj"
}
csr_repos = {
  "gcp-bootstrap" = {
    "id" = "projects/prj-b-cicd-wm4z/repos/gcp-bootstrap"
    "name" = "gcp-bootstrap"
    "project" = "prj-b-cicd-wm4z"
    "url" = "https://source.developers.google.com/p/prj-b-cicd-wm4z/r/gcp-bootstrap"
  }
  "gcp-environments" = {
    "id" = "projects/prj-b-cicd-wm4z/repos/gcp-environments"
    "name" = "gcp-environments"
    "project" = "prj-b-cicd-wm4z"
    "url" = "https://source.developers.google.com/p/prj-b-cicd-wm4z/r/gcp-environments"
  }
  "gcp-networks" = {
    "id" = "projects/prj-b-cicd-wm4z/repos/gcp-networks"
    "name" = "gcp-networks"
    "project" = "prj-b-cicd-wm4z"
    "url" = "https://source.developers.google.com/p/prj-b-cicd-wm4z/r/gcp-networks"
  }
  "gcp-org" = {
    "id" = "projects/prj-b-cicd-wm4z/repos/gcp-org"
    "name" = "gcp-org"
    "project" = "prj-b-cicd-wm4z"
    "url" = "https://source.developers.google.com/p/prj-b-cicd-wm4z/r/gcp-org"
  }
  "gcp-policies" = {
    "id" = "projects/prj-b-cicd-wm4z/repos/gcp-policies"
    "name" = "gcp-policies"
    "project" = "prj-b-cicd-wm4z"
    "url" = "https://source.developers.google.com/p/prj-b-cicd-wm4z/r/gcp-policies"
  }
  "gcp-projects" = {
    "id" = "projects/prj-b-cicd-wm4z/repos/gcp-projects"
    "name" = "gcp-projects"
    "project" = "prj-b-cicd-wm4z"
    "url" = "https://source.developers.google.com/p/prj-b-cicd-wm4z/r/gcp-projects"
  }
  "tf-cloudbuilder" = {
    "id" = "projects/prj-b-cicd-wm4z/repos/tf-cloudbuilder"
    "name" = "tf-cloudbuilder"
    "project" = "prj-b-cicd-wm4z"
    "url" = "https://source.developers.google.com/p/prj-b-cicd-wm4z/r/tf-cloudbuilder"
  }
}
environment_step_terraform_service_account_email = "sa-terraform-env@prj-b-seed-31ca.iam.gserviceaccount.com"
gcs_bucket_cloudbuild_artifacts = {
  "bootstrap" = "bkt-prj-b-cicd-wm4z-gcp-bootstrap-build-artifacts"
  "env" = "bkt-prj-b-cicd-wm4z-gcp-environments-build-artifacts"
  "net" = "bkt-prj-b-cicd-wm4z-gcp-networks-build-artifacts"
  "org" = "bkt-prj-b-cicd-wm4z-gcp-org-build-artifacts"
  "proj" = "bkt-prj-b-cicd-wm4z-gcp-projects-build-artifacts"
}
gcs_bucket_cloudbuild_logs = {
  "bootstrap" = "bkt-prj-b-cicd-wm4z-gcp-bootstrap-build-logs"
  "env" = "bkt-prj-b-cicd-wm4z-gcp-environments-build-logs"
  "net" = "bkt-prj-b-cicd-wm4z-gcp-networks-build-logs"
  "org" = "bkt-prj-b-cicd-wm4z-gcp-org-build-logs"
  "proj" = "bkt-prj-b-cicd-wm4z-gcp-projects-build-logs"
}
gcs_bucket_tfstate = "bkt-prj-b-seed-tfstate-cca4"
group_billing_admins = "gcp-billing-admins@obrienlabs.app"
group_org_admins = "gcp-organization-admins@obrienlabs.app"
networks_step_terraform_service_account_email = "sa-terraform-net@prj-b-seed-31ca.iam.gserviceaccount.com"
optional_groups = {}
organization_step_terraform_service_account_email = "sa-terraform-org@prj-b-seed-31ca.iam.gserviceaccount.com"
projects_gcs_bucket_tfstate = "bkt-prj-b-seed-31ca-gcp-projects-tfstate"
projects_step_terraform_service_account_email = "sa-terraform-proj@prj-b-seed-31ca.iam.gserviceaccount.com"
required_groups = {}
seed_project_id = "prj-b-seed-31ca"

0-bootstrap is up 1000

1-environments dev branch only

Fortinet

SDN connector today for fortinet meet no config for vdoms after ha cluster deployed

enable-vdom - config fails right away use jump box - no default route needed - use standard subnet route (don't need vdom) add jump box to tf

can we download the config from fortinet help with the integration of their example

1-org

0-bootstrap inventory

cicd and seed projects

Cloud Source Repositories

all empty except for tf-cloudbuilder as expected

preparing for 1-org

michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ terraform output
bootstrap_step_terraform_service_account_email = "sa-terraform-bootstrap@prj-b-seed-31ca.iam.gserviceaccount.com"
cloud_build_peered_network_id = "projects/prj-b-cicd-wm4z/global/networks/vpc-b-cbpools"
cloud_build_private_worker_pool_id = "projects/prj-b-cicd-wm4z/locations/us-central1/workerPools/private-pool-4ika"
cloud_build_worker_peered_ip_range = "192.168.0.0/24"
cloud_build_worker_range_id = "projects/prj-b-cicd-wm4z/global/addresses/ga-b-cbpools-worker-pool-range"
cloud_builder_artifact_repo = "projects/prj-b-cicd-wm4z/locations/us-central1/repositories/tf-runners"
cloudbuild_project_id = "prj-b-cicd-wm4z"
common_config = {
  "billing_account" = "012...B"
  "bootstrap_folder_name" = "folders/865611452734"
  "default_region" = "us-central1"
  "folder_prefix" = "fldr"
  "org_id" = "630259462753"
  "parent_folder" = "1078109772786"
  "parent_id" = "folders/1078109772786"
  "project_prefix" = "prj"
}
csr_repos = {
  "gcp-bootstrap" = {
    "id" = "projects/prj-b-cicd-wm4z/repos/gcp-bootstrap"
    "name" = "gcp-bootstrap"
    "project" = "prj-b-cicd-wm4z"
    "url" = "https://source.developers.google.com/p/prj-b-cicd-wm4z/r/gcp-bootstrap"
  }
  "gcp-environments" = {
    "id" = "projects/prj-b-cicd-wm4z/repos/gcp-environments"
    "name" = "gcp-environments"
    "project" = "prj-b-cicd-wm4z"
    "url" = "https://source.developers.google.com/p/prj-b-cicd-wm4z/r/gcp-environments"
  }
  "gcp-networks" = {
    "id" = "projects/prj-b-cicd-wm4z/repos/gcp-networks"
    "name" = "gcp-networks"
    "project" = "prj-b-cicd-wm4z"
    "url" = "https://source.developers.google.com/p/prj-b-cicd-wm4z/r/gcp-networks"
  }
  "gcp-org" = {
    "id" = "projects/prj-b-cicd-wm4z/repos/gcp-org"
    "name" = "gcp-org"
    "project" = "prj-b-cicd-wm4z"
    "url" = "https://source.developers.google.com/p/prj-b-cicd-wm4z/r/gcp-org"
  }
  "gcp-policies" = {
    "id" = "projects/prj-b-cicd-wm4z/repos/gcp-policies"
    "name" = "gcp-policies"
    "project" = "prj-b-cicd-wm4z"
    "url" = "https://source.developers.google.com/p/prj-b-cicd-wm4z/r/gcp-policies"
  }
  "gcp-projects" = {
    "id" = "projects/prj-b-cicd-wm4z/repos/gcp-projects"
    "name" = "gcp-projects"
    "project" = "prj-b-cicd-wm4z"
    "url" = "https://source.developers.google.com/p/prj-b-cicd-wm4z/r/gcp-projects"
  }
  "tf-cloudbuilder" = {
    "id" = "projects/prj-b-cicd-wm4z/repos/tf-cloudbuilder"
    "name" = "tf-cloudbuilder"
    "project" = "prj-b-cicd-wm4z"
    "url" = "https://source.developers.google.com/p/prj-b-cicd-wm4z/r/tf-cloudbuilder"
  }
}
environment_step_terraform_service_account_email = "sa-terraform-env@prj-b-seed-31ca.iam.gserviceaccount.com"
gcs_bucket_cloudbuild_artifacts = {
  "bootstrap" = "bkt-prj-b-cicd-wm4z-gcp-bootstrap-build-artifacts"
  "env" = "bkt-prj-b-cicd-wm4z-gcp-environments-build-artifacts"
  "net" = "bkt-prj-b-cicd-wm4z-gcp-networks-build-artifacts"
  "org" = "bkt-prj-b-cicd-wm4z-gcp-org-build-artifacts"
  "proj" = "bkt-prj-b-cicd-wm4z-gcp-projects-build-artifacts"
}
gcs_bucket_cloudbuild_logs = {
  "bootstrap" = "bkt-prj-b-cicd-wm4z-gcp-bootstrap-build-logs"
  "env" = "bkt-prj-b-cicd-wm4z-gcp-environments-build-logs"
  "net" = "bkt-prj-b-cicd-wm4z-gcp-networks-build-logs"
  "org" = "bkt-prj-b-cicd-wm4z-gcp-org-build-logs"
  "proj" = "bkt-prj-b-cicd-wm4z-gcp-projects-build-logs"
}
gcs_bucket_tfstate = "bkt-prj-b-seed-tfstate-cca4"
group_billing_admins = "gcp-billing-admins@obrienlabs.app"
group_org_admins = "gcp-organization-admins@obrienlabs.app"
networks_step_terraform_service_account_email = "sa-terraform-net@prj-b-seed-31ca.iam.gserviceaccount.com"
optional_groups = {}
organization_step_terraform_service_account_email = "sa-terraform-org@prj-b-seed-31ca.iam.gserviceaccount.com"
projects_gcs_bucket_tfstate = "bkt-prj-b-seed-31ca-gcp-projects-tfstate"
projects_step_terraform_service_account_email = "sa-terraform-proj@prj-b-seed-31ca.iam.gserviceaccount.com"
required_groups = {}
seed_project_id = "prj-b-seed-31ca"

follow step 8 of https://github.com/terraform-google-modules/terraform-example-foundation/blob/master/0-bootstrap/README.md#deploying-with-cloud-build

ichael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ export network_step_sa=$(terraform output -raw networks_step_terraform_service_account_email)
export projects_step_sa=$(terraform output -raw projects_step_terraform_service_account_email)
export projects_gcs_bucket_tfstate=$(terraform output -raw projects_gcs_bucket_tfstate)

echo "network step service account = ${network_step_sa}"
echo "projects step service account = ${projects_step_sa}"
echo "projects gcs bucket tfstate = ${projects_gcs_bucket_tfstate}"
network step service account = sa-terraform-net@prj-b-seed-31ca.iam.gserviceaccount.com
projects step service account = sa-terraform-proj@prj-b-seed-31ca.iam.gserviceaccount.com
projects gcs bucket tfstate = bkt-prj-b-seed-31ca-gcp-projects-tfstate

michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ export cloudbuild_project_id=$(terraform output -raw cloudbuild_project_id)
echo "cloud build project ID = ${cloudbuild_project_id}"
cloud build project ID = prj-b-cicd-wm4z

update backend.tf

michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ terraform init

Initializing the backend...
Initializing modules...

Initializing provider plugins...
- Reusing previous version of hashicorp/random from the dependency lock file
- Reusing previous version of hashicorp/time from the dependency lock file
- Reusing previous version of hashicorp/google-beta from the dependency lock file
- Reusing previous version of hashicorp/null from the dependency lock file
- Reusing previous version of hashicorp/external from the dependency lock file
- Reusing previous version of hashicorp/google from the dependency lock file
- Using previously-installed hashicorp/random v3.6.0
- Using previously-installed hashicorp/time v0.10.0
- Using previously-installed hashicorp/google-beta v5.19.0
- Using previously-installed hashicorp/null v3.2.2
- Using previously-installed hashicorp/external v2.3.3
- Using previously-installed hashicorp/google v5.19.0

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ export backend_bucket=$(terraform output -raw gcs_bucket_tfstate)
echo "backend_bucket = ${backend_bucket}"
backend_bucket = bkt-prj-b-seed-tfstate-cca4
michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ export backend_bucket_projects=$(terraform output -raw projects_gcs_bucket_tfstate)
echo "backend_bucket_projects = ${backend_bucket_projects}"
backend_bucket_projects = bkt-prj-b-seed-31ca-gcp-projects-tfstate
michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ cp backend.tf.example backend.tf
michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ cd ..
michael@cloudshell:~/tef-olapp/github/terraform-example-foundation (tef-olapp)$ for i in `find . -name 'backend.tf'`; do sed -i'' -e "s/UPDATE_ME/${backend_bucket}/" $i; done
michael@cloudshell:~/tef-olapp/github/terraform-example-foundation (tef-olapp)$ for i in `find . -name 'backend.tf'`; do sed -i'' -e "s/UPDATE_PROJECTS_BACKEND/${backend_bucket_projects}/" $i; done

cd 0-bootstrap
michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ cat backend.tf
/**
 * Copyright 2021 Google LLC
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *      http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

terraform {
  backend "gcs" {
    bucket = "bkt-prj-b-seed-tfstate-cca4"
    prefix = "terraform/bootstrap/state"
  }
}

michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ terraform init

Initializing the backend...
Acquiring state lock. This may take a few moments...
Do you want to copy existing state to the new backend?
  Pre-existing state was found while migrating the previous "local" backend to the
  newly configured "gcs" backend. No existing state was found in the newly
  configured "gcs" backend. Do you want to copy this state to the new "gcs"
  backend? Enter "yes" to copy and "no" to start with an empty state.

  Enter a value: yes

Successfully configured the backend "gcs"! Terraform will automatically
use this backend unless the backend configuration changes.
Initializing modules...

Initializing provider plugins...
- Reusing previous version of hashicorp/google from the dependency lock file
- Reusing previous version of hashicorp/random from the dependency lock file
- Reusing previous version of hashicorp/time from the dependency lock file
- Reusing previous version of hashicorp/google-beta from the dependency lock file
- Reusing previous version of hashicorp/null from the dependency lock file
- Reusing previous version of hashicorp/external from the dependency lock file
- Using previously-installed hashicorp/null v3.2.2
- Using previously-installed hashicorp/external v2.3.3
- Using previously-installed hashicorp/google v5.19.0
- Using previously-installed hashicorp/random v3.6.0
- Using previously-installed hashicorp/time v0.10.0
- Using previously-installed hashicorp/google-beta v5.19.0

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ terraform plan

No changes. Your infrastructure matches the configuration.

Terraform has compared your real infrastructure against your configuration and found no differences, so no changes are needed.

clone the policies CSR repo and commit backend changes

cd ../..

michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ cd ../..
michael@cloudshell:~/tef-olapp/github (tef-olapp)$ echo ${cloudbuild_project_id}
prj-b-cicd-wm4z
michael@cloudshell:~/tef-olapp/github (tef-olapp)$ gcloud source repos clone gcp-policies --project=${cloudbuild_project_id}

Cloning into '/home/michael/tef-olapp/github/gcp-policies'...
warning: You appear to have cloned an empty repository.
Project [prj-b-cicd-wm4z] repository [gcp-policies] was cloned to [/home/michael/tef-olapp/github/gcp-policies].
michael@cloudshell:~/tef-olapp/github (tef-olapp)$ cd gcp-policies/
michael@cloudshell:~/tef-olapp/github/gcp-policies (tef-olapp)$ git checkout -b main
Switched to a new branch 'main'
michael@cloudshell:~/tef-olapp/github/gcp-policies (tef-olapp)$ cp -RT ../terraform-example-foundation/policy-library/ .
michael@cloudshell:~/tef-olapp/github/gcp-policies (tef-olapp)$ ls
lib  policies
michael@cloudshell:~/tef-olapp/github/gcp-policies (tef-olapp)$ git add .
michael@cloudshell:~/tef-olapp/github/gcp-policies (tef-olapp)$ git commit -m 'Initialize policy library repo'
Author identity unknown

*** Please tell me who you are.

Run

  git config --global user.email "you@example.com"
  git config --global user.name "Your Name"

to set your account's default identity.
Omit --global to set the identity only in this repository.

fatal: unable to auto-detect email address (got 'michael@cs-606565321060-default.(none)')
michael@cloudshell:~/tef-olapp/github/gcp-policies (tef-olapp)$ git config --global user.email "michael@obrienlabs.org"
michael@cloudshell:~/tef-olapp/github/gcp-policies (tef-olapp)$ git config --global user.name "Michael OBrien"
michael@cloudshell:~/tef-olapp/github/gcp-policies (tef-olapp)$ git commit -m 'Initialize policy library repo'
[main (root-commit) ff10fd6] Initialize policy library repo
 112 files changed, 9682 insertions(+)
 create mode 100644 lib/constraints.rego
 create mode 100644 lib/util.rego
 create mode 100644 lib/util_test.rego
 create mode 100644 policies/constraints/appengine_versions.yaml
 create mode 100644 policies/constraints/bigquery_world_readable.yaml
 create mode 100644 policies/constraints/dnssec_prevent_rsasha1_ksk.yaml
 create mode 100644 policies/constraints/dnssec_prevent_rsasha1_zsk.yaml
 create mode 100644 policies/constraints/gke_allow_only_private_cluster.yaml
 create mode 100644 policies/constraints/gke_allowed_node_sa_scope.yaml
 create mode 100644 policies/constraints/gke_container_optimized_os.yaml
 create mode 100644 policies/constraints/gke_dashboard_disable.yaml
 create mode 100644 policies/constraints/gke_disable_default_service_account.yaml
 create mode 100644 policies/constraints/gke_disable_legacy_endpoints.yaml
 create mode 100644 policies/constraints/gke_enable_alias_ip_ranges.yaml
 create mode 100644 policies/constraints/gke_legacy_abac.yaml
 create mode 100644 policies/constraints/gke_master_authorized_networks_enabled.yaml
 create mode 100644 policies/constraints/gke_node_pool_auto_repair.yaml
 create mode 100644 policies/constraints/gke_node_pool_auto_upgrade.yaml
 create mode 100644 policies/constraints/gke_restrict_client_auth_methods.yaml
 create mode 100644 policies/constraints/gke_restrict_pod_traffic.yaml
 create mode 100644 policies/constraints/iam_deny_public.yaml
 create mode 100644 policies/constraints/network_enable_flow_logs.yaml
 create mode 100644 policies/constraints/network_enable_private_google_access.yaml
 create mode 100644 policies/constraints/restrict_fw_rules_rdp_world_open.yaml
 create mode 100644 policies/constraints/restrict_fw_rules_ssh_world_open.yaml
 create mode 100644 policies/constraints/restrict_fw_rules_world_open.yaml
 create mode 100644 policies/constraints/serviceusage_allow_basic_apis.yaml
 create mode 100644 policies/constraints/sql_public_ip.yaml
 create mode 100644 policies/constraints/sql_ssl.yaml
 create mode 100644 policies/constraints/storage_bucket_policy_only.yaml
 create mode 100644 policies/constraints/storage_denylist_public.yaml
 create mode 100644 policies/templates/gcp_allowed_resource_types.yaml
 create mode 100644 policies/templates/gcp_always_violates_v1.yaml
 create mode 100644 policies/templates/gcp_app_service_versions.yaml
 create mode 100644 policies/templates/gcp_appengine_location_v1.yaml
 create mode 100644 policies/templates/gcp_bigquery_cmek_encryption_v1.yaml
 create mode 100644 policies/templates/gcp_bigquery_dataset_world_readable_v1.yaml
 create mode 100644 policies/templates/gcp_bigquery_table_retention_v1.yaml
 create mode 100644 policies/templates/gcp_bq_dataset_location_v1.yaml
 create mode 100644 policies/templates/gcp_cmek_rotation_v1.yaml
 create mode 100644 policies/templates/gcp_cmek_settings_v1.yaml
 create mode 100644 policies/templates/gcp_compute_allowed_networks.yaml
 create mode 100644 policies/templates/gcp_compute_disk_resource_policies_v1.yaml
 create mode 100644 policies/templates/gcp_compute_external_ip_address.yaml
 create mode 100644 policies/templates/gcp_compute_ip_forward.yaml
 create mode 100644 policies/templates/gcp_compute_zone_v1.yaml
 create mode 100644 policies/templates/gcp_dataproc_location_v1.yaml
 create mode 100644 policies/templates/gcp_dnssec_prevent_rsasha1_v1.yaml
 create mode 100644 policies/templates/gcp_dnssec_v1.yaml
 create mode 100644 policies/templates/gcp_enforce_labels_v1.yaml
 create mode 100644 policies/templates/gcp_enforce_naming_v1.yaml
 create mode 100644 policies/templates/gcp_gke_allowed_node_sa_v1.yaml
 create mode 100644 policies/templates/gcp_gke_cluster_location.yaml
 create mode 100644 policies/templates/gcp_gke_cluster_version_v1.yaml
 create mode 100644 policies/templates/gcp_gke_container_optimized_os.yaml
 create mode 100644 policies/templates/gcp_gke_dashboard_v1.yaml
 create mode 100644 policies/templates/gcp_gke_disable_default_service_account_v1.yaml
 create mode 100644 policies/templates/gcp_gke_disable_legacy_endpoints_v1.yaml
 create mode 100644 policies/templates/gcp_gke_enable_alias_ip_ranges.yaml
 create mode 100644 policies/templates/gcp_gke_enable_private_endpoint.yaml
 create mode 100644 policies/templates/gcp_gke_enable_shielded_nodes_v1.yaml
 create mode 100644 policies/templates/gcp_gke_enable_stackdriver_kubernetes_engine_monitoring_v1.yaml
 create mode 100644 policies/templates/gcp_gke_enable_stackdriver_logging_v1.yaml
 create mode 100644 policies/templates/gcp_gke_enable_stackdriver_monitoring_v1.yaml
 create mode 100644 policies/templates/gcp_gke_enable_workload_identity_v1.yaml
 create mode 100644 policies/templates/gcp_gke_legacy_abac_v1.yaml
 create mode 100644 policies/templates/gcp_gke_master_authorized_networks_enabled_v1.yaml
 create mode 100644 policies/templates/gcp_gke_node_auto_repair_v1.yaml
 create mode 100644 policies/templates/gcp_gke_node_auto_upgrade_v1.yaml
 create mode 100644 policies/templates/gcp_gke_private_cluster_v1.yaml
 create mode 100644 policies/templates/gcp_gke_restrict_client_auth_methods_v1.yaml
 create mode 100644 policies/templates/gcp_gke_restrict_pod_traffic_v1.yaml
 create mode 100644 policies/templates/gcp_glb_external_ip_access_constraint_v1.yaml
 create mode 100644 policies/templates/gcp_iam_allow_ban_roles_v1.yaml
 create mode 100644 policies/templates/gcp_iam_allowed_bindings.yaml
 create mode 100644 policies/templates/gcp_iam_allowed_policy_member_domains.yaml
 create mode 100644 policies/templates/gcp_iam_audit_log.yaml
 create mode 100644 policies/templates/gcp_iam_custom_role_permissions_v1.yaml
 create mode 100644 policies/templates/gcp_iam_required_bindings_v1.yaml
 create mode 100644 policies/templates/gcp_iam_restrict_service_account_creation_v1.yaml
 create mode 100644 policies/templates/gcp_iam_restrict_service_account_key_age_v1.yaml
 create mode 100644 policies/templates/gcp_iam_restrict_service_account_key_type_v1.yaml
 create mode 100644 policies/templates/gcp_lb_forwarding_rules.yaml
 create mode 100644 policies/templates/gcp_network_enable_firewall_logs_v1.yaml
 create mode 100644 policies/templates/gcp_network_enable_flow_logs_v1.yaml
 create mode 100644 policies/templates/gcp_network_enable_private_google_access_v1.yaml
 create mode 100644 policies/templates/gcp_network_restrict_default_v1.yaml
 create mode 100644 policies/templates/gcp_network_routing_v1.yaml
 create mode 100644 policies/templates/gcp_resource_value_pattern_v1.yaml
 create mode 100644 policies/templates/gcp_restricted_firewall_rules_v1.yaml
 create mode 100644 policies/templates/gcp_serviceusage_allowed_services_v1.yaml
 create mode 100644 policies/templates/gcp_spanner_location_v1.yaml
 create mode 100644 policies/templates/gcp_sql_allowed_authorized_networks_v1.yaml
 create mode 100644 policies/templates/gcp_sql_backup_v1.yaml
 create mode 100644 policies/templates/gcp_sql_instance_type_v1.yaml
 create mode 100644 policies/templates/gcp_sql_location_v1.yaml
 create mode 100644 policies/templates/gcp_sql_maintenance_window_v1.yaml
 create mode 100644 policies/templates/gcp_sql_public_ip_v1.yaml
 create mode 100644 policies/templates/gcp_sql_ssl_v1.yaml
 create mode 100644 policies/templates/gcp_sql_world_readable_v1.yaml
 create mode 100644 policies/templates/gcp_storage_bucket_policy_only_v1.yaml
 create mode 100644 policies/templates/gcp_storage_bucket_retention_v1.yaml
 create mode 100644 policies/templates/gcp_storage_bucket_world_readable_v1.yaml
 create mode 100644 policies/templates/gcp_storage_cmek_encryption_v1.yaml
 create mode 100644 policies/templates/gcp_storage_location_v1.yaml
 create mode 100644 policies/templates/gcp_storage_logging_v1.yaml
 create mode 100644 policies/templates/gcp_vpc_sc_allowed_regions.yaml
 create mode 100644 policies/templates/gcp_vpc_sc_ensure_access_levels_v1.yaml
 create mode 100644 policies/templates/gcp_vpc_sc_ensure_project_v1.yaml
 create mode 100644 policies/templates/gcp_vpc_sc_ensure_services_v1.yaml
 create mode 100644 policies/templates/gcp_vpc_sc_ip_range_v1.yaml
 create mode 100644 policies/templates/gcp_vpc_sc_project_perimeter.yaml
michael@cloudshell:~/tef-olapp/github/gcp-policies (tef-olapp)$ git push --set-upstream origin main
Enumerating objects: 118, done.
Counting objects: 100% (118/118), done.
Delta compression using up to 4 threads
Compressing objects: 100% (118/118), done.
Writing objects: 100% (118/118), 72.63 KiB | 2.90 MiB/s, done.
Total 118 (delta 87), reused 0 (delta 0), pack-reused 0
remote: Resolving deltas: 100% (87/87)
remote: Waiting for private key checker: 112/112 objects left
To https://source.developers.google.com/p/prj-b-cicd-wm4z/r/gcp-policies
 * [new branch]      main -> main
Branch 'main' set up to track remote branch 'main' from 'origin'.

step 15 gcp bootstrap repo

michael@cloudshell:~/tef-olapp/github/gcp-policies (tef-olapp)$ cd ..
michael@cloudshell:~/tef-olapp/github (tef-olapp)$ gcloud source repos clone gcp-bootstrap --project=${cloudbuild_project_id}
Cloning into '/home/michael/tef-olapp/github/gcp-bootstrap'...
warning: You appear to have cloned an empty repository.
Project [prj-b-cicd-wm4z] repository [gcp-bootstrap] was cloned to [/home/michael/tef-olapp/github/gcp-bootstrap].

michael@cloudshell:~/tef-olapp/github (tef-olapp)$ cd gcp-bootstrap
michael@cloudshell:~/tef-olapp/github/gcp-bootstrap (tef-olapp)$ git checkout -b plan
Switched to a new branch 'plan'
michael@cloudshell:~/tef-olapp/github/gcp-bootstrap (tef-olapp)$ mkdir -p envs/shared
michael@cloudshell:~/tef-olapp/github/gcp-bootstrap (tef-olapp)$ cp -RT ../terraform-example-foundation/0-bootstrap/ ./envs/shared
michael@cloudshell:~/tef-olapp/github/gcp-bootstrap (tef-olapp)$ ls
envs
michael@cloudshell:~/tef-olapp/github/gcp-bootstrap (tef-olapp)$ cp ../terraform-example-foundation/build/cloudbuild-tf-* .
michael@cloudshell:~/tef-olapp/github/gcp-bootstrap (tef-olapp)$ cp ../terraform-example-foundation/build/tf-wrapper.sh .
michael@cloudshell:~/tef-olapp/github/gcp-bootstrap (tef-olapp)$ chmod 755 ./tf-wrapper.sh
michael@cloudshell:~/tef-olapp/github/gcp-bootstrap (tef-olapp)$ git add .
michael@cloudshell:~/tef-olapp/github/gcp-bootstrap (tef-olapp)$ git commit -m 'Initialize bootstrap repo'
[plan (root-commit) d5317d8] Initialize bootstrap repo
 62 files changed, 9949 insertions(+)
 create mode 100644 cloudbuild-tf-apply.yaml
 create mode 100644 cloudbuild-tf-plan.yaml
 create mode 100644 envs/shared/.gitignore
 create mode 100644 envs/shared/.terraform.lock.hcl
 create mode 100644 envs/shared/Dockerfile
 create mode 100644 envs/shared/README-GitHub.md
 create mode 100644 envs/shared/README-GitLab.md
 create mode 100644 envs/shared/README-Jenkins.md
 create mode 100644 envs/shared/README-Terraform-Cloud.md
 create mode 100644 envs/shared/README.md
 create mode 100644 envs/shared/backend.tf
 create mode 100644 envs/shared/backend.tf.cloud.example
 create mode 100644 envs/shared/backend.tf.example
 create mode 100644 envs/shared/bootstrap.json
 create mode 100644 envs/shared/bootstrap.tfplan
 create mode 100644 envs/shared/cb.tf
 create mode 100644 envs/shared/files/private_key_example.png
 create mode 100644 envs/shared/github.tf.example
 create mode 100644 envs/shared/gitlab.tf.example
 create mode 100644 envs/shared/groups.tf
 create mode 100644 envs/shared/jenkins.tf.example
 create mode 100644 envs/shared/main.tf
 create mode 100644 envs/shared/modules/cb-private-pool/README.md
 create mode 100644 envs/shared/modules/cb-private-pool/main.tf
 create mode 100644 envs/shared/modules/cb-private-pool/network.tf
 create mode 100644 envs/shared/modules/cb-private-pool/outputs.tf
 create mode 100644 envs/shared/modules/cb-private-pool/variables.tf
 create mode 100644 envs/shared/modules/cb-private-pool/versions.tf
 create mode 100644 envs/shared/modules/cb-private-pool/vpn_ha.tf
 create mode 100644 envs/shared/modules/gitlab-oidc/main.tf
 create mode 100644 envs/shared/modules/gitlab-oidc/outputs.tf
 create mode 100644 envs/shared/modules/gitlab-oidc/variables.tf
 create mode 100644 envs/shared/modules/gitlab-oidc/versions.tf
 create mode 100644 envs/shared/modules/jenkins-agent/README.md
 create mode 100755 envs/shared/modules/jenkins-agent/files/jenkins_gce_startup_script.sh
 create mode 100644 envs/shared/modules/jenkins-agent/main.tf
 create mode 100644 envs/shared/modules/jenkins-agent/outputs.tf
 create mode 100644 envs/shared/modules/jenkins-agent/variables.tf
 create mode 100644 envs/shared/modules/jenkins-agent/versions.tf
 create mode 100644 envs/shared/modules/jenkins-agent/vpn_ha.tf
 create mode 100644 envs/shared/modules/parent-iam-member/main.tf
 create mode 100644 envs/shared/modules/parent-iam-member/variables.tf
 create mode 100644 envs/shared/modules/parent-iam-member/versions.tf
 create mode 100644 envs/shared/modules/parent-iam-remove-role/main.tf
 create mode 100644 envs/shared/modules/parent-iam-remove-role/variables.tf
 create mode 100644 envs/shared/modules/parent-iam-remove-role/versions.tf
 create mode 100644 envs/shared/modules/tfc-agent-gke/README.md
 create mode 100644 envs/shared/modules/tfc-agent-gke/main.tf
 create mode 100644 envs/shared/modules/tfc-agent-gke/outputs.tf
 create mode 100644 envs/shared/modules/tfc-agent-gke/variables.tf
 create mode 100644 envs/shared/modules/tfc-agent-gke/versions.tf
 create mode 100644 envs/shared/onprem.md
 create mode 100644 envs/shared/outputs.tf
 create mode 100644 envs/shared/provider.tf
 create mode 100644 envs/shared/sa.tf
 create mode 100755 envs/shared/scripts/git_create_branches_helper.sh
 create mode 100755 envs/shared/scripts/push-to-repo.sh
 create mode 100644 envs/shared/terraform.tfvars
 create mode 100644 envs/shared/terraform_cloud.tf.example
 create mode 100644 envs/shared/variables.tf
 create mode 100644 envs/shared/versions.tf
 create mode 100755 tf-wrapper.sh
michael@cloudshell:~/tef-olapp/github/gcp-bootstrap (tef-olapp)$ git push --set-upstream origin plan
Enumerating objects: 76, done.
Counting objects: 100% (76/76), done.
Delta compression using up to 4 threads
Compressing objects: 100% (75/75), done.
Writing objects: 100% (76/76), 425.13 KiB | 7.73 MiB/s, done.
Total 76 (delta 22), reused 0 (delta 0), pack-reused 0
remote: Resolving deltas: 100% (22/22)
remote: Waiting for private key checker: 51/62 objects left
To https://source.developers.google.com/p/prj-b-cicd-wm4z/r/gcp-bootstrap
 * [new branch]      plan -> plan
Branch 'plan' set up to track remote branch 'plan' from 'origin'.

moving to 1-org

1-org

https://github.com/terraform-google-modules/terraform-example-foundation/blob/master/1-org/README.md#prerequisites

FinOps: turn data_access_logs_enabled off enable_hub_and_spoke variable to true

https://github.com/terraform-google-modules/terraform-example-foundation/blob/master/1-org/README.md#deploying-with-cloud-build

michael@cloudshell:~/tef-olapp/github/gcp-bootstrap (tef-olapp)$ cd ..
michael@cloudshell:~/tef-olapp/github (tef-olapp)$ export CLOUD_BUILD_PROJECT_ID=$(terraform -chdir="terraform-example-foundation/0-bootstrap/" output -raw cloudbuild_project_id)
echo ${CLOUD_BUILD_PROJECT_ID}

gcloud source repos clone gcp-org --project=${CLOUD_BUILD_PROJECT_ID}
prj-b-cicd-wm4z
Cloning into '/home/michael/tef-olapp/github/gcp-org'...
warning: You appear to have cloned an empty repository.
Project [prj-b-cicd-wm4z] repository [gcp-org] was cloned to [/home/michael/tef-olapp/github/gcp-org].

michael@cloudshell:~/tef-olapp/github (tef-olapp)$ ls
_CloudLandingZone-main  gcp-bootstrap  gcp-org  gcp-policies  terraform-example-foundation

michael@cloudshell:~/tef-olapp/github (tef-olapp)$ cd gcp-org
michael@cloudshell:~/tef-olapp/github/gcp-org (tef-olapp)$ git checkout -b plan
Switched to a new branch 'plan'
michael@cloudshell:~/tef-olapp/github/gcp-org (tef-olapp)$ cp -RT ../terraform-example-foundation/1-org/ .
michael@cloudshell:~/tef-olapp/github/gcp-org (tef-olapp)$ cp ../terraform-example-foundation/build/cloudbuild-tf-* .
michael@cloudshell:~/tef-olapp/github/gcp-org (tef-olapp)$ cp ../terraform-example-foundation/build/tf-wrapper.sh .
michael@cloudshell:~/tef-olapp/github/gcp-org (tef-olapp)$ chmod 755 ./tf-wrapper.sh
michael@cloudshell:~/tef-olapp/github/gcp-org (tef-olapp)$ mv ./envs/shared/terraform.example.tfvars ./envs/shared/terraform.tfvars

raised for security command center notifications


michael@cloudshell:~/tef-olapp/github/gcp-org (tef-olapp)$ export ORGANIZATION_ID=$(terraform -chdir="../terraform-example-foundation/0-bootstrap/" output -json common_config | jq '.org_id' --raw-output)
michael@cloudshell:~/tef-olapp/github/gcp-org (tef-olapp)$ echo $ORGANIZATION_ID 
630259462753
michael@cloudshell:~/tef-olapp/github/gcp-org (tef-olapp)$ gcloud scc notifications describe "scc-notify" --organization=${ORGANIZATION_ID}
ERROR: (gcloud.scc.notifications.describe) PERMISSION_DENIED: Permission 'securitycenter.notificationconfig.get' denied on resource '//securitycenter.googleapis.com/organizations/630259462753/notificationConfigs/scc-notify' (or it may not exist).
- '@type': type.googleapis.com/google.rpc.ErrorInfo
  domain: securitycenter.googleapis.com
  metadata:
    permission: securitycenter.notificationconfig.get
    resource: organizations/630259462753/notificationConfigs/scc-notify
  reason: IAM_PERMISSION_DENIED
michael@cloudshell:~/tef-olapp/github/gcp-org (tef-olapp)$ gcloud scc notifications describe "scc-notify" --organization=${ORGANIZATION_ID}

after setting permission

michael@cloudshell:~/tef-olapp/github/gcp-org (tef-olapp)$ gcloud scc notifications describe "scc-notify" --organization=${ORGANIZATION_ID}
ERROR: (gcloud.scc.notifications.describe) INVALID_ARGUMENT: Security Command Center Legacy has been permanently disabled as of June 7, 2021. Migrate to Security Command Center's Standard tier or Premium tier to maintain access to Security Command Center. See https://cloud.google.com/security-command-center/docs/quickstart-security-command-center for more info.

enable Security Command Center

free version

skip data residency for now because I am testing in us-central1 not northamerica-northeast1

now grant roles that caused issues in https://github.com/terraform-google-modules/terraform-example-foundation/issues/1145

rerun

michael@cloudshell:~/tef-olapp/github/gcp-org (tef-olapp)$ gcloud scc notifications describe "scc-notify" --organization=${ORGANIZATION_ID}
ERROR: (gcloud.scc.notifications.describe) NOT_FOUND: Requested entity was not found.
michael@cloudshell:~/tef-olapp/github/gcp-org (tef-olapp)$

enable SSC on project -allready enabled

api call is deprecated

issue with step 5 - ACM Policy https://github.com/terraform-google-modules/terraform-example-foundation/issues/1146 Same as #1145

step 5 https://github.com/terraform-google-modules/terraform-example-foundation/blob/master/1-org/README.md#deploying-with-cloud-build

michael@cloudshell:~/tef-olapp/github/gcp-org (tef-olapp)$ export ACCESS_CONTEXT_MANAGER_ID=$(gcloud access-context-manager policies list --organization ${ORGANIZATION_ID} --format="value(name)")
echo "access_context_manager_policy_id = ${ACCESS_CONTEXT_MANAGER_ID}"
ERROR: (gcloud.access-context-manager.policies.list) PERMISSION_DENIED: The caller does not have permission
access_context_manager_policy_id =

1-org step 5 continued

step 5 of https://github.com/terraform-google-modules/terraform-example-foundation/blob/master/1-org/README.md#deploying-with-cloud-build

michael@cloudshell:~/tef-olapp/github/gcp-org (tef-olapp)$ export ACCESS_CONTEXT_MANAGER_ID=$(gcloud access-context-manager policies list --organization ${ORGANIZATION_ID} --format="value(name)")ERROR: (gcloud.access-context-manager.policies.list) PERMISSION_DENIED: The caller does not have permission

fix add to super admin - "Access Context Manager Admin"

no ACM policies yet

michael@cloudshell:~/tef-olapp/github/gcp-org (tef-olapp)$ export ACCESS_CONTEXT_MANAGER_ID=$(gcloud access-context-manager policies list --organization ${ORGANIZATION_ID} --format="value(name)")michael@cloudshell:~/tef-olapp/github/gcp-org (tef-olapp)$ echo "access_context_manager_policy_id = ${ACCESS_CONTEXT_MANAGER_ID}"
access_context_manager_policy_id = 

michael@cloudshell:~/tef-olapp/github/gcp-org (tef-olapp)$ gcloud access-context-manager policies list --organization ${ORGANIZATION_ID}
Listed 0 items.

updated https://github.com/terraform-google-modules/terraform-example-foundation/issues/1146

1-org step 6 - ACM false

in the gcp-org leave the tfvars variable commented https://github.com/terraform-google-modules/terraform-example-foundation/blob/master/1-org/envs/shared/terraform.example.tfvars#L34

//create_access_context_manager_access_policy = false

1-org step 6 - update tfvars variables

https://github.com/terraform-google-modules/terraform-example-foundation/blob/master/1-org/envs/shared/README.md

domains_to_allow = ["example.com"]
essential_contacts_domains_to_allow = ["@example.com"]
billing_data_users = "gcp-billing-data-users@example.com"
audit_data_users = "gcp-security-admins@example.com"
scc_notification_name = "scc-notify"
remote_state_bucket = "REMOTE_STATE_BUCKET"

semi-automated

michael@cloudshell:~/tef-olapp/github/gcp-org (tef-olapp)$ echo "remote_state_bucket = ${backend_bucket}"
remote_state_bucket = bkt-prj-b-seed-tfstate-cca4
sed -i'' -e "s/REMOTE_STATE_BUCKET/${backend_bucket}/" ./envs/shared/terraform.tfvars

writes
remote_state_bucket = "bkt-prj-b-seed-tfstate-cca4"

michael@cloudshell:~/tef-olapp/github/gcp-org (tef-olapp)$ if [ ! -z "${ACCESS_CONTEXT_MANAGER_ID}" ]; then sed -i'' -e "s=//create_access_context_manager_access_policy=create_access_context_manager_access_policy=" ./envs/shared/terraform.tfvars; fi
michael@cloudshell:~/tef-olapp/github/gcp-org (tef-olapp)$

manual

domains_to_allow = ["obr...pp"]
essential_contacts_domains_to_allow = ["@obr..p"]
billing_data_users = "gcp-billing-data-users@ob..p"
audit_data_users = "gcp-security-admins@ob..p"

1-org - step 7 initial commit to gcp-org repo

michael@cloudshell:~/tef-olapp/github/gcp-org (tef-olapp)$ git status
On branch plan

No commits yet

Untracked files:
  (use "git add <file>..." to include in what will be committed)
        .gitignore
        README.md
        cloudbuild-tf-apply.yaml
        cloudbuild-tf-plan.yaml
        envs/
        modules/
        tf-wrapper.sh

nothing added to commit but untracked files present (use "git add" to track)
michael@cloudshell:~/tef-olapp/github/gcp-org (tef-olapp)$ git diff
michael@cloudshell:~/tef-olapp/github/gcp-org (tef-olapp)$ git add .
michael@cloudshell:~/tef-olapp/github/gcp-org (tef-olapp)$ git status
On branch plan

No commits yet

Changes to be committed:
  (use "git rm --cached <file>..." to unstage)
        new file:   .gitignore
        new file:   README.md
        new file:   cloudbuild-tf-apply.yaml
        new file:   cloudbuild-tf-plan.yaml
        new file:   envs/shared/README.md
        new file:   envs/shared/backend.tf
        new file:   envs/shared/backend.tf.cloud.example
        new file:   envs/shared/cai_monitoring.tf
        new file:   envs/shared/essential_contacts.tf
        new file:   envs/shared/folders.tf
        new file:   envs/shared/iam.tf
        new file:   envs/shared/log_sinks.tf
        new file:   envs/shared/org_policy.tf
        new file:   envs/shared/outputs.tf
        new file:   envs/shared/projects.tf
        new file:   envs/shared/providers.tf
        new file:   envs/shared/remote.tf
        new file:   envs/shared/remote.tf.cloud.example
        new file:   envs/shared/scc_notification.tf
        new file:   envs/shared/tags.tf
        new file:   envs/shared/terraform.tfvars
        new file:   envs/shared/variables.tf
        new file:   envs/shared/versions.tf
        new file:   modules/cai-monitoring/README.md
        new file:   modules/cai-monitoring/function-source/index.js
        new file:   modules/cai-monitoring/function-source/package-lock.json
        new file:   modules/cai-monitoring/function-source/package.json
        new file:   modules/cai-monitoring/iam.tf
        new file:   modules/cai-monitoring/main.tf
        new file:   modules/cai-monitoring/outputs.tf
        new file:   modules/cai-monitoring/providers.tf
        new file:   modules/cai-monitoring/variables.tf
        new file:   modules/cai-monitoring/versions.tf
        new file:   modules/centralized-logging/README.md
        new file:   modules/centralized-logging/main.tf
        new file:   modules/centralized-logging/outputs.tf
        new file:   modules/centralized-logging/variables.tf
        new file:   modules/centralized-logging/versions.tf
        new file:   modules/network/main.tf
        new file:   modules/network/outputs.tf
        new file:   modules/network/variables.tf
        new file:   modules/network/versions.tf
        new file:   tf-wrapper.sh

git commit -m 'Initialize org repo'

1-org - step 8 push to gcp-org repo

michael@cloudshell:~/tef-olapp/github/gcp-org (tef-olapp)$ git push --set-upstream origin plan
Enumerating objects: 52, done.
Counting objects: 100% (52/52), done.
Delta compression using up to 4 threads
Compressing objects: 100% (51/51), done.
Writing objects: 100% (52/52), 57.79 KiB | 3.04 MiB/s, done.
Total 52 (delta 11), reused 0 (delta 0), pack-reused 0
remote: Resolving deltas: 100% (11/11)
remote: Waiting for private key checker: 43/43 objects left
To https://source.developers.google.com/p/prj-b-cicd-wm4z/r/gcp-org
 * [new branch]      plan -> plan
Branch 'plan' set up to track remote branch 'plan' from 'origin'.

1-org - step 9 merge to the production branch - check cloud build

michael@cloudshell:~/tef-olapp/github/gcp-org (tef-olapp)$ git checkout -b production
Switched to a new branch 'production'
michael@cloudshell:~/tef-olapp/github/gcp-org (tef-olapp)$ git push origin production
Total 0 (delta 0), reused 0 (delta 0), pack-reused 0
To https://source.developers.google.com/p/prj-b-cicd-wm4z/r/gcp-org
 * [new branch]      production -> production

https://console.cloud.google.com/cloud-build/triggers;region=us-central1?hl=en&project=prj-b-cicd-wm4z

1-org - step 9 checking tf plan issues - was due to bootstrap

gcp-org - plan

**************************************************
data.terraform_remote_state.bootstrap: Reading...
module.cai_monitoring.data.archive_file.function_source_zip: Reading...
module.cai_monitoring.data.archive_file.function_source_zip: Read complete after 0s [id=1e9314009f01646867d2cae991af75d380d72df9]
module.org_domain_restricted_sharing.data.google_organization.orgs["obrienlabs.app"]: Reading...
module.org_domain_restricted_sharing.data.google_organization.orgs["obrienlabs.app"]: Read complete after 0s [id=organizations/630259462753]

Error: Error loading state error

  with data.terraform_remote_state.bootstrap,
  on remote.tf line 38, in data "terraform_remote_state" "bootstrap":
  38:   backend = "gcs"

error loading the remote state: 16 problems:

- unsupported checkable object kind "var"

same issue for gcp-bootstrap

Initializing the backend...

Successfully configured the backend "gcs"! Terraform will automatically
use this backend unless the backend configuration changes.
Error loading state: 16 problems:

- unsupported checkable object kind "var"

Terraform 1.3.0 (docker) and 1.7.4 (console) mismatch may require terraform downgrade - before creating state file

it looks like we have the existing terraform version mismatch error

see https://github.com/terraform-google-modules/terraform-example-foundation/issues/1149 and https://discuss.hashicorp.com/t/failed-to-load-state-unsupported-checkable-object-kind-var/61844

1-org tf plan check step 9 requires a 0-bootstrap plan check on backend validation errors - due to terraform 1.3.0 (docker) and 1.7.4 (console) mismatch - may require console terraform downgrade before starting deployment

1-org step 9 terraform version mismatch mitigation

I was hoping to avoid a terraform downgrade until we get into refactoring but it looks like a 1.7.4 to 1.3.0 TF downgrade is required before creating the state file - or we get into a situation where the cloud build triggered plan running 1.3.0 cannot reconcile with the state file created in the gcp console under 1.7.4. Either we restart the deployment after downgrading or upgrade the TEF to TF 1.7 - will need to do this before moving on to 2-environments

The validation script needs to be updated https://github.com/terraform-google-modules/terraform-example-foundation/blob/master/scripts/validate-requirements.sh#L94

TF_VERSION="1.3.0"
...
        TERRAFORM_CURRENT_VERSION=$(terraform version -json | jq -r .terraform_version)
        if [ "$(compare_version "$TERRAFORM_CURRENT_VERSION" "$TF_VERSION")" -gt 1 ]; then
            echo_wrong_version "Terraform" "greater than or equal to" "$TF_VERSION" "https://learn.hashicorp.com/tutorials/terraform/install-cli" "$TERRAFORM_CURRENT_VERSION"
            ERRORS+=$'  Terraform version is incompatible.\n'
        fi

change

-greater than or equal to
+equal to

-        if [ "$(compare_version "$TERRAFORM_CURRENT_VERSION" "$TF_VERSION")" -gt 1 ]; then
 +       if [ "$(compare_version "$TERRAFORM_CURRENT_VERSION" "$TF_VERSION")" -eq 1 ]; then
## test this change first (locally in just a script unit test or in a new-org deploy)

Note we can go to 1.3.7 - on a minor version but 1.4+ is untested

Terraform local downgrade procedure - OSX (watch the sh line endings) - do windows 11 as well

get 1.3 https://releases.hashicorp.com/terraform/ https://releases.hashicorp.com/terraform/1.3.0/ https://releases.hashicorp.com/terraform/1.3.0/terraform_1.3.0_darwin_arm64.zip

drop terraform binary into a path folder
michaelobrien@mbp7 TEF-GCP-LZ-HS % terraform --version
Terraform v1.3.0
on darwin_arm64

Your version of Terraform is out of date! The latest version
is 1.7.4. You can update by downloading from https://www.terraform.io/downloads.html

Deploy 2 - clean org to test out onboarding adjustments and run with Terraform 1.3.0 downgrade

TEF V4 is our focus now as the base LZ with Fortigate integration from the above repo https://github.com/terraform-google-modules/terraform-example-foundation

follow previous: https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/issues/243 review previous issues: https://github.com/terraform-google-modules/terraform-example-foundation/issues/940

Org: olxyz repo https://github.com/CloudLandingZone/terraform-example-foundation issue https://github.com/terraform-google-modules/terraform-example-foundation/issues/1133

On a local macbook (check ls issue)

michaelobrien@mbp7 github % mkdir olxyz-1133
michaelobrien@mbp7 github % cd olxyz-1133 
michaelobrien@mbp7 olxyz-1133 % terraform version
Terraform v1.3.0
on darwin_arm64

Your version of Terraform is out of date! The latest version
is 1.7.4. You can update by downloading from https://www.terraform.io/downloads.html

create a new branch under the PR fork repo

gh1133-olxyz

michaelobrien@mbp7 olxyz-1133 % ssh-add ~/keys/obrie...thub  
Identity added: /Users/michaelobrien/keys/obrienl...hub (mich....org)
michaelobrien@mbp7 olxyz-1133 % git clone git@github.com:CloudLandingZone/terraform-example-foundation.git        
Cloning into 'terraform-example-foundation'...
remote: Enumerating objects: 9992, done.
remote: Counting objects: 100% (1037/1037), done.
remote: Compressing objects: 100% (594/594), done.
remote: Total 9992 (delta 641), reused 646 (delta 404), pack-reused 8955
Receiving objects: 100% (9992/9992), 2.85 MiB | 2.72 MiB/s, done.
Resolving deltas: 100% (7129/7129), done.

switch branches
michaelobrien@mbp7 terraform-example-foundation % git checkout gh1133-olxyz      
branch 'gh1133-olxyz' set up to track 'origin/gh1133-olxyz'.
Switched to a new branch 'gh1133-olxyz'

0 - bootstrap - local SDK login

michaelobrien@mbp7 terraform-example-foundation % gcloud auth login
Your browser has been opened to visit:

    https://accounts.google.com/o/oauth2/auth?response_type=code&client_id=32555940559.apps.googleusercontent.com&redirect_uri=http%3A%2F%2Flocalhost%3A8085%2F&scope=openid+https%3A%2F%2Fwww.googleap

You are now logged in as [michael@obrienlabs.xyz].
Your current project is [bigquery-ol].  You can change this setting by running:
  $ gcloud config set project PROJECT_ID

Updates are available for some Google Cloud CLI components.  To install them,
please run:
  $ gcloud components update

To take a quick anonymous survey, run:
  $ gcloud survey

michaelobrien@mbp7 terraform-example-foundation

reviewing with Fortinet

https://github.com/40net-cloud/fortinet-gcp-solutions https://github.com/40net-cloud/fortinet-gcp-solutions/tree/master/FortiGate/architectures/200-ha-active-passive-lb-sandwich

off the original Fortinet source

Discussion

dns filtering (policy on the fortigate - requests sent to block portal web page (anything from fortiguard Load balancing
require IP and httphost - lb is done via the httphost but IP is still required
check fortiweb or fortiadc
when L7 expected - check zero day attack notes

MCC for routing - check finops - integration into BGP via the cloud router that usually is free of cost as it comes with a VPN for inter-vpc or C2G or an interconnect for BGP propagation https://docs.fortinet.com/document/fortiswitch/6.4.2/administration-guide/939731/bgp-routing#:~:text=The%20FortiSwitch%20unit%20accepts%20routes,to%20its%20local%20routing%20tables. https://docs.fortinet.com/document/fortigate-public-cloud/7.4.0/gcp-administration-guide/214977/creating-a-gcp-route-table

Continue olxyz org local and cloud shell terraform 1.3 run

Marian: verify proxy subnets - why needed
Marian: terraform-modules - instead of ../ use a local repo - see email
dev subnet only has 1 - need public/private/db - pull out locals into into a module with the definition in the tfvars -
add option for one region only - not ha
org level collision between 2 deployments - disable scc scp for 2nd org - or use parameterized naming + randomized suffix (or timestamp like azure)
pull in 1-org changes 2 days ago in https://github.com/terraform-google-modules/terraform-example-foundation/pull/1110/files
run with TEF groups creation instead of cloud-setup (foundations)

Target bootstrap PR for the following

see scripting example in https://github.com/GoogleCloudPlatform/pubsec-declarative-toolkit/blob/gh766-script/solutions/setup.sh

create folder then use script

wait 2 min for propagation

folder id in this case is 736660879367

IAM SA roles

Billing Account Administrator
Compute Shared VPC Admin
Folder Admin
Folder Creator
Organization Administrator
Organization Policy Administrator
Project Billing Manager
Project Creator
Project Deleter
Project IAM Admin
Security Center Admin
Security Center Notification Configurations Editor
Service Account Token Creator
Service Usage Admin

michael@cloudshell:~$ ROOT_FOLDER_ID=736660879367
michael@cloudshell:~$ BOOT_PROJECT_ID=tef-olxyz
#BILLING_ID=$(gcloud billing projects describe $BOOT_PROJECT_ID $BILLING_FORMAT | sed 's/.*\///')
michael@cloudshell:~$ gcloud projects create "$BOOT_PROJECT_ID" --name="${BOOT_PROJECT_ID}" --set-as-default --folder="$ROOT_FOLDER_ID"
Create in progress for [https://cloudresourcemanager.googleapis.com/v1/projects/tef-olxyz].
Waiting for [operations/cp.9134984346595650639] to finish...done.                                                                                                                         
Enabling service [cloudapis.googleapis.com] on project [tef-olxyz]...
Operation "operations/acat.p2-438381210056-0f410b88-37c8-4953-baf3-3af50f7d4db1" finished successfully.
Updated property [core/project] to [tef-olxyz].
michael@cloudshell:~ (tef-olxyz)$ BILLING_ID=01BCCE-4EC0EE-DC58C8

michael@cloudshell:~ (tef-olxyz)$ SUPER_ADMIN_EMAIL=$(gcloud config list --format json|jq .core.account | sed 's/"//g')
michael@cloudshell:~ (tef-olxyz)$ echo $SUPER_ADMIN_EMAIL
mich...bs.xyz
michael@cloudshell:~ (tef-olxyz)$ ORG_ID=$(gcloud projects get-ancestors $BOOT_PROJECT_ID --format='get(id)' | tail -1)
michael@cloudshell:~ (tef-olxyz)$ gcloud beta billing projects link "${BOOT_PROJECT_ID}" --billing-account "${BILLING_ID}"

gcloud organizations add-iam-policy-binding ${ORG_ID}  --member=user:$SUPER_ADMIN_EMAIL --role=roles/billing.admin --quiet > /dev/null 1>&1
gcloud organizations add-iam-policy-binding ${ORG_ID}  --member=user:$SUPER_ADMIN_EMAIL --role=roles/accesscontextmanager.policyAdmin --quiet > /dev/null 1>&1
gcloud organizations add-iam-policy-binding ${ORG_ID}  --member=user:$SUPER_ADMIN_EMAIL --role=roles/billing.user --quiet > /dev/null 1>&1
gcloud organizations add-iam-policy-binding ${ORG_ID}  --member=user:$SUPER_ADMIN_EMAIL --role=roles/resourcemanager.folderAdmin --quiet > /dev/null 1>&1
gcloud organizations add-iam-policy-binding ${ORG_ID}  --member=user:$SUPER_ADMIN_EMAIL --role=roles/resourcemanager.organizationAdmin --quiet > /dev/null 1>&1
gcloud organizations add-iam-policy-binding ${ORG_ID}  --member=user:$SUPER_ADMIN_EMAIL --role=roles/resourcemanager.projectCreator --quiet > /dev/null 1>&1
gcloud organizations add-iam-policy-binding ${ORG_ID}  --member=user:$SUPER_ADMIN_EMAIL --role=roles/resourcemanager.projectDeleter --quiet > /dev/null 1>&1
gcloud organizations add-iam-policy-binding ${ORG_ID}  --member=user:$SUPER_ADMIN_EMAIL --role=roles/orgpolicy.policyAdmin --quiet > /dev/null 1>&1
gcloud organizations add-iam-policy-binding ${ORG_ID}  --member=user:$SUPER_ADMIN_EMAIL --role=roles/resourcemanager.projectIamAdmin --quiet > /dev/null 1>&1
gcloud organizations add-iam-policy-binding ${ORG_ID}  --member=user:$SUPER_ADMIN_EMAIL --role=roles/serviceusage.serviceUsageAdmin --quiet > /dev/null 1>&1
gcloud organizations add-iam-policy-binding ${ORG_ID}  --member=user:$SUPER_ADMIN_EMAIL --role=roles/storage.admin --quiet > /dev/null 1>&1
gcloud organizations add-iam-policy-binding ${ORG_ID}  --member=user:$SUPER_ADMIN_EMAIL --role=roles/iam.serviceAccountTokenCreator --quiet > /dev/null 1>&1

gcloud organizations add-iam-policy-binding ${ORG_ID}  --member=user:$SUPER_ADMIN_EMAIL --role=roles/securitycenter.admin --quiet > /dev/null 1>&1

bootstrap project

https://github.com/terraform-google-modules/terraform-example-foundation/issues/1139 https://github.com/terraform-google-modules/terraform-example-foundation/issues/1140 https://github.com/terraform-google-modules/terraform-example-foundation/issues/1142 https://github.com/terraform-google-modules/terraform-example-foundation/issues/1143


# check services
michael@cloudshell:~ (tef-olxyz)$ gcloud services list | grep NAME
NAME: analyticshub.googleapis.com
NAME: bigquery.googleapis.com
NAME: bigqueryconnection.googleapis.com
NAME: bigquerydatapolicy.googleapis.com
NAME: bigquerymigration.googleapis.com
NAME: bigqueryreservation.googleapis.com
NAME: bigquerystorage.googleapis.com
NAME: cloudapis.googleapis.com
NAME: cloudtrace.googleapis.com
NAME: dataform.googleapis.com
NAME: dataplex.googleapis.com
NAME: datastore.googleapis.com
NAME: logging.googleapis.com
NAME: monitoring.googleapis.com
NAME: servicemanagement.googleapis.com
NAME: serviceusage.googleapis.com
NAME: sql-component.googleapis.com
NAME: storage-api.googleapis.com
NAME: storage-component.googleapis.com
NAME: storage.googleapis.com

michael@cloudshell:~ (tef-olxyz)$ gcloud services enable cloudresourcemanager.googleapis.com
Operation "operations/acat.p2-438381210056-f7e0fb71-9240-434b-918d-6e4bfe1fadfb" finished successfully.
michael@cloudshell:~ (tef-olxyz)$ gcloud services enable cloudbilling.googleapis.com
Operation "operations/acat.p2-438381210056-9316b2a5-637a-40ad-bd64-981dc1af28f3" finished successfully.
michael@cloudshell:~ (tef-olxyz)$ gcloud services enable iam.googleapis.com
Operation "operations/acat.p2-438381210056-2344a0fc-dc11-4432-b9aa-53a89f0fbb4d" finished successfully.
michael@cloudshell:~ (tef-olxyz)$ gcloud services enable cloudkms.googleapis.com
Operation "operations/acat.p2-438381210056-4379d575-1d69-40b6-9104-113ff4f48704" finished successfully.
michael@cloudshell:~ (tef-olxyz)$ gcloud services enable servicenetworking.googleapis.com
Operation "operations/acat.p2-438381210056-b21bf432-2188-49c1-a9b9-44ec824f8d43" finished successfully.

# existing
michael@cloudshell:~ (tef-olxyz)$ gcloud services enable  accesscontextmanager.googleapis.com
Operation "operations/acat.p2-438381210056-66c8f99c-fe16-4481-8d64-e9d467f8fe8d" finished successfully.
michael@cloudshell:~ (tef-olxyz)$ gcloud services enable appengine.googleapis.com
Operation "operations/acat.p2-438381210056-4426e470-344b-4b1e-9d47-52a0f2befe21" finished successfully.
michael@cloudshell:~ (tef-olxyz)$ gcloud services enable artifactregistry.googleapis.com
Operation "operations/acat.p2-438381210056-0d5d0ccd-3845-4edf-a062-e3ff3448ff03" finished successfully.
michael@cloudshell:~ (tef-olxyz)$ gcloud services enable assuredworkloads.googleapis.com
Operation "operations/acat.p2-438381210056-f189974a-ab71-49e1-a81b-48bd896df8a0" finished successfully.
michael@cloudshell:~ (tef-olxyz)$ gcloud services enable billingbudgets.googleapis.com
Operation "operations/acat.p2-438381210056-7b38125e-23ec-4564-bbfc-277fc3297802" finished successfully.
michael@cloudshell:~ (tef-olxyz)$ gcloud services enable cloudasset.googleapis.com
gcloud services enable cloudbuild.googleapis.comOperation "operations/acat.p2-438381210056-bafe06bf-3786-4608-b994-3c3069f6042e" finished successfully.
michael@cloudshell:~ (tef-olxyz)$ gcloud services enable cloudbuild.googleapis.com
Operation "operations/acf.p2-438381210056-039eab93-9f6a-420b-943f-01297e27b339" finished successfully.
michael@cloudshell:~ (tef-olxyz)$ gcloud services enable cloudidentity.googleapis.com
Operation "operations/acat.p2-438381210056-7989271a-604a-4e7a-9c4e-4acd5e8ab8a5" finished successfully.
michael@cloudshell:~ (tef-olxyz)$ gcloud services enable containerregistry.googleapis.com
michael@cloudshell:~ (tef-olxyz)$ gcloud services enable essentialcontacts.googleapis.com
Operation "operations/acat.p2-438381210056-8230082d-b801-4096-a74c-4977faf1a85c" finished successfully.
michael@cloudshell:~ (tef-olxyz)$ gcloud services enable securitycenter.googleapis.com
Operation "operations/acat.p2-438381210056-e1a24825-5697-465d-ad46-1496b47f1b58" finished successfully.

# post check
michael@cloudshell:~ (tef-olxyz)$ gcloud services list | grep NAME 
NAME: accesscontextmanager.googleapis.com
NAME: analyticshub.googleapis.com
NAME: appengine.googleapis.com
NAME: artifactregistry.googleapis.com
NAME: assuredworkloads.googleapis.com
NAME: bigquery.googleapis.com
NAME: bigqueryconnection.googleapis.com
NAME: bigquerydatapolicy.googleapis.com
NAME: bigquerymigration.googleapis.com
NAME: bigqueryreservation.googleapis.com
NAME: bigquerystorage.googleapis.com
NAME: billingbudgets.googleapis.com
NAME: cloudapis.googleapis.com
NAME: cloudasset.googleapis.com
NAME: cloudbilling.googleapis.com
NAME: cloudbuild.googleapis.com
NAME: cloudidentity.googleapis.com
NAME: cloudkms.googleapis.com
NAME: cloudresourcemanager.googleapis.com
NAME: cloudtrace.googleapis.com
NAME: containerregistry.googleapis.com
NAME: dataform.googleapis.com
NAME: dataplex.googleapis.com
NAME: datastore.googleapis.com
NAME: essentialcontacts.googleapis.com
NAME: iam.googleapis.com
NAME: iamcredentials.googleapis.com
NAME: logging.googleapis.com
NAME: monitoring.googleapis.com
NAME: pubsub.googleapis.com
NAME: securitycenter.googleapis.com
NAME: securitycentermanagement.googleapis.com
NAME: servicemanagement.googleapis.com
NAME: servicenetworking.googleapis.com
NAME: serviceusage.googleapis.com
NAME: sql-component.googleapis.com
NAME: storage-api.googleapis.com
NAME: storage-component.googleapis.com
NAME: storage.googleapis.com

michael@cloudshell:~ (tef-olxyz)$ gcloud services list | grep NAME | wc -l
39

see issue https://github.com/terraform-google-modules/terraform-example-foundation/issues/1161

start repo clone and bootstrap deployment on olxyz with default tf 1.3 in cloud shell - for 1161 jira

michael@cloudshell:~ (tef-olxyz)$ mkdir tef-olxyz
michael@cloudshell:~ (tef-olxyz)$ cd tef-olxyz/
michael@cloudshell:~/tef-olxyz (tef-olxyz)$ mkdir github
michael@cloudshell:~/tef-olxyz (tef-olxyz)$ cd github/
michael@cloudshell:~/tef-olxyz/github (tef-olxyz)$ mkdir _CloudLandingZone-main
michael@cloudshell:~/tef-olxyz/github (tef-olxyz)$ git clone https://github.com/CloudLandingZone/terraform-example-foundation.git
Cloning into 'terraform-example-foundation'...
remote: Enumerating objects: 10045, done.
remote: Counting objects: 100% (1090/1090), done.
remote: Compressing objects: 100% (631/631), done.
remote: Total 10045 (delta 677), reused 669 (delta 416), pack-reused 8955
Receiving objects: 100% (10045/10045), 2.87 MiB | 9.41 MiB/s, done.
Resolving deltas: 100% (7165/7165), done.

prep

billing quota increase of 30 + 15 granted
project quota increase of 30 - pending

0-bootstrap

follow https://github.com/terraform-google-modules/terraform-example-foundation/tree/master/0-bootstrap#prerequisites

granted roles as above in the readme - as well as in

specifically

roles/securitycenter.admin https://github.com/terraform-google-modules/terraform-example-foundation/issues/1145
```
gcloud organizations add-iam-policy-binding ${ORG_ID}  --member=user:$SUPER_ADMIN_EMAIL --role=roles/securitycenter.admin --quiet > /dev/null 1>&1
```
see pull https://github.com/terraform-google-modules/terraform-example-foundation/pull/1175

20240404

pull latest master changes

From https://github.com/CloudLandingZone/terraform-example-foundation
   1dbe943..fdf67cb  gh1133-bootstrap-1136 -> origin/gh1133-bootstrap-1136
   dd6c09c..0a03623  master                -> origin/master
Updating 1dbe943..fdf67cb
Fast-forward
 1-org/envs/shared/README.md                    |   4 ++--
 1-org/envs/shared/log_sinks.tf                 |  14 ++++++-------
 1-org/envs/shared/outputs.tf                   |  16 +++++++-------
 1-org/modules/centralized-logging/README.md    |   6 +++---
 1-org/modules/centralized-logging/main.tf      | 150 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++--------------------------------------
 1-org/modules/centralized-logging/outputs.tf   |  16 +++++++-------
 1-org/modules/centralized-logging/variables.tf |  23 ++++++++++----------
 1-org/modules/centralized-logging/versions.tf  |   7 +++++++
 test/integration/org/org_test.go               |  74 ++++++++++++++++++++++++++++++++++++++++++++++++----------------
 9 files changed, 208 insertions(+), 102 deletions(-)

Current SA roles

Current services on the bootstrap of the bootstrap projects

michael@cloudshell:~/tef-olxyz/github/terraform-example-foundation (tef-olxyz)$ gcloud services list  | grep NAME
NAME: accesscontextmanager.googleapis.com
NAME: analyticshub.googleapis.com
NAME: appengine.googleapis.com
NAME: artifactregistry.googleapis.com
NAME: assuredworkloads.googleapis.com
NAME: bigquery.googleapis.com
NAME: bigqueryconnection.googleapis.com
NAME: bigquerydatapolicy.googleapis.com
NAME: bigquerymigration.googleapis.com
NAME: bigqueryreservation.googleapis.com
NAME: bigquerystorage.googleapis.com
NAME: billingbudgets.googleapis.com
NAME: cloudapis.googleapis.com
NAME: cloudasset.googleapis.com
NAME: cloudbilling.googleapis.com
NAME: cloudbuild.googleapis.com
NAME: cloudidentity.googleapis.com
NAME: cloudkms.googleapis.com
NAME: cloudresourcemanager.googleapis.com
NAME: cloudtrace.googleapis.com
NAME: containerregistry.googleapis.com
NAME: dataform.googleapis.com
NAME: dataplex.googleapis.com
NAME: datastore.googleapis.com
NAME: essentialcontacts.googleapis.com
NAME: iam.googleapis.com
NAME: iamcredentials.googleapis.com
NAME: logging.googleapis.com
NAME: monitoring.googleapis.com
NAME: pubsub.googleapis.com
NAME: securitycenter.googleapis.com
NAME: securitycentermanagement.googleapis.com
NAME: servicemanagement.googleapis.com
NAME: servicenetworking.googleapis.com
NAME: serviceusage.googleapis.com
NAME: sql-component.googleapis.com
NAME: storage-api.googleapis.com
NAME: storage-component.googleapis.com
NAME: storage.googleapis.com

Terraform version downgrade

to avoid TF state file errors during 1-org transition from 1.7 to 1.3 - https://github.com/terraform-google-modules/terraform-example-foundation/issues/1151 https://releases.hashicorp.com/terraform/ Current version is 1.8.0 in RC, in gcloud we are running

michael@cloudshell:~/tef-olxyz/github/terraform-example-foundation (tef-olxyz)$ terraform --version
Terraform v1.7.5
on linux_amd64

Installing local terraform 1.3.10

https://releases.hashicorp.com/terraform/1.3.10/ https://releases.hashicorp.com/terraform/1.3.10/terraform_1.3.10_linux_amd64.zip

3-networks-hub-and-spoke deployed OK in test org 2 https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/issues/360

PRs merged to the terraform-example-foundation upstream repo

20240426 https://github.com/terraform-google-modules/terraform-example-foundation/pull/1175

terraform-google-modules / terraform-example-foundation