GoogleCloudPlatform / pbmm-on-gcp-onboarding

GCP Canadian Public Sector Landing Zone overlay on top of the TEF via CFT modules - a secure cloud foundation
https://cloud.google.com/architecture/security-foundations
Apache License 2.0
45 stars 56 forks source link

Cloud Build module in bootstrap fails on CSR clone gcloud API auth during push-to-repo.sh for CB trigger creation on particular client - use ssh key and ssh-agent #431

Closed fmichaelobrien closed 2 months ago

fmichaelobrien commented 6 months ago

Pending to check

Todo

- check a 2nd super admin account

- Cloud shell optional workaround - use local shell (OSX, or google/cloud-sdk docker container on windows - see https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/issues/429

## Client Issue - Cloud Build module in bootstrap fails on CSR clone auth during push-to-repo.sh for CB trigger creation
- looks like the same CSR credential issue in the terraform apply - as in the shell account - nor TF service account being used here yet
- https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/blob/main/0-bootstrap/scripts/push-to-repo.sh#L32

gcloud source repos clone "${CSR_NAME}" "${tmp_dir}" --project "${CSR_PROJECT_ID}"

- standard tf 1.3.10 0-bootstrap on cloud shell
- passed tf apply 271 all the way to 5 remaining
- TF re apply "INVALID ARGUMENT" in cb module sh script - push-to-repo.sh:32
- 

<img width="1323" alt="Screenshot 2024-05-10 at 8 00 15 AM" src="https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/assets/94715080/e7e41bde-457a-4988-83a1-eafba82bc5ca">

<img width="1496" alt="Screenshot 2024-05-10 at 7 49 09 AM" src="https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/assets/94715080/c1995548-3cd8-4119-8c8b-cbd2832bc4ff">

- manual csr clone test

ERROR: (gcloud.source.repos.clone) Command '['git', 'clone', 'https://source.developers.google.com/p/prj-b-cicd-orcl/r/tf-cloudbuilder', '/home/USER/lz-tef-dev-.../temp/tf-cloudbuilder', '--config', 'credential.https://source.developers.google.com/.helper=', '--config', 'credential.https://source.developers.google.com/.helper=!gcloud auth git-helper --account=USER@gcp.....ca --ignore-unknown $@']' returned non-zero exit status 128.

USER@cloudshell:~/lz-tef-dev-.../temp (lz-tef-dev-...)$ gcloud config set project prj-b-cicd-orcl Updated property [core/project].

USER@cloudshell:~/lz-tef-dev-...2/temp (prj-b-cicd-orcl)$ gcloud source repos clone tf-cloudbuilder --project prj-b-cicd-orcl Cloning into '/home/USER/lz-tef-dev-...2/temp/tf-cloudbuilder'... remote: INVALID_ARGUMENT: Request contains an invalid argument

remote: [type.googleapis.com/google.rpc.LocalizedMessage] remote: locale: "en-US" remote: message: "Invalid authentication credentials. Please generate a new identifier: https://source.developers.google.com/new-password"

remote: remote: [type.googleapis.com/google.rpc.RequestInfo] remote: request_id: "9fe7..2308"

fatal: unable to access 'https://source.developers.google.com/p/prj-b-cicd-orcl/r/tf-cloudbuilder/': The requested URL returned error: 400

ERROR: (gcloud.source.repos.clone) Command '['git', 'clone', 'https://source.developers.google.com/p/prj-b-cicd-orcl/r/tf-cloudbuilder', '/home/USER/lz-tef-dev-...2/temp/tf-cloudbuilder', '--config', 'credential.https://source.developers.google.com/.helper=', '--config', 'credential.https://source.developers.google.com/.helper=!gcloud auth git-helper --account=USER@gcp....ca --ignore-unknown $@']' returned non-zero exit status 128.

USER@cloudshell:~/lz-tef-dev-...2/temp (prj-b-cicd-orcl)$ ^C


## Reproduction

## triage reference - full run in 360 and 421
- full run #360
- last main ci/cid #421

- see normal session timeout for 360 run in https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/issues/360#issuecomment-2041622470

module.tf_workspace["proj"].google_cloudbuild_trigger.triggers["apply"]: Creation complete after 1s [id=projects/prj-b-cicd-82vv/locations/us-central1/triggers/a1129dc9-c1e6-47d5-9a20-7518c82fe900] module.tf_workspace["bootstrap"].google_cloudbuild_trigger.triggers["apply"]: Creation complete after 1s [id=projects/prj-b-cicd-82vv/locations/us-central1/triggers/70966480-5d2c-4aa4-a7aa-1e0aaeb711f1] module.tf_workspace["proj"].google_storage_bucket_iam_member.artifacts_admin: Creation complete after 4s [id=b/bkt-prj-b-cicd-82vv-gcp-projects-build-artifacts/roles/storage.admin/serviceAccount:sa-terraform-proj@prj-b-seed-8919.iam.gserviceaccount.com] module.tf_workspace["org"].google_storage_bucket_iam_member.artifacts_admin: Creation complete after 4s [id=b/bkt-prj-b-cicd-82vv-gcp-org-build-artifacts/roles/storage.admin/serviceAccount:sa-terraform-org@prj-b-seed-8919.iam.gserviceaccount.com] ╷ │ Error: local-exec provisioner error │ │ with module.bootstrap_csr_repo.null_resource.run_command[0], │ on .terraform/modules/bootstrap_csr_repo/main.tf line 232, in resource "null_resource" "run_command": │ 232: provisioner "local-exec" { │ │ Error running command 'PATH=/google-cloud-sdk/bin:$PATH │ ./scripts/push-to-repo.sh prj-b-cicd-82vv tf-cloudbuilder ./Dockerfile │ ': exit status 1. Output: + '[' 3 -lt 3 ']' │ + CSR_PROJECT_ID=prj-b-cicd-82vv │ + CSR_NAME=tf-cloudbuilder │ + DOCKERFILE_PATH=./Dockerfile │ ++ mktemp -d │ + tmp_dir=/tmp/tmp.xMNfsxhn6Q │ + gcloud source repos clone tf-cloudbuilder /tmp/tmp.xMNfsxhn6Q --project prj-b-cicd-82vv │ ERROR: (gcloud.source.repos.clone) UNAUTHENTICATED: Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication │ credential. See https://developers.google.com/identity/sign-in/web/devconsole-project. │


## triage separate system oldev
- CSR local test

<img width="1223" alt="Screenshot 2024-05-10 at 7 39 56 AM" src="https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/assets/94715080/ec40a2a3-f986-46c7-9ea2-e534d010975e">

michael@cloudshell:~/tef-oldev3 (tef-oldev2)$ gcloud config set project tef-oldev3 Updated property [core/project]. michael@cloudshell:~/tef-oldev3 (tef-oldev3)$ ls pbmm-on-gcp-onboarding terraform michael@cloudshell:~/tef-oldev3 (tef-oldev3)$ mkdir _test_repo michael@cloudshell:~/tef-oldev3 (tef-oldev3)$ cd _test_repo michael@cloudshell:~/tef-oldev3/_test_repo (tef-oldev3)$ gcloud source repos clone gcp-policies --project=prj-b-cicd-fgbs Cloning into '/home/michael/tef-oldev3/_test_repo/gcp-policies'... warning: You appear to have cloned an empty repository. Project [prj-b-cicd-fgbs] repository [gcp-policies] was cloned to [/home/michael/tef-oldev3/_test_repo/gcp-policies].

- see older test of main that required a cb pool delete/recreate last week in https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/issues/421#issuecomment-2094404182
- see full test of bootstrap below including full CSR clone

odule.tf_source.google_project_iam_member.org_admins_source_repo_admin[0]: Creation complete after 8s [id=prj-b-cicd-fgbs/roles/source.admin/group:gcp-organization-admins3@obrienlabs.dev] google_sourcerepo_repository_iam_member.member["org"]: Creating... module.bootstrap_csr_repo.null_resource.run_command[0]: Creating... module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/dns.admin"]: Creating... module.bootstrap_csr_repo.null_resource.run_command[0]: Provisioning with 'local-exec'... module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): Executing: ["/bin/sh" "-c" "PATH=/google-cloud-sdk/bin:$PATH\n./scripts/push-to-repo.sh prj-b-cicd-fgbs tf-cloudbuilder ./Dockerfile\n"] google_sourcerepo_repository_iam_member.member["env"]: Creating... google_sourcerepo_repository_iam_member.member["net"]: Creating... google_sourcerepo_repository_iam_member.member["bootstrap"]: Creating... module.tf_cloud_builder.google_service_account.workflow_sa[0]: Creating... google_sourcerepo_repository_iam_member.member["proj"]: Creating... module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/artifactregistry.admin"]: Creating... module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/compute.networkAdmin"]: Creating... module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + '[' 3 -lt 3 ']' module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + CSR_PROJECT_ID=prj-b-cicd-fgbs module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + CSR_NAME=tf-cloudbuilder module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + DOCKERFILE_PATH=./Dockerfile module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): ++ mktemp -d module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + tmp_dir=/tmp/tmp.UPb5Ov3BbM module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + gcloud source repos clone tf-cloudbuilder /tmp/tmp.UPb5Ov3BbM --project prj-b-cicd-fgbs module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): Cloning into '/tmp/tmp.UPb5Ov3BbM'... module.tf_cloud_builder.google_service_account.workflow_sa[0]: Creation complete after 0s [id=projects/prj-b-cicd-fgbs/serviceAccounts/terraform-runner-workflow-sa@prj-b-cicd-fgbs.iam.gserviceaccount.com] module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/cloudscheduler.admin"]: Creating... module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): warning: You appear to have cloned an empty repository. module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): Project [prj-b-cicd-fgbs] repository [tf-cloudbuilder] was cloned to [/tmp/tmp.UPb5Ov3BbM]. module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + cp ./Dockerfile /tmp/tmp.UPb5Ov3BbM module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + pushd /tmp/tmp.UPb5Ov3BbM module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): /tmp/tmp.UPb5Ov3BbM ~/tef-oldev3/pbmm-on-gcp-onboarding/0-bootstrap module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + git config credential.helper gcloud.sh module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + git config init.defaultBranch main module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + git config user.email terraform-robot@example.com module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + git config user.name 'TF Robot' module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + git checkout main module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): error: pathspec 'main' did not match any file(s) known to git module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + git checkout -b main module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): Switched to a new branch 'main' module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + git add Dockerfile module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + git commit -m 'Initialize tf dockerfile repo' module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): [main (root-commit) c1c0f29] Initialize tf dockerfile repo module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): 1 file changed, 39 insertions(+) module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): create mode 100644 Dockerfile module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + git push origin main -f google_sourcerepo_repository_iam_member.member["org"]: Creation complete after 4s [id=projects/prj-b-cicd-fgbs/repos/gcp-policies/roles/viewer/serviceAccount:sa-terraform-org@prj-b-seed-cf20.iam.gserviceaccount.com] module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/workflows.admin"]: Creating... module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): remote: Waiting for private key checker: 1/1 objects left module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): To https://source.developers.google.com/p/prj-b-cicd-fgbs/r/tf-cloudbuilder module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): * [new branch] main -> main module.bootstrap_csr_repo.null_resource.run_command[0]: Creation complete after 6s [id=5317162065932165996] module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/iam.workloadIdentityPoolAdmin"]: Creating...


- ok until normal cb pool quota error on 2nd lz
- 
## triage client
- normal groups creation eventually consistent error - tf re apply
- normal auth timeout issue - tf re apply
- no quota error on cb pool per region
- car repo exists but cannot be cloned locally
- refreshed cloud shell browser session
- no need: but refreshed auth credentials
- Todo: check billing project
- Todo: check gcloud versions
- Todo: clear jwt token

# Workaround

## Historical reference on region restriction: 
- https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/issues/238
- #26
- https://github.com/GoogleCloudPlatform/esp-v2/issues/148
- check existing org policy on location restriction 
- <img width="1112" alt="Screenshot 2024-05-09 at 23 32 18" src="https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/assets/24765473/79bf136d-cee7-4f62-8577-17999ccf0e3b">
obriensystems commented 6 months ago

retest main 20240509


 505  cd ../tef-oldev3
  506  ls
  507  gcloud config set project tef-oldev3
  508  git clone https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding.git
  509  ls
  510  cd pbmm-on-gcp-onboarding/
  511  cd 0-bootstrap/
  512  cd ..
  513  ls
  514  ./terraform --version
  515  cd tef-oldev-p1gen6/
  516  ls
  517  cd ../tef-oldev
  518  ls
  519  cd terraform1310/
  520  ls
  521  ./terraform --version
  522  cp ../../tef-oldev3
  523  cp terraform ../../tef-oldev3
  524  which terraform 
  525  sudo cp terraform /usr/bin
  526  cd ../../tef-oldev3/pbmm-on-gcp-onboarding/0-bootstrap/
  527  terraform --version
  528  terraform init
  529  terraform plan -input=false -out bootstrap.tfplan
  530  terraform apply bootstrap.tfplan 
  531  gcloud services enable cloudidentity.googleapis.com
  532  terraform apply bootstrap.tfplan 
  533  terraform plan -input=false -out bootstrap.tfplan
  534  terraform apply bootstrap.tfplan 

org_id = "583675367868" # format "000000000000"

billing_account = "019283-6F1AB5-7AD576" # format "000000-000000-000000"

// For enabling the automatic groups creation, uncoment the
// variables and update the values with the group names
groups = {
  create_required_groups = true # Change to true to create the required_groups
  create_optional_groups = true # Change to true to create the optional_groups
  #####
  # check billing_project
  #####
  billing_project        = "tef-oldev3"  # Fill with bootstrap project id (the one you are starting with) to create required or optional groups
  required_groups = {
    group_org_admins           = "gcp-organization-admins3@obrienlabs.dev" # example "gcp-organization-admins@example.com"
    group_billing_admins       = "gcp-billing-admins3@obrienlabs.dev" # example "gcp-billing-admins@example.com"
    billing_data_users         = "gcp-billing-data3@obrienlabs.dev" # example "gcp-billing-data@example.com"
    audit_data_users           = "gcp-audit-data3@obrienlabs.dev" # example "gcp-audit-data@example.com"
    monitoring_workspace_users = "gcp-monitoring-workspace3@obrienlabs.dev" # example "gcp-monitoring-workspace@example.com"
  }
  optional_groups = {
     gcp_security_reviewer      = "gcp_security_reviewer3@obrienlabs.dev" #"gcp_security_reviewer_local_test@example.com"
     gcp_network_viewer         = "gcp_network_viewer3@obrienlabs.dev" #"gcp_network_viewer_local_test@example.com"
     gcp_scc_admin              = "gcp_scc_admin3@obrienlabs.dev" #"gcp_scc_admin_local_test@example.com"
     gcp_global_secrets_admin   = "gcp_global_secrets_admin3@obrienlabs.dev" #"gcp_global_secrets_admin_local_test@example.com"
     gcp_kms_admin              = "gcp_kms_admin3@obrienlabs.dev" #"gcp_kms_admin_local_test@example.com"
   }
}

default_region = "northamerica-northeast1"
#default_region = "northamerica-northeast2"

# Optional - for an organization with existing projects or for development/validation.
# Uncomment this variable to place all the example foundation resources under
# the provided folder instead of the root organization.
# The variable value is the numeric folder ID
# The folder must already exist.
parent_folder = "444651735300"

michael@cloudshell:~/tef-oldev3/pbmm-on-gcp-onboarding (tef-oldev3)$     gcloud services enable cloudresourcemanager.googleapis.com
    gcloud services enable cloudbilling.googleapis.com
    gcloud services enable iam.googleapis.com
    gcloud services enable cloudkms.googleapis.com
    gcloud services enable servicenetworking.googleapis.com
    gcloud services enable cloudbuild.googleapis.com
Operation "operations/acat.p2-757360789205-a93b7e70-1889-46b5-a4e6-462935edf569" finished successfully.
Operation "operations/acat.p2-757360789205-0c311e1f-a178-4f5f-a12e-502e0bf9d1ea" finished successfully.
Operation "operations/acat.p2-757360789205-b922a38b-ef61-4132-adfe-e77af14a7f57" finished successfully.
Operation "operations/acat.p2-757360789205-3942a70a-0953-4378-9ec9-887186657221" finished successfully.
Operation "operations/acat.p2-757360789205-9859369c-395c-4d81-8843-2a0f7a9852d8" finished successfully.
Operation "operations/acf.p2-757360789205-3f2b8c87-675c-4ca6-bf1d-27c19644a73c" finished successfully.

michael@cloudshell:~/tef-oldev3/pbmm-on-gcp-onboarding/0-bootstrap (tef-oldev3)$ terraform --version
Terraform v1.3.10
on linux_amd64

Your version of Terraform is out of date! The latest version
is 1.8.3. You can update by downloading from https://www.terraform.io/downloads.html
michael@cloudshell:~/tef-oldev3/pbmm-on-gcp-onboarding/0-bootstrap (tef-oldev3)$ terraform init
Initializing modules...
Downloading registry.terraform.io/terraform-google-modules/gcloud/google 3.4.0 for bootstrap_csr_repo...
- bootstrap_csr_repo in .terraform/modules/bootstrap_csr_repo
- bootstrap_projects_remove_editor in modules/parent-iam-remove-role
Downloading registry.terraform.io/terraform-google-modules/gcloud/google 3.4.0 for build_terraform_image...
- build_terraform_image in .terraform/modules/build_terraform_image
- cicd_project_iam_member in modules/parent-iam-member
Downloading registry.terraform.io/terraform-google-modules/cloud-storage/google 5.0.0 for gcp_projects_state_bucket...
- gcp_projects_state_bucket in .terraform/modules/gcp_projects_state_bucket/modules/simple_bucket
Downloading registry.terraform.io/terraform-google-modules/group/google 0.6.1 for optional_group...
- optional_group in .terraform/modules/optional_group
- org_iam_member in modules/parent-iam-member
- parent_iam_member in modules/parent-iam-member
Downloading registry.terraform.io/terraform-google-modules/group/google 0.6.1 for required_group...
- required_group in .terraform/modules/required_group
Downloading registry.terraform.io/terraform-google-modules/bootstrap/google 7.2.0 for seed_bootstrap...
- seed_bootstrap in .terraform/modules/seed_bootstrap
Downloading registry.terraform.io/terraform-google-modules/org-policy/google 5.3.0 for seed_bootstrap.enable_cross_project_service_account_usage...
- seed_bootstrap.enable_cross_project_service_account_usage in .terraform/modules/seed_bootstrap.enable_cross_project_service_account_usage
Downloading registry.terraform.io/terraform-google-modules/kms/google 2.3.0 for seed_bootstrap.kms...
- seed_bootstrap.kms in .terraform/modules/seed_bootstrap.kms
Downloading registry.terraform.io/terraform-google-modules/project-factory/google 14.5.0 for seed_bootstrap.seed_project...
- seed_bootstrap.seed_project in .terraform/modules/seed_bootstrap.seed_project
- seed_bootstrap.seed_project.budget in .terraform/modules/seed_bootstrap.seed_project/modules/budget
- seed_bootstrap.seed_project.essential_contacts in .terraform/modules/seed_bootstrap.seed_project/modules/essential_contacts
- seed_bootstrap.seed_project.gsuite_group in .terraform/modules/seed_bootstrap.seed_project/modules/gsuite_group
- seed_bootstrap.seed_project.project-factory in .terraform/modules/seed_bootstrap.seed_project/modules/core_project_factory
- seed_bootstrap.seed_project.project-factory.project_services in .terraform/modules/seed_bootstrap.seed_project/modules/project_services
- seed_bootstrap.seed_project.quotas in .terraform/modules/seed_bootstrap.seed_project/modules/quota_manager
- seed_bootstrap.seed_project.shared_vpc_access in .terraform/modules/seed_bootstrap.seed_project/modules/shared_vpc_access
- seed_project_iam_member in modules/parent-iam-member
Downloading registry.terraform.io/terraform-google-modules/bootstrap/google 7.2.0 for tf_cloud_builder...
- tf_cloud_builder in .terraform/modules/tf_cloud_builder/modules/tf_cloudbuild_builder
Downloading registry.terraform.io/terraform-google-modules/cloud-storage/google 5.0.0 for tf_cloud_builder.bucket...
- tf_cloud_builder.bucket in .terraform/modules/tf_cloud_builder.bucket/modules/simple_bucket
- tf_private_pool in modules/cb-private-pool
Downloading registry.terraform.io/terraform-google-modules/network/google 9.1.0 for tf_private_pool.firewall_rules...
- tf_private_pool.firewall_rules in .terraform/modules/tf_private_pool.firewall_rules/modules/firewall-rules
Downloading registry.terraform.io/terraform-google-modules/network/google 9.1.0 for tf_private_pool.peered_network...
- tf_private_pool.peered_network in .terraform/modules/tf_private_pool.peered_network
- tf_private_pool.peered_network.firewall_rules in .terraform/modules/tf_private_pool.peered_network/modules/firewall-rules
- tf_private_pool.peered_network.routes in .terraform/modules/tf_private_pool.peered_network/modules/routes
- tf_private_pool.peered_network.subnets in .terraform/modules/tf_private_pool.peered_network/modules/subnets
- tf_private_pool.peered_network.vpc in .terraform/modules/tf_private_pool.peered_network/modules/vpc
Downloading registry.terraform.io/terraform-google-modules/vpn/google 4.0.0 for tf_private_pool.vpn_ha_cb_to_onprem...
- tf_private_pool.vpn_ha_cb_to_onprem in .terraform/modules/tf_private_pool.vpn_ha_cb_to_onprem/modules/vpn_ha
Downloading registry.terraform.io/terraform-google-modules/bootstrap/google 7.2.0 for tf_source...
- tf_source in .terraform/modules/tf_source/modules/tf_cloudbuild_source
Downloading registry.terraform.io/terraform-google-modules/cloud-storage/google 5.0.0 for tf_source.cloudbuild_bucket...
- tf_source.cloudbuild_bucket in .terraform/modules/tf_source.cloudbuild_bucket/modules/simple_bucket
Downloading registry.terraform.io/terraform-google-modules/project-factory/google 14.5.0 for tf_source.cloudbuild_project...
- tf_source.cloudbuild_project in .terraform/modules/tf_source.cloudbuild_project
- tf_source.cloudbuild_project.budget in .terraform/modules/tf_source.cloudbuild_project/modules/budget
- tf_source.cloudbuild_project.essential_contacts in .terraform/modules/tf_source.cloudbuild_project/modules/essential_contacts
- tf_source.cloudbuild_project.gsuite_group in .terraform/modules/tf_source.cloudbuild_project/modules/gsuite_group
- tf_source.cloudbuild_project.project-factory in .terraform/modules/tf_source.cloudbuild_project/modules/core_project_factory
- tf_source.cloudbuild_project.project-factory.project_services in .terraform/modules/tf_source.cloudbuild_project/modules/project_services
- tf_source.cloudbuild_project.quotas in .terraform/modules/tf_source.cloudbuild_project/modules/quota_manager
- tf_source.cloudbuild_project.shared_vpc_access in .terraform/modules/tf_source.cloudbuild_project/modules/shared_vpc_access
Downloading registry.terraform.io/terraform-google-modules/bootstrap/google 7.2.0 for tf_workspace...
- tf_workspace in .terraform/modules/tf_workspace/modules/tf_cloudbuild_workspace
Downloading registry.terraform.io/terraform-google-modules/cloud-storage/google 5.0.0 for tf_workspace.artifacts_bucket...
- tf_workspace.artifacts_bucket in .terraform/modules/tf_workspace.artifacts_bucket/modules/simple_bucket
Downloading registry.terraform.io/terraform-google-modules/cloud-storage/google 5.0.0 for tf_workspace.log_bucket...
- tf_workspace.log_bucket in .terraform/modules/tf_workspace.log_bucket/modules/simple_bucket
Downloading registry.terraform.io/terraform-google-modules/cloud-storage/google 5.0.0 for tf_workspace.state_bucket...
- tf_workspace.state_bucket in .terraform/modules/tf_workspace.state_bucket/modules/simple_bucket

Initializing the backend...

Initializing provider plugins...
- Finding hashicorp/google-beta versions matching ">= 3.43.0, >= 3.50.0, >= 3.64.0, >= 3.67.0, >= 3.77.0, >= 4.11.0, >= 4.17.0, >= 4.28.0, != 4.31.0, >= 4.64.0, >= 5.7.0, < 6.0.0"...
- Finding hashicorp/external versions matching ">= 2.2.2"...
- Finding hashicorp/null versions matching ">= 2.1.0"...
- Finding hashicorp/google versions matching ">= 3.33.0, >= 3.43.0, >= 3.50.0, >= 3.53.0, >= 3.64.0, >= 3.67.0, >= 3.77.0, >= 3.83.0, >= 4.17.0, >= 4.25.0, >= 4.28.0, != 4.31.0, >= 4.46.0, >= 4.64.0, >= 5.7.0, < 6.0.0"...
- Finding hashicorp/random versions matching ">= 2.1.0, >= 2.2.0, >= 3.1.0, ~> 3.4"...
- Finding hashicorp/time versions matching ">= 0.5.0"...
- Installing hashicorp/time v0.11.1...
- Installed hashicorp/time v0.11.1 (signed by HashiCorp)
- Installing hashicorp/google-beta v5.28.0...
- Installed hashicorp/google-beta v5.28.0 (signed by HashiCorp)
- Installing hashicorp/external v2.3.3...
- Installed hashicorp/external v2.3.3 (signed by HashiCorp)
- Installing hashicorp/null v3.2.2...
- Installed hashicorp/null v3.2.2 (signed by HashiCorp)
- Installing hashicorp/google v5.28.0...
- Installed hashicorp/google v5.28.0 (signed by HashiCorp)
- Installing hashicorp/random v3.6.1...
- Installed hashicorp/random v3.6.1 (signed by HashiCorp)

Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

  # module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["workflows.googleapis.com"] will be created
  + resource "google_project_service" "project_services" {
      + disable_dependent_services = true
      + disable_on_destroy         = false
      + id                         = (known after apply)
      + project                    = (known after apply)
      + service                    = "workflows.googleapis.com"
    }

Plan: 271 to add, 0 to change, 0 to destroy.

Changes to Outputs:
  + bootstrap_step_terraform_service_account_email    = (known after apply)
  + cloud_build_peered_network_id                     = (known after apply)
  + cloud_build_private_worker_pool_id                = (known after apply)
  + cloud_build_worker_peered_ip_range                = "192.168.0.0/24"
  + cloud_build_worker_range_id                       = (known after apply)
  + cloud_builder_artifact_repo                       = (known after apply)
  + cloudbuild_project_id                             = (known after apply)
  + common_config                                     = {
      + billing_account       = "019283-6F1AB5-7AD576"
      + bootstrap_folder_name = (known after apply)
      + default_region        = "northamerica-northeast1"
      + folder_prefix         = "fldr"
      + org_id                = "583675367868"
      + parent_folder         = "444651735300"
      + parent_id             = "folders/444651735300"
      + project_prefix        = "prj"
    }
  + csr_repos                                         = {
      + gcp-bootstrap    = {
          + id      = (known after apply)
          + name    = "gcp-bootstrap"
          + project = (known after apply)
          + url     = (known after apply)
        }
      + gcp-environments = {
          + id      = (known after apply)
          + name    = "gcp-environments"
          + project = (known after apply)
          + url     = (known after apply)
        }
      + gcp-networks     = {
          + id      = (known after apply)
          + name    = "gcp-networks"
          + project = (known after apply)
          + url     = (known after apply)
        }
      + gcp-org          = {
          + id      = (known after apply)
          + name    = "gcp-org"
          + project = (known after apply)
          + url     = (known after apply)
        }
      + gcp-policies     = {
          + id      = (known after apply)
          + name    = "gcp-policies"
          + project = (known after apply)
          + url     = (known after apply)
        }
      + gcp-projects     = {
          + id      = (known after apply)
          + name    = "gcp-projects"
          + project = (known after apply)
          + url     = (known after apply)
        }
      + tf-cloudbuilder  = {
          + id      = (known after apply)
          + name    = "tf-cloudbuilder"
          + project = (known after apply)
          + url     = (known after apply)
        }
    }
  + environment_step_terraform_service_account_email  = (known after apply)
  + gcs_bucket_cloudbuild_artifacts                   = {
      + bootstrap = (known after apply)
      + env       = (known after apply)
      + net       = (known after apply)
      + org       = (known after apply)
      + proj      = (known after apply)
    }
  + gcs_bucket_cloudbuild_logs                        = {
      + bootstrap = (known after apply)
      + env       = (known after apply)
      + net       = (known after apply)
      + org       = (known after apply)
      + proj      = (known after apply)
    }
  + gcs_bucket_tfstate                                = (known after apply)
  + networks_step_terraform_service_account_email     = (known after apply)
  + optional_groups                                   = {
      + "gcp_global_secrets_admin" = "gcp_global_secrets_admin3@obrienlabs.dev"
      + "gcp_kms_admin"            = "gcp_kms_admin3@obrienlabs.dev"
      + "gcp_network_viewer"       = "gcp_network_viewer3@obrienlabs.dev"
      + "gcp_scc_admin"            = "gcp_scc_admin3@obrienlabs.dev"
      + "gcp_security_reviewer"    = "gcp_security_reviewer3@obrienlabs.dev"
    }
  + organization_step_terraform_service_account_email = (known after apply)
  + projects_gcs_bucket_tfstate                       = (known after apply)
  + projects_step_terraform_service_account_email     = (known after apply)
  + required_groups                                   = {
      + "audit_data_users"           = "gcp-audit-data3@obrienlabs.dev"
      + "billing_data_users"         = "gcp-billing-data3@obrienlabs.dev"
      + "group_billing_admins"       = "gcp-billing-admins3@obrienlabs.dev"
      + "group_org_admins"           = "gcp-organization-admins3@obrienlabs.dev"
      + "monitoring_workspace_users" = "gcp-monitoring-workspace3@obrienlabs.dev"
    }
  + seed_project_id                                   = (known after apply)

───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

Saved the plan to: bootstrap.tfplan

To perform exactly these actions, run the following command to apply:
    terraform apply "bootstrap.tfplan"

michael@cloudshell:~/tef-oldev3/pbmm-on-gcp-onboarding/0-bootstrap (tef-oldev3)$ terraform apply bootstrap.tfplan 
module.bootstrap_csr_repo.null_resource.run_destroy_command[0]: Creating...
module.bootstrap_csr_repo.null_resource.run_destroy_command[0]: Creation complete after 0s [id=8231255537637410667]
random_string.suffix: Creating...
module.seed_bootstrap.random_id.suffix: Creating...
module.tf_private_pool.random_string.suffix: Creating...
module.seed_bootstrap.module.seed_project.module.project-factory.random_id.random_project_id_suffix: Creating...
module.seed_bootstrap.random_id.suffix: Creation complete after 0s [id=i-o]
module.seed_bootstrap.module.seed_project.module.project-factory.random_id.random_project_id_suffix: Creation complete after 0s [id=zyA]
module.tf_private_pool.random_string.suffix: Creation complete after 0s [id=ymbi]
random_string.suffix: Creation complete after 0s [id=fgbs]
module.seed_bootstrap.google_organization_iam_member.org_admins_group["roles/billing.user"]: Creating...
module.required_group["group_org_admins"].google_cloud_identity_group.group: Creating...
module.seed_bootstrap.google_folder_iam_member.tmp_project_creator[0]: Creating...
module.seed_bootstrap.google_folder_iam_member.org_admin_service_account_user[0]: Creating...
module.seed_bootstrap.google_organization_iam_member.org_billing_admin: Creating...
module.seed_bootstrap.google_organization_iam_member.org_admins_group["roles/resourcemanager.organizationAdmin"]: Creating...
module.seed_bootstrap.google_organization_iam_binding.billing_creator: Creating...
module.seed_bootstrap.google_folder_iam_member.org_admin_serviceusage_consumer[0]: Creating...
google_folder.bootstrap: Creating...
module.required_group["monitoring_workspace_users"].google_cloud_identity_group.group: Creating...
module.required_group["group_billing_admins"].google_cloud_identity_group.group: Creating...
module.required_group["audit_data_users"].google_cloud_identity_group.group: Creating...
module.required_group["billing_data_users"].google_cloud_identity_group.group: Creating...
module.optional_group["gcp_network_viewer"].google_cloud_identity_group.group: Creating...
module.optional_group["gcp_kms_admin"].google_cloud_identity_group.group: Creating...
module.optional_group["gcp_global_secrets_admin"].google_cloud_identity_group.group: Creating...
module.optional_group["gcp_security_reviewer"].google_cloud_identity_group.group: Creating...
module.optional_group["gcp_scc_admin"].google_cloud_identity_group.group: Creating...
google_folder.bootstrap: Still creating... [10s elapsed]
google_folder.bootstrap: Creation complete after 12s [id=folders/236258101664]
╷
│ Error: Error creating Group: googleapi: Error 403: Cloud Identity API has not been used in project tef-oldev3 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/cloudidentity.googleapis.com/overview?project=tef-oldev3 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.
│ Details:
│ [
│   {
│     "@type": "type.googleapis.com/google.rpc.Help",
│     "links": [
│       {
│         "description": "Google developers console API activation",
│         "url": "https://console.developers.google.com/apis/api/cloudidentity.googleapis.com/overview?project=tef-oldev3"
│       }
│     ]
│   },
│   {
│     "@type": "type.googleapis.com/google.rpc.ErrorInfo",
│     "domain": "googleapis.com",
│     "metadata": {
│       "consumer": "projects/tef-oldev3",
│       "service": "cloudidentity.googleapis.com"
│     },
│     "reason": "SERVICE_DISABLED"
│   }
│ ]
│ 
│   with module.optional_group["gcp_scc_admin"].google_cloud_identity_group.group,
│   on .terraform/modules/optional_group/main.tf line 35, in resource "google_cloud_identity_group" "group":
│   35: resource "google_cloud_identity_group" "group" {
│ 
╵
╷
│ Error: Error creating Group: googleapi: Error 403: Cloud Identity API has not been used in project tef-oldev3 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/cloudidentity.googleapis.com/overview?project=tef-oldev3 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.
│ Details:
│ [
│   {
│     "@type": "type.googleapis.com/google.rpc.Help",
│     "links": [
│       {
│         "description": "Google developers console API activation",
│         "url": "https://console.developers.google.com/apis/api/cloudidentity.googleapis.com/overview?project=tef-oldev3"
│       }
│     ]
│   },
│   {
│     "@type": "type.googleapis.com/google.rpc.ErrorInfo",
│     "domain": "googleapis.com",
│     "metadata": {
│       "consumer": "projects/tef-oldev3",
│       "service": "cloudidentity.googleapis.com"
│     },
│     "reason": "SERVICE_DISABLED"
│   }
│ ]
│ 
│   with module.optional_group["gcp_network_viewer"].google_cloud_identity_group.group,
│   on .terraform/modules/optional_group/main.tf line 35, in resource "google_cloud_identity_group" "group":
│   35: resource "google_cloud_identity_group" "group" {
│ 
╵
╷
│ Error: Error creating Group: googleapi: Error 403: Cloud Identity API has not been used in project tef-oldev3 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/cloudidentity.googleapis.com/overview?project=tef-oldev3 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.
│ Details:
│ [
│   {
│     "@type": "type.googleapis.com/google.rpc.Help",
│     "links": [
│       {
│         "description": "Google developers console API activation",
│         "url": "https://console.developers.google.com/apis/api/cloudidentity.googleapis.com/overview?project=tef-oldev3"
│       }
│     ]
│   },
│   {
│     "@type": "type.googleapis.com/google.rpc.ErrorInfo",
│     "domain": "googleapis.com",
│     "metadata": {
│       "consumer": "projects/tef-oldev3",
│       "service": "cloudidentity.googleapis.com"
│     },
│     "reason": "SERVICE_DISABLED"
│   }
│ ]
│ 
│   with module.optional_group["gcp_security_reviewer"].google_cloud_identity_group.group,
│   on .terraform/modules/optional_group/main.tf line 35, in resource "google_cloud_identity_group" "group":
│   35: resource "google_cloud_identity_group" "group" {
│ 
╵
╷
│ Error: Error creating Group: googleapi: Error 403: Cloud Identity API has not been used in project tef-oldev3 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/cloudidentity.googleapis.com/overview?project=tef-oldev3 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.
│ Details:
│ [
│   {
│     "@type": "type.googleapis.com/google.rpc.Help",
│     "links": [
│       {
│         "description": "Google developers console API activation",
│         "url": "https://console.developers.google.com/apis/api/cloudidentity.googleapis.com/overview?project=tef-oldev3"
│       }
│     ]
│   },
│   {
│     "@type": "type.googleapis.com/google.rpc.ErrorInfo",
│     "domain": "googleapis.com",
│     "metadata": {
│       "consumer": "projects/tef-oldev3",
│       "service": "cloudidentity.googleapis.com"
│     },
│     "reason": "SERVICE_DISABLED"
│   }
│ ]
│ 
│   with module.optional_group["gcp_global_secrets_admin"].google_cloud_identity_group.group,
│   on .terraform/modules/optional_group/main.tf line 35, in resource "google_cloud_identity_group" "group":
│   35: resource "google_cloud_identity_group" "group" {
│ 
╵
╷
│ Error: Error creating Group: googleapi: Error 403: Cloud Identity API has not been used in project tef-oldev3 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/cloudidentity.googleapis.com/overview?project=tef-oldev3 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.
│ Details:
│ [
│   {
│     "@type": "type.googleapis.com/google.rpc.Help",
│     "links": [
│       {
│         "description": "Google developers console API activation",
│         "url": "https://console.developers.google.com/apis/api/cloudidentity.googleapis.com/overview?project=tef-oldev3"
│       }
│     ]
│   },
│   {
│     "@type": "type.googleapis.com/google.rpc.ErrorInfo",
│     "domain": "googleapis.com",
│     "metadata": {
│       "consumer": "projects/tef-oldev3",
│       "service": "cloudidentity.googleapis.com"
│     },
│     "reason": "SERVICE_DISABLED"
│   }
│ ]
│ 
│   with module.optional_group["gcp_kms_admin"].google_cloud_identity_group.group,
│   on .terraform/modules/optional_group/main.tf line 35, in resource "google_cloud_identity_group" "group":
│   35: resource "google_cloud_identity_group" "group" {
│ 
╵
╷
│ Error: Error creating Group: googleapi: Error 403: Cloud Identity API has not been used in project tef-oldev3 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/cloudidentity.googleapis.com/overview?project=tef-oldev3 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.
│ Details:
│ [
│   {
│     "@type": "type.googleapis.com/google.rpc.Help",
│     "links": [
│       {
│         "description": "Google developers console API activation",
│         "url": "https://console.developers.google.com/apis/api/cloudidentity.googleapis.com/overview?project=tef-oldev3"
│       }
│     ]
│   },
│   {
│     "@type": "type.googleapis.com/google.rpc.ErrorInfo",
│     "domain": "googleapis.com",
│     "metadata": {
│       "consumer": "projects/tef-oldev3",
│       "service": "cloudidentity.googleapis.com"
│     },
│     "reason": "SERVICE_DISABLED"
│   }
│ ]
│ 
│   with module.required_group["group_org_admins"].google_cloud_identity_group.group,
│   on .terraform/modules/required_group/main.tf line 35, in resource "google_cloud_identity_group" "group":
│   35: resource "google_cloud_identity_group" "group" {
│ 
╵
╷
│ Error: Error creating Group: googleapi: Error 403: Cloud Identity API has not been used in project tef-oldev3 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/cloudidentity.googleapis.com/overview?project=tef-oldev3 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.
│ Details:
│ [
│   {
│     "@type": "type.googleapis.com/google.rpc.Help",
│     "links": [
│       {
│         "description": "Google developers console API activation",
│         "url": "https://console.developers.google.com/apis/api/cloudidentity.googleapis.com/overview?project=tef-oldev3"
│       }
│     ]
│   },
│   {
│     "@type": "type.googleapis.com/google.rpc.ErrorInfo",
│     "domain": "googleapis.com",
│     "metadata": {
│       "consumer": "projects/tef-oldev3",
│       "service": "cloudidentity.googleapis.com"
│     },
│     "reason": "SERVICE_DISABLED"
│   }
│ ]
│ 
│   with module.required_group["billing_data_users"].google_cloud_identity_group.group,
│   on .terraform/modules/required_group/main.tf line 35, in resource "google_cloud_identity_group" "group":
│   35: resource "google_cloud_identity_group" "group" {
│ 
╵
╷
│ Error: Error creating Group: googleapi: Error 403: Cloud Identity API has not been used in project tef-oldev3 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/cloudidentity.googleapis.com/overview?project=tef-oldev3 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.
│ Details:
│ [
│   {
│     "@type": "type.googleapis.com/google.rpc.Help",
│     "links": [
│       {
│         "description": "Google developers console API activation",
│         "url": "https://console.developers.google.com/apis/api/cloudidentity.googleapis.com/overview?project=tef-oldev3"
│       }
│     ]
│   },
│   {
│     "@type": "type.googleapis.com/google.rpc.ErrorInfo",
│     "domain": "googleapis.com",
│     "metadata": {
│       "consumer": "projects/tef-oldev3",
│       "service": "cloudidentity.googleapis.com"
│     },
│     "reason": "SERVICE_DISABLED"
│   }
│ ]
│ 
│   with module.required_group["audit_data_users"].google_cloud_identity_group.group,
│   on .terraform/modules/required_group/main.tf line 35, in resource "google_cloud_identity_group" "group":
│   35: resource "google_cloud_identity_group" "group" {
│ 
╵
╷
│ Error: Error creating Group: googleapi: Error 403: Cloud Identity API has not been used in project tef-oldev3 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/cloudidentity.googleapis.com/overview?project=tef-oldev3 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.
│ Details:
│ [
│   {
│     "@type": "type.googleapis.com/google.rpc.Help",
│     "links": [
│       {
│         "description": "Google developers console API activation",
│         "url": "https://console.developers.google.com/apis/api/cloudidentity.googleapis.com/overview?project=tef-oldev3"
│       }
│     ]
│   },
│   {
│     "@type": "type.googleapis.com/google.rpc.ErrorInfo",
│     "domain": "googleapis.com",
│     "metadata": {
│       "consumer": "projects/tef-oldev3",
│       "service": "cloudidentity.googleapis.com"
│     },
│     "reason": "SERVICE_DISABLED"
│   }
│ ]
│ 
│   with module.required_group["monitoring_workspace_users"].google_cloud_identity_group.group,
│   on .terraform/modules/required_group/main.tf line 35, in resource "google_cloud_identity_group" "group":
│   35: resource "google_cloud_identity_group" "group" {
│ 
╵
╷
│ Error: Error creating Group: googleapi: Error 403: Cloud Identity API has not been used in project tef-oldev3 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/cloudidentity.googleapis.com/overview?project=tef-oldev3 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.
│ Details:
│ [
│   {
│     "@type": "type.googleapis.com/google.rpc.Help",
│     "links": [
│       {
│         "description": "Google developers console API activation",
│         "url": "https://console.developers.google.com/apis/api/cloudidentity.googleapis.com/overview?project=tef-oldev3"
│       }
│     ]
│   },
│   {
│     "@type": "type.googleapis.com/google.rpc.ErrorInfo",
│     "domain": "googleapis.com",
│     "metadata": {
│       "consumer": "projects/tef-oldev3",
│       "service": "cloudidentity.googleapis.com"
│     },
│     "reason": "SERVICE_DISABLED"
│   }
│ ]
│ 
│   with module.required_group["group_billing_admins"].google_cloud_identity_group.group,
│   on .terraform/modules/required_group/main.tf line 35, in resource "google_cloud_identity_group" "group":
│   35: resource "google_cloud_identity_group" "group" {
│ 
╵
╷
│ Error: Error applying IAM policy for folder "folders/444651735300": Error setting IAM policy for folder "folders/444651735300": googleapi: Error 400: Group gcp-organization-admins3@obrienlabs.dev does not exist., badRequest
│ 
│   with module.seed_bootstrap.google_folder_iam_member.tmp_project_creator[0],
│   on .terraform/modules/seed_bootstrap/main.tf line 47, in resource "google_folder_iam_member" "tmp_project_creator":
│   47: resource "google_folder_iam_member" "tmp_project_creator" {
│ 
╵
╷
│ Error: Error applying IAM policy for organization "583675367868": Error setting IAM policy for organization "583675367868": googleapi: Error 400: Group gcp-billing-admins3@obrienlabs.dev does not exist., badRequest
│ 
│   with module.seed_bootstrap.google_organization_iam_binding.billing_creator,
│   on .terraform/modules/seed_bootstrap/main.tf line 156, in resource "google_organization_iam_binding" "billing_creator":
│  156: resource "google_organization_iam_binding" "billing_creator" {
│ 
╵
╷
│ Error: Error applying IAM policy for organization "583675367868": Error setting IAM policy for organization "583675367868": googleapi: Error 400: Group gcp-organization-admins3@obrienlabs.dev does not exist., badRequest
│ 
│   with module.seed_bootstrap.google_organization_iam_member.org_admins_group["roles/billing.user"],
│   on .terraform/modules/seed_bootstrap/main.tf line 184, in resource "google_organization_iam_member" "org_admins_group":
│  184: resource "google_organization_iam_member" "org_admins_group" {
│ 
╵
╷
│ Error: Error applying IAM policy for organization "583675367868": Error setting IAM policy for organization "583675367868": googleapi: Error 400: Group gcp-organization-admins3@obrienlabs.dev does not exist., badRequest
│ 
│   with module.seed_bootstrap.google_organization_iam_member.org_admins_group["roles/resourcemanager.organizationAdmin"],
│   on .terraform/modules/seed_bootstrap/main.tf line 184, in resource "google_organization_iam_member" "org_admins_group":
│  184: resource "google_organization_iam_member" "org_admins_group" {
│ 
╵
╷
│ Error: Error applying IAM policy for organization "583675367868": Error setting IAM policy for organization "583675367868": googleapi: Error 400: Group gcp-billing-admins3@obrienlabs.dev does not exist., badRequest
│ 
│   with module.seed_bootstrap.google_organization_iam_member.org_billing_admin,
│   on .terraform/modules/seed_bootstrap/main.tf line 196, in resource "google_organization_iam_member" "org_billing_admin":
│  196: resource "google_organization_iam_member" "org_billing_admin" {
│ 
╵
╷
│ Error: Error applying IAM policy for folder "folders/444651735300": Error setting IAM policy for folder "folders/444651735300": googleapi: Error 400: Group gcp-organization-admins3@obrienlabs.dev does not exist., badRequest
│ 
│   with module.seed_bootstrap.google_folder_iam_member.org_admin_service_account_user[0],
│   on .terraform/modules/seed_bootstrap/main.tf line 259, in resource "google_folder_iam_member" "org_admin_service_account_user":
│  259: resource "google_folder_iam_member" "org_admin_service_account_user" {
│ 
╵
╷
│ Error: Error applying IAM policy for folder "folders/444651735300": Error setting IAM policy for folder "folders/444651735300": googleapi: Error 400: Group gcp-organization-admins3@obrienlabs.dev does not exist., badRequest
│ 
│   with module.seed_bootstrap.google_folder_iam_member.org_admin_serviceusage_consumer[0],
│   on .terraform/modules/seed_bootstrap/main.tf line 267, in resource "google_folder_iam_member" "org_admin_serviceusage_consumer":
│  267: resource "google_folder_iam_member" "org_admin_serviceusage_consumer" {
│ 
╵
obriensystems commented 6 months ago

rerun

Screenshot 2024-05-09 at 23 32 18
Plan: 265 to add, 0 to change, 0 to destroy.

Changes to Outputs:
  + bootstrap_step_terraform_service_account_email    = (known after apply)
  + cloud_build_peered_network_id                     = (known after apply)
  + cloud_build_private_worker_pool_id                = (known after apply)
  + cloud_build_worker_range_id                       = (known after apply)
  + cloud_builder_artifact_repo                       = (known after apply)
  + cloudbuild_project_id                             = "prj-b-cicd-fgbs"
  + csr_repos                                         = {
      + gcp-bootstrap    = {
          + id      = (known after apply)
          + name    = "gcp-bootstrap"
          + project = "prj-b-cicd-fgbs"
          + url     = (known after apply)
        }
      + gcp-environments = {
          + id      = (known after apply)
          + name    = "gcp-environments"
          + project = "prj-b-cicd-fgbs"
          + url     = (known after apply)
        }
      + gcp-networks     = {
          + id      = (known after apply)
          + name    = "gcp-networks"
          + project = "prj-b-cicd-fgbs"
          + url     = (known after apply)
        }
      + gcp-org          = {
          + id      = (known after apply)
          + name    = "gcp-org"
          + project = "prj-b-cicd-fgbs"
          + url     = (known after apply)
        }
      + gcp-policies     = {
          + id      = (known after apply)
          + name    = "gcp-policies"
          + project = "prj-b-cicd-fgbs"
          + url     = (known after apply)
        }
      + gcp-projects     = {
          + id      = (known after apply)
          + name    = "gcp-projects"
          + project = "prj-b-cicd-fgbs"
          + url     = (known after apply)
        }
      + tf-cloudbuilder  = {
          + id      = (known after apply)
          + name    = "tf-cloudbuilder"
          + project = "prj-b-cicd-fgbs"
          + url     = (known after apply)
        }
    }
  + environment_step_terraform_service_account_email  = (known after apply)
  + gcs_bucket_cloudbuild_artifacts                   = {
      + bootstrap = (known after apply)
      + env       = (known after apply)
      + net       = (known after apply)
      + org       = (known after apply)
      + proj      = (known after apply)
    }
  + gcs_bucket_cloudbuild_logs                        = {
      + bootstrap = (known after apply)
      + env       = (known after apply)
      + net       = (known after apply)
      + org       = (known after apply)
      + proj      = (known after apply)
    }
  + gcs_bucket_tfstate                                = "bkt-prj-b-seed-tfstate-8bea"
  + networks_step_terraform_service_account_email     = (known after apply)
  + organization_step_terraform_service_account_email = (known after apply)
  + projects_gcs_bucket_tfstate                       = "bkt-prj-b-seed-cf20-gcp-projects-tfstate"
  + projects_step_terraform_service_account_email     = (known after apply)
  + seed_project_id                                   = "prj-b-seed-cf20"

───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

Saved the plan to: bootstrap.tfplan

To perform exactly these actions, run the following command to apply:
    terraform apply "bootstrap.tfplan"

michael@cloudshell:~/tef-oldev3/pbmm-on-gcp-onboarding/0-bootstrap (tef-oldev3)$ terraform apply bootstrap.tfplan 
module.seed_bootstrap.google_folder_iam_member.tmp_project_creator[0]: Creating...
module.seed_bootstrap.google_organization_iam_member.org_billing_admin: Creating...
module.seed_bootstrap.google_organization_iam_binding.billing_creator: Creating...
module.seed_bootstrap.google_organization_iam_member.org_admins_group["roles/resourcemanager.organizationAdmin"]: Creating...
module.seed_bootstrap.google_folder_iam_member.org_admin_serviceusage_consumer[0]: Creating...
module.seed_bootstrap.google_organization_iam_member.org_admins_group["roles/billing.user"]: Creating...
module.seed_bootstrap.google_folder_iam_member.org_admin_service_account_user[0]: Creating...
module.required_group["group_billing_admins"].google_cloud_identity_group.group: Creating...
module.required_group["group_org_admins"].google_cloud_identity_group.group: Creating...
module.optional_group["gcp_network_viewer"].google_cloud_identity_group.group: Creating...
module.seed_bootstrap.google_organization_iam_binding.billing_creator: Creation complete after 5s [id=583675367868/roles/billing.creator]
module.seed_bootstrap.google_folder_iam_member.tmp_project_creator[0]: Creation complete after 5s [id=folders/444651735300/roles/resourcemanager.projectCreator/group:gcp-organization-admins3@obrienlabs.dev]
module.required_group["audit_data_users"].google_cloud_identity_group.group: Creating...
module.optional_group["gcp_security_reviewer"].google_cloud_identity_group.group: Creating...
module.optional_group["gcp_network_viewer"].google_cloud_identity_group.group: Creation complete after 9s [id=groups/019c6y180ie01wz]
module.optional_group["gcp_kms_admin"].google_cloud_identity_group.group: Creating...
module.required_group["group_org_admins"].google_cloud_identity_group.group: Creation complete after 9s [id=groups/04iylrwe3zqx2xw]
module.optional_group["gcp_global_secrets_admin"].google_cloud_identity_group.group: Creating...
module.required_group["group_billing_admins"].google_cloud_identity_group.group: Creation complete after 10s [id=groups/04anzqyu3x26sx3]
module.optional_group["gcp_scc_admin"].google_cloud_identity_group.group: Creating...
module.seed_bootstrap.google_organization_iam_member.org_billing_admin: Still creating... [10s elapsed]
module.seed_bootstrap.google_folder_iam_member.org_admin_serviceusage_consumer[0]: Still creating... [10s elapsed]
module.seed_bootstrap.google_organization_iam_member.org_admins_group["roles/resourcemanager.organizationAdmin"]: Still creating... [10s elapsed]
module.seed_bootstrap.google_folder_iam_member.org_admin_service_account_user[0]: Still creating... [10s elapsed]
module.seed_bootstrap.google_organization_iam_member.org_admins_group["roles/billing.user"]: Still creating... [10s elapsed]
module.optional_group["gcp_security_reviewer"].google_cloud_identity_group.group: Creation complete after 8s [id=groups/02250f4o4bbw7rb]
module.required_group["monitoring_workspace_users"].google_cloud_identity_group.group: Creating...
module.seed_bootstrap.google_folder_iam_member.org_admin_service_account_user[0]: Creation complete after 13s [id=folders/444651735300/roles/iam.serviceAccountUser/group:gcp-organization-admins3@obrienlabs.dev]
module.required_group["billing_data_users"].google_cloud_identity_group.group: Creating...
module.seed_bootstrap.google_folder_iam_member.org_admin_serviceusage_consumer[0]: Creation complete after 14s [id=folders/444651735300/roles/serviceusage.serviceUsageConsumer/group:gcp-organization-admins3@obrienlabs.dev]
module.seed_bootstrap.module.seed_project.module.project-factory.google_project.main: Creating...
module.required_group["audit_data_users"].google_cloud_identity_group.group: Creation complete after 9s [id=groups/03ygebqi29t4v3e]
module.optional_group["gcp_kms_admin"].google_cloud_identity_group.group: Creation complete after 8s [id=groups/04iylrwe4f9isc6]
module.optional_group["gcp_global_secrets_admin"].google_cloud_identity_group.group: Creation complete after 9s [id=groups/02fk6b3p3wqcho4]
module.optional_group["gcp_scc_admin"].google_cloud_identity_group.group: Creation complete after 8s [id=groups/01rvwp1q40e3vaf]
module.seed_bootstrap.google_organization_iam_member.org_billing_admin: Creation complete after 18s [id=583675367868/roles/billing.admin/group:gcp-billing-admins3@obrienlabs.dev]
module.seed_bootstrap.google_organization_iam_member.org_admins_group["roles/resourcemanager.organizationAdmin"]: Creation complete after 19s [id=583675367868/roles/resourcemanager.organizationAdmin/group:gcp-organization-admins3@obrienlabs.dev]
module.seed_bootstrap.google_organization_iam_member.org_admins_group["roles/billing.user"]: Creation complete after 19s [id=583675367868/roles/billing.user/group:gcp-organization-admins3@obrienlabs.dev]
module.required_group["billing_data_users"].google_cloud_identity_group.group: Creation complete after 8s [id=groups/04du1wux3vc7n2o]
module.required_group["monitoring_workspace_users"].google_cloud_identity_group.group: Creation complete after 9s [id=groups/01ci93xb2nt77bw]
module.seed_bootstrap.module.seed_project.module.project-factory.google_project.main: Still creating... [10s elapsed]

odule.seed_bootstrap.module.seed_project.module.project-factory.google_project.main: Still creating... [40s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.google_project.main: Still creating... [50s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.google_project.main: Still creating... [1m0s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.google_project.main: Still creating... [1m10s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.google_project.main: Still creating... [1m20s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.google_project.main: Still creating... [1m30s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.google_project.main: Still creating... [1m40s elapsed]

odule.seed_bootstrap.module.seed_project.module.project-factory.google_project.main: Still creating... [1m50s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.google_project.main: Still creating... [2m0s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.google_project.main: Still creating... [2m10s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.google_project.main: Still creating... [2m20s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.google_project.main: Still creating... [2m30s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.google_project.main: Still creating... [2m40s elapsed]

module.seed_bootstrap.module.seed_project.module.project-factory.google_project.main: Still creating... [2m50s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.google_project.main: Still creating... [3m0s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.google_project.main: Still creating... [3m10s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.google_project.main: Still creating... [3m20s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.google_project.main: Creation complete after 3m25s [id=projects/prj-b-seed-cf20]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["cloudbuild.googleapis.com"]: Creating...
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["serviceusage.googleapis.com"]: Creating...
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["iamcredentials.googleapis.com"]: Creating...
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["cloudkms.googleapis.com"]: Creating...
module.seed_bootstrap.module.seed_project.module.project-factory.google_resource_manager_lien.lien[0]: Creating...
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["pubsub.googleapis.com"]: Creating...
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["billingbudgets.googleapis.com"]: Creating...
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["monitoring.googleapis.com"]: Creating...
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["iam.googleapis.com"]: Creating...
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["storage-api.googleapis.com"]: Creating...
module.seed_bootstrap.module.seed_project.module.project-factory.google_resource_manager_lien.lien[0]: Creation complete after 0s [id=p764706277780-l3ed24c33-7a90-4236-a420-b776beb912ab]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["logging.googleapis.com"]: Creating...
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["billingbudgets.googleapis.com"]: Still creating... [10s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["pubsub.googleapis.com"]: Still creating... [10s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["iamcredentials.googleapis.com"]: Still creating... [10s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["cloudkms.googleapis.com"]: Still creating... [10s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["monitoring.googleapis.com"]: Still creating... [10s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["cloudbuild.googleapis.com"]: Still creating... [10s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["storage-api.googleapis.com"]: Still creating... [10s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["iam.googleapis.com"]: Still creating... [10s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["serviceusage.googleapis.com"]: Still creating... [10s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["logging.googleapis.com"]: Still creating... [10s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["iamcredentials.googleapis.com"]: Still creating... [20s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["monitoring.googleapis.com"]: Still creating... [20s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["pubsub.googleapis.com"]: Still creating... [20s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["cloudkms.googleapis.com"]: Still creating... [20s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["billingbudgets.googleapis.com"]: Still creating... [20s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["cloudbuild.googleapis.com"]: Still creating... [20s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["iam.googleapis.com"]: Still creating... [20s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["storage-api.googleapis.com"]: Still creating... [20s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["serviceusage.googleapis.com"]: Still creating... [20s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["logging.googleapis.com"]: Still creating... [20s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["storage-api.googleapis.com"]: Creation complete after 21s [id=prj-b-seed-cf20/storage-api.googleapis.com]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["iam.googleapis.com"]: Creation complete after 21s [id=prj-b-seed-cf20/iam.googleapis.com]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["pubsub.googleapis.com"]: Creation complete after 21s [id=prj-b-seed-cf20/pubsub.googleapis.com]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["billingbudgets.googleapis.com"]: Creation complete after 21s [id=prj-b-seed-cf20/billingbudgets.googleapis.com]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["monitoring.googleapis.com"]: Creation complete after 21s [id=prj-b-seed-cf20/monitoring.googleapis.com]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["serviceusage.googleapis.com"]: Creation complete after 21s [id=prj-b-seed-cf20/serviceusage.googleapis.com]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["logging.googleapis.com"]: Creation complete after 21s [id=prj-b-seed-cf20/logging.googleapis.com]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["iamcredentials.googleapis.com"]: Creation complete after 21s [id=prj-b-seed-cf20/iamcredentials.googleapis.com]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["securitycenter.googleapis.com"]: Creating...
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["cloudasset.googleapis.com"]: Creating...
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["assuredworkloads.googleapis.com"]: Creating...
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["appengine.googleapis.com"]: Creating...
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["accesscontextmanager.googleapis.com"]: Creating...
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["cloudbuild.googleapis.com"]: Creation complete after 21s [id=prj-b-seed-cf20/cloudbuild.googleapis.com]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["cloudresourcemanager.googleapis.com"]: Creating...
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["servicenetworking.googleapis.com"]: Creating...
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["cloudkms.googleapis.com"]: Creation complete after 21s [id=prj-b-seed-cf20/cloudkms.googleapis.com]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["cloudbilling.googleapis.com"]: Creating...
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["compute.googleapis.com"]: Creating...
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["admin.googleapis.com"]: Creating...
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["compute.googleapis.com"]: Creation complete after 3s [id=prj-b-seed-cf20/compute.googleapis.com]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["essentialcontacts.googleapis.com"]: Creating...
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["securitycenter.googleapis.com"]: Still creating... [10s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["cloudasset.googleapis.com"]: Still creating... [10s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["assuredworkloads.googleapis.com"]: Still creating... [10s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["appengine.googleapis.com"]: Still creating... [10s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["accesscontextmanager.googleapis.com"]: Still creating... [10s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["cloudresourcemanager.googleapis.com"]: Still creating... [10s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["servicenetworking.googleapis.com"]: Still creating... [10s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["cloudbilling.googleapis.com"]: Still creating... [10s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["admin.googleapis.com"]: Still creating... [10s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["essentialcontacts.googleapis.com"]: Still creating... [10s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["securitycenter.googleapis.com"]: Still creating... [20s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["cloudasset.googleapis.com"]: Still creating... [20s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["assuredworkloads.googleapis.com"]: Still creating... [20s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["appengine.googleapis.com"]: Still creating... [20s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["accesscontextmanager.googleapis.com"]: Still creating... [20s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["cloudresourcemanager.googleapis.com"]: Still creating... [20s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["servicenetworking.googleapis.com"]: Still creating... [20s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["cloudbilling.googleapis.com"]: Still creating... [20s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["admin.googleapis.com"]: Still creating... [20s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["cloudresourcemanager.googleapis.com"]: Creation complete after 21s [id=prj-b-seed-cf20/cloudresourcemanager.googleapis.com]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["cloudbilling.googleapis.com"]: Creation complete after 21s [id=prj-b-seed-cf20/cloudbilling.googleapis.com]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["appengine.googleapis.com"]: Creation complete after 21s [id=prj-b-seed-cf20/appengine.googleapis.com]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["accesscontextmanager.googleapis.com"]: Creation complete after 21s [id=prj-b-seed-cf20/accesscontextmanager.googleapis.com]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["cloudasset.googleapis.com"]: Creation complete after 21s [id=prj-b-seed-cf20/cloudasset.googleapis.com]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["securitycenter.googleapis.com"]: Creation complete after 21s [id=prj-b-seed-cf20/securitycenter.googleapis.com]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["bigquery.googleapis.com"]: Creating...
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["servicenetworking.googleapis.com"]: Creation complete after 21s [id=prj-b-seed-cf20/servicenetworking.googleapis.com]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["admin.googleapis.com"]: Creation complete after 21s [id=prj-b-seed-cf20/admin.googleapis.com]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["assuredworkloads.googleapis.com"]: Creation complete after 21s [id=prj-b-seed-cf20/assuredworkloads.googleapis.com]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["essentialcontacts.googleapis.com"]: Still creating... [20s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["essentialcontacts.googleapis.com"]: Creation complete after 22s [id=prj-b-seed-cf20/essentialcontacts.googleapis.com]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["bigquery.googleapis.com"]: Still creating... [10s elapsed]

 Still creating... [20s elapsed]
module.seed_bootstrap.module.seed_project.module.project-factory.module.project_services.google_project_service.project_services["bigquery.googleapis.com"]: Creation complete after 21s [id=prj-b-seed-cf20/bigquery.googleapis.com]
module.seed_bootstrap.data.google_storage_project_service_account.gcs_account: Reading...
module.seed_bootstrap.module.seed_project.module.project-factory.google_project_default_service_accounts.default_service_accounts[0]: Creating...
google_service_account.terraform-env-sa["bootstrap"]: Creating...
google_service_account.terraform-env-sa["env"]: Creating...
google_service_account.terraform-env-sa["net"]: Creating...
module.seed_bootstrap.module.kms[0].google_kms_key_ring.key_ring: Creating...
module.seed_bootstrap.module.enable_cross_project_service_account_usage.google_project_organization_policy.project_policy_boolean[0]: Creating...
google_service_account.terraform-env-sa["org"]: Creating...
google_service_account.terraform-env-sa["proj"]: Creating...
module.seed_bootstrap.module.seed_project.module.project-factory.google_project_default_service_accounts.default_service_accounts[0]: Creation complete after 0s [id=projects/prj-b-seed-cf20]
module.seed_bootstrap.module.kms[0].google_kms_key_ring.key_ring: Creation complete after 0s [id=projects/prj-b-seed-cf20/locations/northamerica-northeast1/keyRings/prj-keyring]
module.seed_bootstrap.module.kms[0].google_kms_crypto_key.key[0]: Creating...
module.seed_bootstrap.data.google_storage_project_service_account.gcs_account: Read complete after 0s [id=service-764706277780@gs-project-accounts.iam.gserviceaccount.com]
google_service_account.terraform-env-sa["proj"]: Creation complete after 0s [id=projects/prj-b-seed-cf20/serviceAccounts/sa-terraform-proj@prj-b-seed-cf20.iam.gserviceaccount.com]
module.seed_bootstrap.module.kms[0].google_kms_crypto_key.key[0]: Creation complete after 0s [id=projects/prj-b-seed-cf20/locations/northamerica-northeast1/keyRings/prj-keyring/cryptoKeys/prj-key]
google_service_account.terraform-env-sa["env"]: Creation complete after 0s [id=projects/prj-b-seed-cf20/serviceAccounts/sa-terraform-env@prj-b-seed-cf20.iam.gserviceaccount.com]
module.seed_bootstrap.module.enable_cross_project_service_account_usage.google_project_organization_policy.project_policy_boolean[0]: Creation complete after 0s [id=prj-b-seed-cf20:constraints/iam.disableCrossProjectServiceAccountUsage]
module.seed_bootstrap.module.kms[0].google_kms_crypto_key_iam_binding.decrypters[0]: Creating...
module.seed_bootstrap.module.kms[0].google_kms_crypto_key_iam_binding.encrypters[0]: Creating...
google_service_account.terraform-env-sa["net"]: Creation complete after 1s [id=projects/prj-b-seed-cf20/serviceAccounts/sa-terraform-net@prj-b-seed-cf20.iam.gserviceaccount.com]
google_service_account.terraform-env-sa["bootstrap"]: Creation complete after 1s [id=projects/prj-b-seed-cf20/serviceAccounts/sa-terraform-bootstrap@prj-b-seed-cf20.iam.gserviceaccount.com]
google_service_account.terraform-env-sa["org"]: Creation complete after 1s [id=projects/prj-b-seed-cf20/serviceAccounts/sa-terraform-org@prj-b-seed-cf20.iam.gserviceaccount.com]
google_billing_account_iam_member.tf_billing_user["env"]: Creating...
google_billing_account_iam_member.tf_billing_user["proj"]: Creating...
google_billing_account_iam_member.tf_billing_user["bootstrap"]: Creating...
google_billing_account_iam_member.billing_account_sink: Creating...
google_billing_account_iam_member.tf_billing_user["org"]: Creating...
google_billing_account_iam_member.tf_billing_user["net"]: Creating...
module.seed_project_iam_member["proj"].google_project_iam_member.project_parent_iam["roles/storage.objectAdmin"]: Creating...
module.seed_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/cloudkms.admin"]: Creating...
google_billing_account_iam_member.tf_billing_user["proj"]: Creation complete after 4s [id=019283-6F1AB5-7AD576/roles/billing.user/serviceAccount:sa-terraform-proj@prj-b-seed-cf20.iam.gserviceaccount.com]
module.seed_project_iam_member["env"].google_project_iam_member.project_parent_iam["roles/storage.objectAdmin"]: Creating...

module.seed_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/storage.admin"]: Creating...
module.seed_project_iam_member["proj"].google_project_iam_member.project_parent_iam["roles/storage.objectAdmin"]: Creation complete after 8s [id=prj-b-seed-cf20/roles/storage.objectAdmin/serviceAccount:sa-terraform-proj@prj-b-seed-cf20.iam.gserviceaccount.com]
module.seed_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/iam.serviceAccountAdmin"]: Creating...
module.seed_bootstrap.module.kms[0].google_kms_crypto_key_iam_binding.encrypters[0]: Creation complete after 8s [id=projects/prj-b-seed-cf20/locations/northamerica-northeast1/keyRings/prj-keyring/cryptoKeys/prj-key/roles/cloudkms.cryptoKeyEncrypter]
module.seed_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/resourcemanager.projectDeleter"]: Creating...
module.seed_bootstrap.module.kms[0].google_kms_crypto_key_iam_binding.decrypters[0]: Creation complete after 9s [id=projects/prj-b-seed-cf20/locations/northamerica-northeast1/keyRings/prj-keyring/cryptoKeys/prj-key/roles/cloudkms.cryptoKeyDecrypter]
module.seed_project_iam_member["org"].google_project_iam_member.project_parent_iam["roles/storage.objectAdmin"]: Creating...
google_billing_account_iam_member.tf_billing_user["env"]: Still creating... [10s elapsed]
google_billing_account_iam_member.tf_billing_user["bootstrap"]: Still creating... [10s elapsed]
google_billing_account_iam_member.billing_account_sink: Still creating... [10s elapsed]
google_billing_account_iam_member.tf_billing_user["org"]: Still creating... [10s elapsed]
google_billing_account_iam_member.tf_billing_user["net"]: Still creating... [10s elapsed]
module.seed_project_iam_member["env"].google_project_iam_member.project_parent_iam["roles/storage.objectAdmin"]: Still creating... [10s elapsed]
module.seed_project_iam_member["env"].google_project_iam_member.project_parent_iam["roles/storage.objectAdmin"]: Creation complete after 12s [id=prj-b-seed-cf20/roles/storage.objectAdmin/serviceAccount:sa-terraform-env@prj-b-seed-cf20.iam.gserviceaccount.com]
module.seed_project_iam_member["net"].google_project_iam_member.project_parent_iam["roles/storage.objectAdmin"]: Creating...

f20/roles/storage.objectAdmin/serviceAccount:sa-terraform-org@prj-b-seed-cf20.iam.gserviceaccount.com]
module.parent_iam_member["org"].google_folder_iam_member.folder_parent_iam["roles/resourcemanager.folderAdmin"]: Creating...
module.seed_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/resourcemanager.projectDeleter"]: Creation complete after 9s [id=prj-b-seed-cf20/roles/resourcemanager.projectDeleter/serviceAccount:sa-terraform-bootstrap@prj-b-seed-cf20.iam.gserviceaccount.com]
module.parent_iam_member["net"].google_folder_iam_member.folder_parent_iam["roles/dns.admin"]: Creating...
module.seed_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/iam.serviceAccountAdmin"]: Creation complete after 9s [id=prj-b-seed-cf20/roles/iam.serviceAccountAdmin/serviceAccount:sa-terraform-bootstrap@prj-b-seed-cf20.iam.gserviceaccount.com]
module.parent_iam_member["net"].google_folder_iam_member.folder_parent_iam["roles/compute.networkAdmin"]: Creating...
module.seed_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/storage.admin"]: Creation complete after 9s [id=prj-b-seed-cf20/roles/storage.admin/serviceAccount:sa-terraform-bootstrap@prj-b-seed-cf20.iam.gserviceaccount.com]
module.parent_iam_member["proj"].google_folder_iam_member.folder_parent_iam["roles/compute.xpnAdmin"]: Creating...
google_billing_account_iam_member.tf_billing_user["env"]: Still creating... [20s elapsed]
google_billing_account_iam_member.billing_account_sink: Still creating... [20s elapsed]
google_billing_account_iam_member.tf_billing_user["bootstrap"]: Still creating... [20s elapsed]
google_billing_account_iam_member.tf_billing_user["org"]: Still creating... [20s elapsed]
google_billing_account_iam_member.tf_billing_user["net"]: Still creating... [20s elapsed]
module.parent_iam_member["org"].google_folder_iam_member.folder_parent_iam["roles/resourcemanager.folderAdmin"]: Creation complete after 4s [id=folders/444651735300/roles/resourcemanager.folderAdmin/serviceAccount:sa-terraform-org@prj-b-seed-cf20.iam.gserviceaccount.com]
module.parent_iam_member["net"].google_folder_iam_member.folder_parent_iam["roles/compute.orgSecurityPolicyAdmin"]: Creating...
google_billing_account_iam_member.tf_billing_user["bootstrap"]: Creation complete after 24s [id=019283-6F1AB5-7AD576/roles/billing.user/serviceAccount:sa-terraform-bootstrap@prj-b-seed-cf20.iam.gserviceaccount.com]
module.seed_project_iam_member["net"].google_project_iam_member.project_parent_iam["roles/storage.objectAdmin"]: Creation complete after 8s [id=prj-b-seed-cf20/roles/storage.objectAdmin/serviceAccount:sa-terraform-net@prj-b-seed-cf20.iam.gserviceaccount.com]
module.parent_iam_member["proj"].google_folder_iam_member.folder_parent_iam["roles/artifactregistry.admin"]: Creating...
module.parent_iam_member["net"].google_folder_iam_member.folder_parent_iam["roles/resourcemanager.folderViewer"]: Creating...
google_billing_account_iam_member.tf_billing_user["env"]: Creation complete after 24s [id=019283-6F1AB5-7AD576/roles/billing.user/serviceAccount:sa-terraform-env@prj-b-seed-cf20.iam.gserviceaccount.com]
module.parent_iam_member["proj"].google_folder_iam_member.folder_parent_iam["roles/compute.networkAdmin"]: Creating...
google_billing_account_iam_member.billing_account_sink: Creation complete after 24s [id=019283-6F1AB5-7AD576/roles/logging.configWriter/serviceAccount:sa-terraform-org@prj-b-seed-cf20.iam.gserviceaccount.com]
module.parent_iam_member["net"].google_folder_iam_member.folder_parent_iam["roles/compute.orgSecurityResourceAdmin"]: Creating...
google_billing_account_iam_member.tf_billing_user["org"]: Creation complete after 24s [id=019283-6F1AB5-7AD576/roles/billing.user/serviceAccount:sa-terraform-org@prj-b-seed-cf20.iam.gserviceaccount.com]
module.parent_iam_member["net"].google_folder_iam_member.folder_parent_iam["roles/compute.securityAdmin"]: Creating...
google_billing_account_iam_member.tf_billing_user["net"]: Creation complete after 24s [id=019283-6F1AB5-7AD576/roles/billing.user/serviceAccount:sa-terraform-net@prj-b-seed-cf20.iam.gserviceaccount.com]
module.parent_iam_member["bootstrap"].google_folder_iam_member.folder_parent_iam["roles/resourcemanager.folderAdmin"]: Creating...
module.parent_iam_member["net"].google_folder_iam_member.folder_parent_iam["roles/dns.admin"]: Still creating... [10s elapsed]
module.parent_iam_member["net"].google_folder_iam_member.folder_parent_iam["roles/compute.networkAdmin"]: Still creating... [10s elapsed]
module.parent_iam_member["proj"].google_folder_iam_member.folder_parent_iam["roles/compute.xpnAdmin"]: Still creating... [10s elapsed]

module.parent_iam_member["proj"].google_folder_iam_member.folder_parent_iam["roles/artifactregistry.admin"]: Still creating... [10s elapsed]
module.parent_iam_member["net"].google_folder_iam_member.folder_parent_iam["roles/resourcemanager.folderViewer"]: Still creating... [10s elapsed]
module.parent_iam_member["proj"].google_folder_iam_member.folder_parent_iam["roles/compute.networkAdmin"]: Still creating... [10s elapsed]
module.parent_iam_member["net"].google_folder_iam_member.folder_parent_iam["roles/compute.orgSecurityResourceAdmin"]: Still creating... [10s elapsed]
module.parent_iam_member["net"].google_folder_iam_member.folder_parent_iam["roles/compute.securityAdmin"]: Still creating... [10s elapsed]
module.parent_iam_member["bootstrap"].google_folder_iam_member.folder_parent_iam["roles/resourcemanager.folderAdmin"]: Still creating... [10s elapsed]
module.parent_iam_member["net"].google_folder_iam_member.folder_parent_iam["roles/dns.admin"]: Still creating... [20s elapsed]
module.parent_iam_member["net"].google_folder_iam_member.folder_parent_iam["roles/compute.networkAdmin"]: Still creating... [20s elapsed]
module.parent_iam_member["proj"].google_folder_iam_member.folder_parent_iam["roles/compute.xpnAdmin"]: Still creating... [20s elapsed]
module.parent_iam_member["net"].google_folder_iam_member.folder_parent_iam["roles/compute.orgSecurityPolicyAdmin"]: Still creating... [20s elapsed]
module.parent_iam_member["net"].google_folder_iam_member.folder_parent_iam["roles/resourcemanager.folderViewer"]: Still creating... [20s elapsed]
module.parent_iam_member["proj"].google_folder_iam_member.folder_parent_iam["roles/artifactregistry.admin"]: Still creating... [20s elapsed]
module.parent_iam_member["proj"].google_folder_iam_member.folder_parent_iam["roles/compute.networkAdmin"]: Still creating... [20s elapsed]
module.parent_iam_member["net"].google_folder_iam_member.folder_parent_iam["roles/compute.orgSecurityResourceAdmin"]: Still creating... [20s elapsed]
module.parent_iam_member["net"].google_folder_iam_member.folder_parent_iam["roles/compute.securityAdmin"]: Still creating... [20s elapsed]
module.parent_iam_member["bootstrap"].google_folder_iam_member.folder_parent_iam["roles/resourcemanager.folderAdmin"]: Still creating... [20s elapsed]
module.parent_iam_member["net"].google_folder_iam_member.folder_parent_iam["roles/dns.admin"]: Still creating... [30s elapsed]
module.parent_iam_member["net"].google_folder_iam_member.folder_parent_iam["roles/compute.networkAdmin"]: Still creating... [30s elapsed]
module.parent_iam_member["proj"].google_folder_iam_member.folder_parent_iam["roles/compute.xpnAdmin"]: Still creating... [30s elapsed]
module.parent_iam_member["net"].google_folder_iam_member.folder_parent_iam["roles/compute.orgSecurityPolicyAdmin"]: Still creating... [30s elapsed]
module.parent_iam_member["proj"].google_folder_iam_member.folder_parent_iam["roles/artifactregistry.admin"]: Still creating... [30s elapsed]
module.parent_iam_member["net"].google_folder_iam_member.folder_parent_iam["roles/resourcemanager.folderViewer"]: Still creating... [30s elapsed]
module.parent_iam_member["proj"].google_folder_iam_member.folder_parent_iam["roles/compute.networkAdmin"]: Still creating... [30s elapsed]
module.parent_iam_member["net"].google_folder_iam_member.folder_parent_iam["roles/compute.orgSecurityResourceAdmin"]: Still creating... [30s elapsed]
module.parent_iam_member["net"].google_folder_iam_member.folder_parent_iam["roles/compute.securityAdmin"]: Still creating... [30s elapsed]
module.parent_iam_member["bootstrap"].google_folder_iam_member.folder_parent_iam["roles/resourcemanager.folderAdmin"]: Still creating... [30s elapsed]
module.parent_iam_member["net"].google_folder_iam_member.folder_parent_iam["roles/dns.admin"]: Still creating... [40s elapsed]
module.parent_iam_member["net"].google_folder_iam_member.folder_parent_iam["roles/compute.networkAdmin"]: Still creating... [40s elapsed]
module.parent_iam_member["proj"].google_folder_iam_member.folder_parent_iam["roles/compute.xpnAdmin"]: Still creating... [40s elapsed]

module.parent_iam_member["proj"].google_folder_iam_member.folder_parent_iam["roles/artifactregistry.admin"]: Still creating... [40s elapsed]
module.parent_iam_member["net"].google_folder_iam_member.folder_parent_iam["roles/resourcemanager.folderViewer"]: Still creating... [40s elapsed]
module.parent_iam_member["proj"].google_folder_iam_member.folder_parent_iam["roles/compute.networkAdmin"]: Still creating... [40s elapsed]
module.parent_iam_member["net"].google_folder_iam_member.folder_parent_iam["roles/compute.orgSecurityResourceAdmin"]: Still creating... [40s elapsed]
module.parent_iam_member["net"].google_folder_iam_member.folder_parent_iam["roles/compute.securityAdmin"]: Still creating... [40s elapsed]
module.parent_iam_member["bootstrap"].google_folder_iam_member.folder_parent_iam["roles/resourcemanager.folderAdmin"]: Still creating... [40s elapsed]
module.parent_iam_member["net"].google_folder_iam_member.folder_parent_iam["roles/dns.admin"]: Creation complete after 49s [id=folders/444651735300/roles/dns.admin/serviceAccount:sa-terraform-net@prj-b-seed-cf20.iam.gserviceaccount.com]
module.parent_iam_member["proj"].google_folder_iam_member.folder_parent_iam["roles/resourcemanager.folderAdmin"]: Creating...
module.parent_iam_member["net"].google_folder_iam_member.folder_parent_iam["roles/compute.networkAdmin"]: Creation complete after 49s [id=folders/444651735300/roles/compute.networkAdmin/serviceAccount:sa-terraform-net@prj-b-seed-cf20.iam.gserviceaccount.com]
module.parent_iam_member["env"].google_folder_iam_member.folder_parent_iam["roles/resourcemanager.folderAdmin"]: Creating...
module.parent_iam_member["proj"].google_folder_iam_member.folder_parent_iam["roles/compute.xpnAdmin"]: Creation complete after 50s [id=folders/444651735300/roles/compute.xpnAdmin/serviceAccount:sa-terraform-proj@prj-b-seed-cf20.iam.gserviceaccount.com]
module.org_iam_member["bootstrap"].google_organization_iam_member.org_parent_iam["roles/accesscontextmanager.policyAdmin"]: Creating...
module.parent_iam_member["net"].google_folder_iam_member.folder_parent_iam["roles/compute.orgSecurityPolicyAdmin"]: Creation complete after 46s [id=folders/444651735300/roles/compute.orgSecurityPolicyAdmin/serviceAccount:sa-terraform-net@prj-b-seed-cf20.iam.gserviceaccount.com]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.tagAdmin"]: Creating...
module.parent_iam_member["proj"].google_folder_iam_member.folder_parent_iam["roles/artifactregistry.admin"]: Creation complete after 43s [id=folders/444651735300/roles/artifactregistry.admin/serviceAccount:sa-terraform-proj@prj-b-seed-cf20.iam.gserviceaccount.com]
module.org_iam_member["proj"].google_organization_iam_member.org_parent_iam["roles/serviceusage.serviceUsageConsumer"]: Creating...
module.parent_iam_member["net"].google_folder_iam_member.folder_parent_iam["roles/resourcemanager.folderViewer"]: Creation complete after 43s [id=folders/444651735300/roles/resourcemanager.folderViewer/serviceAccount:sa-terraform-net@prj-b-seed-cf20.iam.gserviceaccount.com]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/cloudasset.owner"]: Creating...
module.parent_iam_member["proj"].google_folder_iam_member.folder_parent_iam["roles/compute.networkAdmin"]: Creation complete after 43s [id=folders/444651735300/roles/compute.networkAdmin/serviceAccount:sa-terraform-proj@prj-b-seed-cf20.iam.gserviceaccount.com]
module.org_iam_member["bootstrap"].google_organization_iam_member.org_parent_iam["roles/browser"]: Creating...
module.parent_iam_member["net"].google_folder_iam_member.folder_parent_iam["roles/compute.orgSecurityResourceAdmin"]: Creation complete after 44s [id=folders/444651735300/roles/compute.orgSecurityResourceAdmin/serviceAccount:sa-terraform-net@prj-b-seed-cf20.iam.gserviceaccount.com]
module.org_iam_member["env"].google_organization_iam_member.org_parent_iam["roles/assuredworkloads.admin"]: Creating...
module.parent_iam_member["net"].google_folder_iam_member.folder_parent_iam["roles/compute.securityAdmin"]: Creation complete after 44s [id=folders/444651735300/roles/compute.securityAdmin/serviceAccount:sa-terraform-net@prj-b-seed-cf20.iam.gserviceaccount.com]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/logging.configWriter"]: Creating...
module.parent_iam_member["bootstrap"].google_folder_iam_member.folder_parent_iam["roles/resourcemanager.folderAdmin"]: Creation complete after 44s [id=folders/444651735300/roles/resourcemanager.folderAdmin/serviceAccount:sa-terraform-bootstrap@prj-b-seed-cf20.iam.gserviceaccount.com]
module.org_iam_member["bootstrap"].google_organization_iam_member.org_parent_iam["roles/serviceusage.serviceUsageConsumer"]: Creating...
module.org_iam_member["bootstrap"].google_organization_iam_member.org_parent_iam["roles/accesscontextmanager.policyAdmin"]: Creation complete after 4s [id=583675367868/roles/accesscontextmanager.policyAdmin/serviceAccount:sa-terraform-bootstrap@prj-b-seed-cf20.iam.gserviceaccount.com]
module.org_iam_member["env"].google_organization_iam_member.org_parent_iam["roles/browser"]: Creating...

module.parent_iam_member["env"].google_folder_iam_member.folder_parent_iam["roles/resourcemanager.folderAdmin"]: Still creating... [10s elapsed]
module.parent_iam_member["env"].google_folder_iam_member.folder_parent_iam["roles/resourcemanager.folderAdmin"]: Creation complete after 11s [id=folders/444651735300/roles/resourcemanager.folderAdmin/serviceAccount:sa-terraform-env@prj-b-seed-cf20.iam.gserviceaccount.com]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/orgpolicy.policyAdmin"]: Creating...
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.tagAdmin"]: Still creating... [10s elapsed]
module.parent_iam_member["proj"].google_folder_iam_member.folder_parent_iam["roles/resourcemanager.folderAdmin"]: Creation complete after 11s [id=folders/444651735300/roles/resourcemanager.folderAdmin/serviceAccount:sa-terraform-proj@prj-b-seed-cf20.iam.gserviceaccount.com]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.tagUser"]: Creating...
module.org_iam_member["proj"].google_organization_iam_member.org_parent_iam["roles/serviceusage.serviceUsageConsumer"]: Still creating... [10s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/cloudasset.owner"]: Still creating... [10s elapsed]
module.org_iam_member["bootstrap"].google_organization_iam_member.org_parent_iam["roles/browser"]: Still creating... [10s elapsed]
module.org_iam_member["env"].google_organization_iam_member.org_parent_iam["roles/assuredworkloads.admin"]: Still creating... [10s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/logging.configWriter"]: Still creating... [10s elapsed]
module.org_iam_member["bootstrap"].google_organization_iam_member.org_parent_iam["roles/serviceusage.serviceUsageConsumer"]: Still creating... [10s elapsed]
module.org_iam_member["env"].google_organization_iam_member.org_parent_iam["roles/browser"]: Still creating... [10s elapsed]

module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/orgpolicy.policyAdmin"]: Still creating... [10s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.tagAdmin"]: Still creating... [20s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.tagUser"]: Still creating... [10s elapsed]
module.org_iam_member["proj"].google_organization_iam_member.org_parent_iam["roles/serviceusage.serviceUsageConsumer"]: Still creating... [20s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/cloudasset.owner"]: Still creating... [20s elapsed]
module.org_iam_member["bootstrap"].google_organization_iam_member.org_parent_iam["roles/browser"]: Still creating... [20s elapsed]
module.org_iam_member["env"].google_organization_iam_member.org_parent_iam["roles/assuredworkloads.admin"]: Still creating... [20s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/logging.configWriter"]: Still creating... [20s elapsed]
module.org_iam_member["bootstrap"].google_organization_iam_member.org_parent_iam["roles/serviceusage.serviceUsageConsumer"]: Still creating... [20s elapsed]
module.org_iam_member["env"].google_organization_iam_member.org_parent_iam["roles/browser"]: Still creating... [20s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/orgpolicy.policyAdmin"]: Still creating... [20s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.tagAdmin"]: Still creating... [30s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.tagUser"]: Still creating... [20s elapsed]
module.org_iam_member["proj"].google_organization_iam_member.org_parent_iam["roles/serviceusage.serviceUsageConsumer"]: Still creating... [30s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/cloudasset.owner"]: Still creating... [30s elapsed]
module.org_iam_member["bootstrap"].google_organization_iam_member.org_parent_iam["roles/browser"]: Still creating... [30s elapsed]
module.org_iam_member["env"].google_organization_iam_member.org_parent_iam["roles/assuredworkloads.admin"]: Still creating... [30s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/logging.configWriter"]: Still creating... [30s elapsed]
module.org_iam_member["bootstrap"].google_organization_iam_member.org_parent_iam["roles/serviceusage.serviceUsageConsumer"]: Still creating... [30s elapsed]
module.org_iam_member["env"].google_organization_iam_member.org_parent_iam["roles/browser"]: Still creating... [30s elapsed]

module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.tagAdmin"]: Still creating... [40s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.tagUser"]: Still creating... [30s elapsed]
module.org_iam_member["proj"].google_organization_iam_member.org_parent_iam["roles/serviceusage.serviceUsageConsumer"]: Still creating... [40s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/cloudasset.owner"]: Still creating... [40s elapsed]
module.org_iam_member["bootstrap"].google_organization_iam_member.org_parent_iam["roles/browser"]: Still creating... [40s elapsed]
module.org_iam_member["env"].google_organization_iam_member.org_parent_iam["roles/assuredworkloads.admin"]: Still creating... [40s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.tagAdmin"]: Creation complete after 41s [id=583675367868/roles/resourcemanager.tagAdmin/serviceAccount:sa-terraform-org@prj-b-seed-cf20.iam.gserviceaccount.com]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/essentialcontacts.admin"]: Creating...
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/logging.configWriter"]: Still creating... [40s elapsed]
module.org_iam_member["bootstrap"].google_organization_iam_member.org_parent_iam["roles/serviceusage.serviceUsageConsumer"]: Still creating... [40s elapsed]
module.org_iam_member["env"].google_organization_iam_member.org_parent_iam["roles/browser"]: Still creating... [40s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/orgpolicy.policyAdmin"]: Still creating... [40s elapsed]
module.org_iam_member["proj"].google_organization_iam_member.org_parent_iam["roles/serviceusage.serviceUsageConsumer"]: Creation complete after 50s [id=583675367868/roles/serviceusage.serviceUsageConsumer/serviceAccount:sa-terraform-proj@prj-b-seed-cf20.iam.gserviceaccount.com]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.organizationViewer"]: Creating...
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.tagUser"]: Still creating... [40s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/cloudasset.owner"]: Creation complete after 50s [id=583675367868/roles/cloudasset.owner/serviceAccount:sa-terraform-org@prj-b-seed-cf20.iam.gserviceaccount.com]
module.org_iam_member["proj"].google_organization_iam_member.org_parent_iam["roles/accesscontextmanager.policyAdmin"]: Creating...
module.org_iam_member["bootstrap"].google_organization_iam_member.org_parent_iam["roles/browser"]: Creation complete after 50s [id=583675367868/roles/browser/serviceAccount:sa-terraform-bootstrap@prj-b-seed-cf20.iam.gserviceaccount.com]
module.org_iam_member["bootstrap"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.organizationAdmin"]: Creating...
module.org_iam_member["env"].google_organization_iam_member.org_parent_iam["roles/assuredworkloads.admin"]: Creation complete after 50s [id=583675367868/roles/assuredworkloads.admin/serviceAccount:sa-terraform-env@prj-b-seed-cf20.iam.gserviceaccount.com]
module.org_iam_member["net"].google_organization_iam_member.org_parent_iam["roles/accesscontextmanager.policyAdmin"]: Creating...
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/logging.configWriter"]: Creation complete after 50s [id=583675367868/roles/logging.configWriter/serviceAccount:sa-terraform-org@prj-b-seed-cf20.iam.gserviceaccount.com]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/essentialcontacts.admin"]: Still creating... [10s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/securitycenter.sourcesEditor"]: Creating...
module.org_iam_member["bootstrap"].google_organization_iam_member.org_parent_iam["roles/serviceusage.serviceUsageConsumer"]: Creation complete after 50s [id=583675367868/roles/serviceusage.serviceUsageConsumer/serviceAccount:sa-terraform-bootstrap@prj-b-seed-cf20.iam.gserviceaccount.com]
module.org_iam_member["net"].google_organization_iam_member.org_parent_iam["roles/browser"]: Creating...
module.org_iam_member["env"].google_organization_iam_member.org_parent_iam["roles/browser"]: Creation complete after 47s [id=583675367868/roles/browser/serviceAccount:sa-terraform-env@prj-b-seed-cf20.iam.gserviceaccount.com]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.organizationAdmin"]: Creating...
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/orgpolicy.policyAdmin"]: Creation complete after 45s [id=583675367868/roles/orgpolicy.policyAdmin/serviceAccount:sa-terraform-org@prj-b-seed-cf20.iam.gserviceaccount.com]
module.org_iam_member["proj"].google_organization_iam_member.org_parent_iam["roles/browser"]: Creating...
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.tagUser"]: Creation complete after 46s [id=583675367868/roles/resourcemanager.tagUser/serviceAccount:sa-terraform-org@prj-b-seed-cf20.iam.gserviceaccount.com]
module.org_iam_member["proj"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.organizationAdmin"]: Creating...

module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.organizationViewer"]: Still creating... [10s elapsed]
module.org_iam_member["proj"].google_organization_iam_member.org_parent_iam["roles/accesscontextmanager.policyAdmin"]: Still creating... [10s elapsed]
module.org_iam_member["bootstrap"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.organizationAdmin"]: Still creating... [10s elapsed]
module.org_iam_member["net"].google_organization_iam_member.org_parent_iam["roles/accesscontextmanager.policyAdmin"]: Still creating... [10s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/essentialcontacts.admin"]: Still creating... [20s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/securitycenter.sourcesEditor"]: Still creating... [10s elapsed]
module.org_iam_member["net"].google_organization_iam_member.org_parent_iam["roles/browser"]: Still creating... [10s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.organizationAdmin"]: Still creating... [10s elapsed]
module.org_iam_member["proj"].google_organization_iam_member.org_parent_iam["roles/browser"]: Still creating... [10s elapsed]
module.org_iam_member["proj"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.organizationAdmin"]: Still creating... [10s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.organizationViewer"]: Still creating... [20s elapsed]
module.org_iam_member["proj"].google_organization_iam_member.org_parent_iam["roles/accesscontextmanager.policyAdmin"]: Still creating... [20s elapsed]
module.org_iam_member["bootstrap"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.organizationAdmin"]: Still creating... [20s elapsed]
module.org_iam_member["net"].google_organization_iam_member.org_parent_iam["roles/accesscontextmanager.policyAdmin"]: Still creating... [20s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/essentialcontacts.admin"]: Still creating... [30s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/securitycenter.sourcesEditor"]: Still creating... [20s elapsed]
module.org_iam_member["net"].google_organization_iam_member.org_parent_iam["roles/browser"]: Still creating... [20s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.organizationAdmin"]: Still creating... [20s elapsed]
module.org_iam_member["proj"].google_organization_iam_member.org_parent_iam["roles/browser"]: Still creating... [20s elapsed]
module.org_iam_member["proj"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.organizationAdmin"]: Still creating... [20s elapsed]

module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.organizationViewer"]: Still creating... [10s elapsed]
module.org_iam_member["proj"].google_organization_iam_member.org_parent_iam["roles/accesscontextmanager.policyAdmin"]: Still creating... [10s elapsed]
module.org_iam_member["bootstrap"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.organizationAdmin"]: Still creating... [10s elapsed]
module.org_iam_member["net"].google_organization_iam_member.org_parent_iam["roles/accesscontextmanager.policyAdmin"]: Still creating... [10s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/essentialcontacts.admin"]: Still creating... [20s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/securitycenter.sourcesEditor"]: Still creating... [10s elapsed]
module.org_iam_member["net"].google_organization_iam_member.org_parent_iam["roles/browser"]: Still creating... [10s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.organizationAdmin"]: Still creating... [10s elapsed]
module.org_iam_member["proj"].google_organization_iam_member.org_parent_iam["roles/browser"]: Still creating... [10s elapsed]
module.org_iam_member["proj"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.organizationAdmin"]: Still creating... [10s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.organizationViewer"]: Still creating... [20s elapsed]
module.org_iam_member["proj"].google_organization_iam_member.org_parent_iam["roles/accesscontextmanager.policyAdmin"]: Still creating... [20s elapsed]
module.org_iam_member["bootstrap"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.organizationAdmin"]: Still creating... [20s elapsed]
module.org_iam_member["net"].google_organization_iam_member.org_parent_iam["roles/accesscontextmanager.policyAdmin"]: Still creating... [20s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/essentialcontacts.admin"]: Still creating... [30s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/securitycenter.sourcesEditor"]: Still creating... [20s elapsed]
module.org_iam_member["net"].google_organization_iam_member.org_parent_iam["roles/browser"]: Still creating... [20s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.organizationAdmin"]: Still creating... [20s elapsed]
module.org_iam_member["proj"].google_organization_iam_member.org_parent_iam["roles/browser"]: Still creating... [20s elapsed]
module.org_iam_member["proj"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.organizationAdmin"]: Still creating... [20s elapsed]

module.org_iam_member["bootstrap"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.organizationAdmin"]: Still creating... [30s elapsed]
module.org_iam_member["net"].google_organization_iam_member.org_parent_iam["roles/accesscontextmanager.policyAdmin"]: Still creating... [30s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/essentialcontacts.admin"]: Still creating... [40s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/securitycenter.sourcesEditor"]: Still creating... [30s elapsed]
module.org_iam_member["net"].google_organization_iam_member.org_parent_iam["roles/browser"]: Still creating... [30s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.organizationAdmin"]: Still creating... [30s elapsed]
module.org_iam_member["proj"].google_organization_iam_member.org_parent_iam["roles/browser"]: Still creating... [30s elapsed]
module.org_iam_member["proj"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.organizationAdmin"]: Still creating... [30s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/essentialcontacts.admin"]: Creation complete after 46s [id=583675367868/roles/essentialcontacts.admin/serviceAccount:sa-terraform-org@prj-b-seed-cf20.iam.gserviceaccount.com]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/accesscontextmanager.policyAdmin"]: Creating...
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.organizationViewer"]: Still creating... [40s elapsed]
module.org_iam_member["proj"].google_organization_iam_member.org_parent_iam["roles/accesscontextmanager.policyAdmin"]: Still creating... [40s elapsed]
module.org_iam_member["bootstrap"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.organizationAdmin"]: Still creating... [40s elapsed]
module.org_iam_member["net"].google_organization_iam_member.org_parent_iam["roles/accesscontextmanager.policyAdmin"]: Still creating... [40s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/securitycenter.sourcesEditor"]: Still creating... [40s elapsed]
module.org_iam_member["net"].google_organization_iam_member.org_parent_iam["roles/browser"]: Still creating... [40s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.organizationAdmin"]: Still creating... [40s elapsed]
module.org_iam_member["proj"].google_organization_iam_member.org_parent_iam["roles/browser"]: Still creating... [40s elapsed]
module.org_iam_member["proj"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.organizationAdmin"]: Still creating... [40s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.organizationViewer"]: Creation complete after 47s [id=583675367868/roles/resourcemanager.organizationViewer/serviceAccount:sa-terraform-org@prj-b-seed-cf20.iam.gserviceaccount.com]
module.org_iam_member["env"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.tagUser"]: Creating...
module.org_iam_member["proj"].google_organization_iam_member.org_parent_iam["roles/accesscontextmanager.policyAdmin"]: Creation complete after 47s [id=583675367868/roles/accesscontextmanager.policyAdmin/serviceAccount:sa-terraform-proj@prj-b-seed-cf20.iam.gserviceaccount.com]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/securitycenter.notificationConfigEditor"]: Creating...
module.org_iam_member["bootstrap"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.organizationAdmin"]: Creation complete after 47s [id=583675367868/roles/resourcemanager.organizationAdmin/serviceAccount:sa-terraform-bootstrap@prj-b-seed-cf20.iam.gserviceaccount.com]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/browser"]: Creating...
module.org_iam_member["net"].google_organization_iam_member.org_parent_iam["roles/accesscontextmanager.policyAdmin"]: Creation complete after 46s [id=583675367868/roles/accesscontextmanager.policyAdmin/serviceAccount:sa-terraform-net@prj-b-seed-cf20.iam.gserviceaccount.com]
module.org_iam_member["net"].google_organization_iam_member.org_parent_iam["roles/compute.xpnAdmin"]: Creating...
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/accesscontextmanager.policyAdmin"]: Still creating... [10s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/securitycenter.sourcesEditor"]: Creation complete after 46s [id=583675367868/roles/securitycenter.sourcesEditor/serviceAccount:sa-terraform-org@prj-b-seed-cf20.iam.gserviceaccount.com]
google_billing_account_iam_member.billing_admin_user["bootstrap"]: Creating...
module.org_iam_member["net"].google_organization_iam_member.org_parent_iam["roles/browser"]: Creation complete after 47s [id=583675367868/roles/browser/serviceAccount:sa-terraform-net@prj-b-seed-cf20.iam.gserviceaccount.com]
google_billing_account_iam_member.billing_admin_user["org"]: Creating...
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.organizationAdmin"]: Creation complete after 47s [id=583675367868/roles/resourcemanager.organizationAdmin/serviceAccount:sa-terraform-org@prj-b-seed-cf20.iam.gserviceaccount.com]
google_billing_account_iam_member.billing_admin_user["env"]: Creating...
google_billing_account_iam_member.billing_admin_user["bootstrap"]: Creation complete after 5s [id=019283-6F1AB5-7AD576/roles/billing.admin/serviceAccount:sa-terraform-bootstrap@prj-b-seed-cf20.iam.gserviceaccount.com]
google_billing_account_iam_member.billing_admin_user["net"]: Creating...
module.org_iam_member["proj"].google_organization_iam_member.org_parent_iam["roles/browser"]: Creation complete after 48s [id=583675367868/roles/browser/serviceAccount:sa-terraform-proj@prj-b-seed-cf20.iam.gserviceaccount.com]
google_billing_account_iam_member.billing_admin_user["proj"]: Creating...
module.org_iam_member["proj"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.organizationAdmin"]: Creation complete after 47s [id=583675367868/roles/resourcemanager.organizationAdmin/serviceAccount:sa-terraform-proj@prj-b-seed-cf20.iam.gserviceaccount.com]
module.seed_bootstrap.google_storage_bucket.org_terraform_state: Creating...
module.seed_bootstrap.google_storage_bucket.org_terraform_state: Creation complete after 1s [id=bkt-prj-b-seed-tfstate-8bea]
module.seed_bootstrap.google_folder_iam_binding.project_creator[0]: Creating...
module.org_iam_member["env"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.tagUser"]: Still creating... [10s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/securitycenter.notificationConfigEditor"]: Still creating... [10s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/browser"]: Still creating... [10s elapsed]
module.org_iam_member["net"].google_organization_iam_member.org_parent_iam["roles/compute.xpnAdmin"]: Still creating... [10s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/accesscontextmanager.policyAdmin"]: Still creating... [20s elapsed]
google_billing_account_iam_member.billing_admin_user["org"]: Still creating... [10s elapsed]
google_billing_account_iam_member.billing_admin_user["env"]: Still creating... [10s elapsed]
module.seed_bootstrap.google_folder_iam_binding.project_creator[0]: Creation complete after 5s [id=folders/444651735300/roles/resourcemanager.projectCreator]
module.seed_bootstrap.google_storage_bucket_iam_member.orgadmins_state_iam[0]: Creating...
google_billing_account_iam_member.billing_admin_user["net"]: Still creating... [10s elapsed]
module.seed_bootstrap.google_storage_bucket_iam_member.orgadmins_state_iam[0]: Creation complete after 4s [id=b/bkt-prj-b-seed-tfstate-8bea/roles/storage.admin/group:gcp-organization-admins3@obrienlabs.dev]
module.gcp_projects_state_bucket.google_storage_bucket.bucket: Creating...
google_billing_account_iam_member.billing_admin_user["proj"]: Still creating... [10s elapsed]
module.gcp_projects_state_bucket.google_storage_bucket.bucket: Creation complete after 1s [id=bkt-prj-b-seed-cf20-gcp-projects-tfstate]
module.tf_source.module.cloudbuild_project.module.project-factory.random_id.random_project_id_suffix: Creating...
module.tf_source.module.cloudbuild_project.module.project-factory.random_id.random_project_id_suffix: Creation complete after 0s [id=q5E]
module.tf_source.module.cloudbuild_project.module.project-factory.google_project.main: Creating...
module.org_iam_member["env"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.tagUser"]: Still creating... [20s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/securitycenter.notificationConfigEditor"]: Still creating... [20s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/browser"]: Still creating... [20s elapsed]
module.org_iam_member["net"].google_organization_iam_member.org_parent_iam["roles/compute.xpnAdmin"]: Still creating... [20s elapsed]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/accesscontextmanager.policyAdmin"]: Still creating... [30s elapsed]
google_billing_account_iam_member.billing_admin_user["org"]: Creation complete after 19s [id=019283-6F1AB5-7AD576/roles/billing.admin/serviceAccount:sa-terraform-org@prj-b-seed-cf20.iam.gserviceaccount.com]
google_billing_account_iam_member.billing_admin_user["env"]: Creation complete after 20s [id=019283-6F1AB5-7AD576/roles/billing.admin/serviceAccount:sa-terraform-env@prj-b-seed-cf20.iam.gserviceaccount.com]
google_billing_account_iam_member.billing_admin_user["net"]: Creation complete after 16s [id=019283-6F1AB5-7AD576/roles/billing.admin/serviceAccount:sa-terraform-net@prj-b-seed-cf20.iam.gserviceaccount.com]
google_billing_account_iam_member.billing_admin_user["proj"]: Creation complete after 15s [id=019283-6F1AB5-7AD576/roles/billing.admin/serviceAccount:sa-terraform-proj@prj-b-seed-cf20.iam.gserviceaccount.com]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/accesscontextmanager.policyAdmin"]: Creation complete after 33s [id=583675367868/roles/accesscontextmanager.policyAdmin/serviceAccount:sa-terraform-org@prj-b-seed-cf20.iam.gserviceaccount.com]
module.org_iam_member["env"].google_organization_iam_member.org_parent_iam["roles/resourcemanager.tagUser"]: Creation complete after 24s [id=583675367868/roles/resourcemanager.tagUser/serviceAccount:sa-terraform-env@prj-b-seed-cf20.iam.gserviceaccount.com]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/securitycenter.notificationConfigEditor"]: Creation complete after 24s [id=583675367868/roles/securitycenter.notificationConfigEditor/serviceAccount:sa-terraform-org@prj-b-seed-cf20.iam.gserviceaccount.com]
module.org_iam_member["org"].google_organization_iam_member.org_parent_iam["roles/browser"]: Creation complete after 24s [id=583675367868/roles/browser/serviceAccount:sa-terraform-org@prj-b-seed-cf20.iam.gserviceaccount.com]
module.org_iam_member["net"].google_organization_iam_member.org_parent_iam["roles/compute.xpnAdmin"]: Creation complete after 24s [id=583675367868/roles/compute.xpnAdmin/serviceAccount:sa-terraform-net@prj-b-seed-cf20.iam.gserviceaccount.com]
module.tf_source.module.cloudbuild_project.module.project-factory.google_project.main: Still creating... [10s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.google_project.main: Still creating... [20s elapsed]

module.tf_source.module.cloudbuild_project.module.project-factory.google_project.main: Still creating... [30s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.google_project.main: Still creating... [40s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.google_project.main: Still creating... [50s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.google_project.main: Still creating... [1m0s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.google_project.main: Still creating... [1m10s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.google_project.main: Still creating... [1m20s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.google_project.main: Still creating... [1m30s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.google_project.main: Still creating... [1m40s elapsed]

module.tf_source.module.cloudbuild_project.module.project-factory.google_project.main: Still creating... [1m50s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.google_project.main: Still creating... [2m0s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.google_project.main: Still creating... [2m10s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.google_project.main: Still creating... [2m20s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.google_project.main: Still creating... [2m30s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.google_project.main: Still creating... [2m40s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.google_project.main: Still creating... [2m50s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.google_project.main: Still creating... [3m0s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.google_project.main: Still creating... [3m10s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.google_project.main: Still creating... [3m20s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.google_project.main: Still creating... [3m30s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.google_project.main: Creation complete after 3m34s [id=projects/prj-b-cicd-fgbs]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["artifactregistry.googleapis.com"]: Creating...
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["admin.googleapis.com"]: Creating...
module.tf_source.module.cloudbuild_project.module.project-factory.google_service_account.default_service_account[0]: Creating...
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["servicenetworking.googleapis.com"]: Creating...
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["serviceusage.googleapis.com"]: Creating...
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["cloudbilling.googleapis.com"]: Creating...
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["cloudresourcemanager.googleapis.com"]: Creating...
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["storage-api.googleapis.com"]: Creating...
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["workflows.googleapis.com"]: Creating...
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["cloudscheduler.googleapis.com"]: Creating...
module.tf_source.module.cloudbuild_project.module.project-factory.google_service_account.default_service_account[0]: Creation complete after 1s [id=projects/prj-b-cicd-fgbs/serviceAccounts/project-service-account@prj-b-cicd-fgbs.iam.gserviceaccount.com]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["appengine.googleapis.com"]: Creating...

till creating... [10s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["artifactregistry.googleapis.com"]: Still creating... [10s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["cloudbilling.googleapis.com"]: Still creating... [10s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["servicenetworking.googleapis.com"]: Still creating... [10s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["serviceusage.googleapis.com"]: Still creating... [10s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["cloudresourcemanager.googleapis.com"]: Still creating... [10s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["storage-api.googleapis.com"]: Still creating... [10s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["workflows.googleapis.com"]: Still creating... [10s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["cloudscheduler.googleapis.com"]: Still creating... [10s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["appengine.googleapis.com"]: Still creating... [10s elapsed]

module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["artifactregistry.googleapis.com"]: Still creating... [20s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["admin.googleapis.com"]: Still creating... [20s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["serviceusage.googleapis.com"]: Still creating... [20s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["cloudbilling.googleapis.com"]: Still creating... [20s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["servicenetworking.googleapis.com"]: Still creating... [20s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["cloudresourcemanager.googleapis.com"]: Still creating... [20s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["storage-api.googleapis.com"]: Still creating... [20s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["workflows.googleapis.com"]: Still creating... [20s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["cloudscheduler.googleapis.com"]: Still creating... [20s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["appengine.googleapis.com"]: Still creating... [20s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["storage-api.googleapis.com"]: Creation complete after 22s [id=prj-b-cicd-fgbs/storage-api.googleapis.com]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["artifactregistry.googleapis.com"]: Creation complete after 22s [id=prj-b-cicd-fgbs/artifactregistry.googleapis.com]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["serviceusage.googleapis.com"]: Creation complete after 22s [id=prj-b-cicd-fgbs/serviceusage.googleapis.com]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["servicenetworking.googleapis.com"]: Creation complete after 22s [id=prj-b-cicd-fgbs/servicenetworking.googleapis.com]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["cloudbuild.googleapis.com"]: Creating...
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["sourcerepo.googleapis.com"]: Creating...
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["billingbudgets.googleapis.com"]: Creating...
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["appengine.googleapis.com"]: Creation complete after 21s [id=prj-b-cicd-fgbs/appengine.googleapis.com]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["cloudscheduler.googleapis.com"]: Creation complete after 22s [id=prj-b-cicd-fgbs/cloudscheduler.googleapis.com]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["cloudresourcemanager.googleapis.com"]: Creation complete after 22s [id=prj-b-cicd-fgbs/cloudresourcemanager.googleapis.com]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["workflows.googleapis.com"]: Creation complete after 22s [id=prj-b-cicd-fgbs/workflows.googleapis.com]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["cloudbilling.googleapis.com"]: Creation complete after 22s [id=prj-b-cicd-fgbs/cloudbilling.googleapis.com]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["admin.googleapis.com"]: Creation complete after 22s [id=prj-b-cicd-fgbs/admin.googleapis.com]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["iam.googleapis.com"]: Creating...
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["logging.googleapis.com"]: Creating...
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["bigquery.googleapis.com"]: Creating...
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["dns.googleapis.com"]: Creating...
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["compute.googleapis.com"]: Creating...
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["compute.googleapis.com"]: Creation complete after 2s [id=prj-b-cicd-fgbs/compute.googleapis.com]

"]: Still creating... [10s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["cloudbuild.googleapis.com"]: Still creating... [10s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["billingbudgets.googleapis.com"]: Still creating... [10s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["iam.googleapis.com"]: Still creating... [10s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["logging.googleapis.com"]: Still creating... [10s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["bigquery.googleapis.com"]: Still creating... [10s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["dns.googleapis.com"]: Still creating... [10s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["billingbudgets.googleapis.com"]: Still creating... [20s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["sourcerepo.googleapis.com"]: Still creating... [20s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["cloudbuild.googleapis.com"]: Still creating... [20s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["iam.googleapis.com"]: Still creating... [20s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["logging.googleapis.com"]: Still creating... [20s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["bigquery.googleapis.com"]: Still creating... [20s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["dns.googleapis.com"]: Still creating... [20s elapsed]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["iam.googleapis.com"]: Creation complete after 22s [id=prj-b-cicd-fgbs/iam.googleapis.com]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["dns.googleapis.com"]: Creation complete after 21s [id=prj-b-cicd-fgbs/dns.googleapis.com]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["cloudbuild.googleapis.com"]: Creation complete after 22s [id=prj-b-cicd-fgbs/cloudbuild.googleapis.com]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["bigquery.googleapis.com"]: Creation complete after 22s [id=prj-b-cicd-fgbs/bigquery.googleapis.com]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["sourcerepo.googleapis.com"]: Creation complete after 22s [id=prj-b-cicd-fgbs/sourcerepo.googleapis.com]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["logging.googleapis.com"]: Creation complete after 22s [id=prj-b-cicd-fgbs/logging.googleapis.com]
module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["billingbudgets.googleapis.com"]: Creation complete after 22s [id=prj-b-cicd-fgbs/billingbudgets.googleapis.com]
module.tf_source.module.cloudbuild_project.module.project-factory.google_project_default_service_accounts.default_service_accounts[0]: Creating...
module.tf_source.google_sourcerepo_repository.gcp_repo["gcp-policies"]: Creating...
module.tf_source.google_project_iam_member.org_admins_cloudbuild_viewer: Creating...
module.tf_source.google_sourcerepo_repository.gcp_repo["gcp-org"]: Creating...
module.tf_source.google_project_iam_member.org_admins_source_repo_admin[0]: Creating...
module.tf_source.google_sourcerepo_repository.gcp_repo["gcp-bootstrap"]: Creating...
module.tf_source.google_sourcerepo_repository.gcp_repo["gcp-projects"]: Creating...
module.tf_source.google_sourcerepo_repository.gcp_repo["gcp-networks"]: Creating...
module.tf_source.google_project_iam_member.org_admins_cloudbuild_editor: Creating...
module.tf_source.google_sourcerepo_repository.gcp_repo["tf-cloudbuilder"]: Creating...
module.tf_source.module.cloudbuild_project.module.project-factory.google_project_default_service_accounts.default_service_accounts[0]: Creation complete after 0s [id=projects/prj-b-cicd-fgbs]
module.tf_source.google_sourcerepo_repository.gcp_repo["gcp-environments"]: Creating...
module.tf_source.google_sourcerepo_repository.gcp_repo["gcp-bootstrap"]: Creation complete after 1s [id=projects/prj-b-cicd-fgbs/repos/gcp-bootstrap]
module.tf_source.google_sourcerepo_repository.gcp_repo["gcp-projects"]: Creation complete after 1s [id=projects/prj-b-cicd-fgbs/repos/gcp-projects]
module.tf_source.module.cloudbuild_bucket.google_storage_bucket.bucket: Creating...
module.tf_source.google_sourcerepo_repository.gcp_repo["gcp-policies"]: Creation complete after 1s [id=projects/prj-b-cicd-fgbs/repos/gcp-policies]
module.tf_source.google_sourcerepo_repository.gcp_repo["gcp-org"]: Creation complete after 1s [id=projects/prj-b-cicd-fgbs/repos/gcp-org]
module.tf_source.google_sourcerepo_repository.gcp_repo["gcp-environments"]: Creation complete after 1s [id=projects/prj-b-cicd-fgbs/repos/gcp-environments]
module.tf_source.module.cloudbuild_bucket.google_storage_bucket.bucket: Creation complete after 1s [id=prj-b-cicd-fgbs_cloudbuild]
module.tf_source.google_storage_bucket_iam_member.cloudbuild_iam: Creating...
module.tf_source.google_sourcerepo_repository.gcp_repo["tf-cloudbuilder"]: Creation complete after 2s [id=projects/prj-b-cicd-fgbs/repos/tf-cloudbuilder]
module.tf_source.google_sourcerepo_repository.gcp_repo["gcp-networks"]: Creation complete after 2s [id=projects/prj-b-cicd-fgbs/repos/gcp-networks]
module.tf_source.google_storage_bucket_iam_member.cloudbuild_iam: Creation complete after 4s [id=b/prj-b-cicd-fgbs_cloudbuild/roles/storage.admin/serviceAccount:991985440030@cloudbuild.gserviceaccount.com]

module.tf_source.google_project_iam_member.org_admins_cloudbuild_editor: Creation complete after 7s [id=prj-b-cicd-fgbs/roles/cloudbuild.builds.editor/group:gcp-organization-admins3@obrienlabs.dev]
module.tf_source.google_project_iam_member.org_admins_cloudbuild_viewer: Creation complete after 7s [id=prj-b-cicd-fgbs/roles/viewer/group:gcp-organization-admins3@obrienlabs.dev]
module.tf_source.google_project_iam_member.org_admins_source_repo_admin[0]: Creation complete after 8s [id=prj-b-cicd-fgbs/roles/source.admin/group:gcp-organization-admins3@obrienlabs.dev]
google_sourcerepo_repository_iam_member.member["org"]: Creating...
module.bootstrap_csr_repo.null_resource.run_command[0]: Creating...
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/dns.admin"]: Creating...
module.bootstrap_csr_repo.null_resource.run_command[0]: Provisioning with 'local-exec'...
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): Executing: ["/bin/sh" "-c" "PATH=/google-cloud-sdk/bin:$PATH\n./scripts/push-to-repo.sh prj-b-cicd-fgbs tf-cloudbuilder ./Dockerfile\n"]
google_sourcerepo_repository_iam_member.member["env"]: Creating...
google_sourcerepo_repository_iam_member.member["net"]: Creating...
google_sourcerepo_repository_iam_member.member["bootstrap"]: Creating...
module.tf_cloud_builder.google_service_account.workflow_sa[0]: Creating...
google_sourcerepo_repository_iam_member.member["proj"]: Creating...
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/artifactregistry.admin"]: Creating...
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/compute.networkAdmin"]: Creating...
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + '[' 3 -lt 3 ']'
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + CSR_PROJECT_ID=prj-b-cicd-fgbs
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + CSR_NAME=tf-cloudbuilder
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + DOCKERFILE_PATH=./Dockerfile
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): ++ mktemp -d
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + tmp_dir=/tmp/tmp.UPb5Ov3BbM
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + gcloud source repos clone tf-cloudbuilder /tmp/tmp.UPb5Ov3BbM --project prj-b-cicd-fgbs
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): Cloning into '/tmp/tmp.UPb5Ov3BbM'...
module.tf_cloud_builder.google_service_account.workflow_sa[0]: Creation complete after 0s [id=projects/prj-b-cicd-fgbs/serviceAccounts/terraform-runner-workflow-sa@prj-b-cicd-fgbs.iam.gserviceaccount.com]
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/cloudscheduler.admin"]: Creating...
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): warning: You appear to have cloned an empty repository.
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): Project [prj-b-cicd-fgbs] repository [tf-cloudbuilder] was cloned to [/tmp/tmp.UPb5Ov3BbM].
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + cp ./Dockerfile /tmp/tmp.UPb5Ov3BbM
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + pushd /tmp/tmp.UPb5Ov3BbM
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): /tmp/tmp.UPb5Ov3BbM ~/tef-oldev3/pbmm-on-gcp-onboarding/0-bootstrap
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + git config credential.helper gcloud.sh
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + git config init.defaultBranch main
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + git config user.email terraform-robot@example.com
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + git config user.name 'TF Robot'
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + git checkout main
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): error: pathspec 'main' did not match any file(s) known to git
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + git checkout -b main
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): Switched to a new branch 'main'
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + git add Dockerfile
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + git commit -m 'Initialize tf dockerfile repo'
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): [main (root-commit) c1c0f29] Initialize tf dockerfile repo
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec):  1 file changed, 39 insertions(+)
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec):  create mode 100644 Dockerfile
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + git push origin main -f
google_sourcerepo_repository_iam_member.member["org"]: Creation complete after 4s [id=projects/prj-b-cicd-fgbs/repos/gcp-policies/roles/viewer/serviceAccount:sa-terraform-org@prj-b-seed-cf20.iam.gserviceaccount.com]
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/workflows.admin"]: Creating...
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): remote: Waiting for private key checker: 1/1 objects left
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): To https://source.developers.google.com/p/prj-b-cicd-fgbs/r/tf-cloudbuilder
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec):  * [new branch]      main -> main
module.bootstrap_csr_repo.null_resource.run_command[0]: Creation complete after 6s [id=5317162065932165996]
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/iam.workloadIdentityPoolAdmin"]: Creating...
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/cloudscheduler.admin"]: Creation complete after 6s [id=prj-b-cicd-fgbs/roles/cloudscheduler.admin/serviceAccount:sa-terraform-bootstrap@prj-b-seed-cf20.iam.gserviceaccount.com]
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/resourcemanager.projectDeleter"]: Creating...
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/artifactregistry.admin"]: Creation complete after 7s [id=prj-b-cicd-fgbs/roles/artifactregistry.admin/serviceAccount:sa-terraform-bootstrap@prj-b-seed-cf20.iam.gserviceaccount.com]
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/cloudbuild.builds.editor"]: Creating...
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/compute.networkAdmin"]: Creation complete after 7s [id=prj-b-cicd-fgbs/roles/compute.networkAdmin/serviceAccount:sa-terraform-bootstrap@prj-b-seed-cf20.iam.gserviceaccount.com]
module.tf_cloud_builder.google_artifact_registry_repository.tf-image-repo: Creating...
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/dns.admin"]: Creation complete after 8s [id=prj-b-cicd-fgbs/roles/dns.admin/serviceAccount:sa-terraform-bootstrap@prj-b-seed-cf20.iam.gserviceaccount.com]
module.tf_cloud_builder.google_service_account.cb_sa[0]: Creating...
module.tf_cloud_builder.google_service_account.cb_sa[0]: Creation complete after 1s [id=projects/prj-b-cicd-fgbs/serviceAccounts/tf-cb-builder-sa@prj-b-cicd-fgbs.iam.gserviceaccount.com]
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/source.admin"]: Creating...
google_sourcerepo_repository_iam_member.member["env"]: Still creating... [10s elapsed]
google_sourcerepo_repository_iam_member.member["net"]: Still creating... [10s elapsed]
google_sourcerepo_repository_iam_member.member["bootstrap"]: Still creating... [10s elapsed]
google_sourcerepo_repository_iam_member.member["proj"]: Still creating... [10s elapsed]

module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/source.admin"]: Creation complete after 7s [id=prj-b-cicd-fgbs/roles/source.admin/serviceAccount:sa-terraform-bootstrap@prj-b-seed-cf20.iam.gserviceaccount.com]
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/iam.serviceAccountAdmin"]: Creating...
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/iam.workloadIdentityPoolAdmin"]: Still creating... [10s elapsed]
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/workflows.admin"]: Creation complete after 12s [id=prj-b-cicd-fgbs/roles/workflows.admin/serviceAccount:sa-terraform-bootstrap@prj-b-seed-cf20.iam.gserviceaccount.com]
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/cloudbuild.workerPoolOwner"]: Creating...
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/iam.workloadIdentityPoolAdmin"]: Creation complete after 11s [id=prj-b-cicd-fgbs/roles/iam.workloadIdentityPoolAdmin/serviceAccount:sa-terraform-bootstrap@prj-b-seed-cf20.iam.gserviceaccount.com]
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/storage.admin"]: Creating...
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/cloudbuild.builds.editor"]: Creation complete after 10s [id=prj-b-cicd-fgbs/roles/cloudbuild.builds.editor/serviceAccount:sa-terraform-bootstrap@prj-b-seed-cf20.iam.gserviceaccount.com]
module.tf_cloud_builder.google_project_iam_member.trigger_builds: Creating...
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/resourcemanager.projectDeleter"]: Creation complete after 10s [id=prj-b-cicd-fgbs/roles/resourcemanager.projectDeleter/serviceAccount:sa-terraform-bootstrap@prj-b-seed-cf20.iam.gserviceaccount.com]
module.tf_cloud_builder.google_workflows_workflow.builder: Creating...
module.tf_cloud_builder.google_artifact_registry_repository.tf-image-repo: Still creating... [10s elapsed]
module.tf_cloud_builder.google_artifact_registry_repository.tf-image-repo: Creation complete after 11s [id=projects/prj-b-cicd-fgbs/locations/northamerica-northeast1/repositories/tf-runners]
module.tf_cloud_builder.google_project_iam_member.invoke_workflow_scheduler: Creating...
google_sourcerepo_repository_iam_member.member["net"]: Creation complete after 19s [id=projects/prj-b-cicd-fgbs/repos/gcp-policies/roles/viewer/serviceAccount:sa-terraform-net@prj-b-seed-cf20.iam.gserviceaccount.com]
module.tf_cloud_builder.google_service_account_iam_member.use_cb_sa: Creating...
google_sourcerepo_repository_iam_member.member["env"]: Creation complete after 19s [id=projects/prj-b-cicd-fgbs/repos/gcp-policies/roles/viewer/serviceAccount:sa-terraform-env@prj-b-seed-cf20.iam.gserviceaccount.com]
module.tf_private_pool.module.peered_network[0].module.vpc.google_compute_network.network: Creating...
google_sourcerepo_repository_iam_member.member["bootstrap"]: Creation complete after 19s [id=projects/prj-b-cicd-fgbs/repos/gcp-policies/roles/viewer/serviceAccount:sa-terraform-bootstrap@prj-b-seed-cf20.iam.gserviceaccount.com]
module.tf_cloud_builder.module.bucket.google_storage_bucket.bucket: Creating...
google_sourcerepo_repository_iam_member.member["proj"]: Creation complete after 19s [id=projects/prj-b-cicd-fgbs/repos/gcp-policies/roles/viewer/serviceAccount:sa-terraform-proj@prj-b-seed-cf20.iam.gserviceaccount.com]
module.tf_cloud_builder.google_artifact_registry_repository_iam_member.workflow_list: Creating...
module.tf_cloud_builder.google_workflows_workflow.builder: Creation complete after 3s [id=projects/prj-b-cicd-fgbs/locations/northamerica-northeast1/workflows/terraform-runner-workflow]
module.tf_cloud_builder.google_artifact_registry_repository_iam_member.push_images: Creating...
module.tf_cloud_builder.module.bucket.google_storage_bucket.bucket: Creation complete after 1s [id=bkt-prj-b-cicd-fgbs-tf-cloudbuilder-build-logs]
module.tf_cloud_builder.google_project_iam_member.logs_writer: Creating...
module.tf_cloud_builder.google_service_account_iam_member.use_cb_sa: Creation complete after 4s [id=projects/prj-b-cicd-fgbs/serviceAccounts/tf-cb-builder-sa@prj-b-cicd-fgbs.iam.gserviceaccount.com/roles/iam.serviceAccountUser/serviceAccount:terraform-runner-workflow-sa@prj-b-cicd-fgbs.iam.gserviceaccount.com]
module.tf_cloud_builder.google_sourcerepo_repository_iam_member.member[0]: Creating...
module.tf_cloud_builder.google_project_iam_member.invoke_workflow_scheduler: Creation complete after 6s [id=prj-b-cicd-fgbs/roles/workflows.invoker/serviceAccount:terraform-runner-workflow-sa@prj-b-cicd-fgbs.iam.gserviceaccount.com]
module.tf_cloud_builder.google_cloud_scheduler_job.trigger_workflow: Creating...
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/iam.serviceAccountAdmin"]: Still creating... [10s elapsed]
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/cloudbuild.workerPoolOwner"]: Still creating... [10s elapsed]
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/storage.admin"]: Still creating... [10s elapsed]
module.tf_cloud_builder.google_sourcerepo_repository_iam_member.member[0]: Creation complete after 4s [id=projects/prj-b-cicd-fgbs/repos/tf-cloudbuilder/roles/viewer/serviceAccount:tf-cb-builder-sa@prj-b-cicd-fgbs.iam.gserviceaccount.com]
google_artifact_registry_repository_iam_member.terraform_sa_artifact_registry_reader["bootstrap"]: Creating...
module.tf_cloud_builder.google_project_iam_member.trigger_builds: Still creating... [10s elapsed]
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/cloudbuild.workerPoolOwner"]: Creation complete after 11s [id=prj-b-cicd-fgbs/roles/cloudbuild.workerPoolOwner/serviceAccount:sa-terraform-bootstrap@prj-b-seed-cf20.iam.gserviceaccount.com]
google_artifact_registry_repository_iam_member.terraform_sa_artifact_registry_reader["net"]: Creating...
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/storage.admin"]: Creation complete after 11s [id=prj-b-cicd-fgbs/roles/storage.admin/serviceAccount:sa-terraform-bootstrap@prj-b-seed-cf20.iam.gserviceaccount.com]
google_artifact_registry_repository_iam_member.terraform_sa_artifact_registry_reader["proj"]: Creating...
module.tf_cloud_builder.google_project_iam_member.trigger_builds: Creation complete after 11s [id=prj-b-cicd-fgbs/roles/cloudbuild.builds.editor/serviceAccount:terraform-runner-workflow-sa@prj-b-cicd-fgbs.iam.gserviceaccount.com]
google_artifact_registry_repository_iam_member.terraform_sa_artifact_registry_reader["env"]: Creating...
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/iam.serviceAccountAdmin"]: Creation complete after 12s [id=prj-b-cicd-fgbs/roles/iam.serviceAccountAdmin/serviceAccount:sa-terraform-bootstrap@prj-b-seed-cf20.iam.gserviceaccount.com]
google_artifact_registry_repository_iam_member.terraform_sa_artifact_registry_reader["org"]: Creating...
module.tf_cloud_builder.google_project_iam_member.logs_writer: Creation complete after 8s [id=prj-b-cicd-fgbs/roles/logging.logWriter/serviceAccount:tf-cb-builder-sa@prj-b-cicd-fgbs.iam.gserviceaccount.com]
module.tf_cloud_builder.google_storage_bucket_iam_member.member: Creating...
module.tf_cloud_builder.google_artifact_registry_repository_iam_member.workflow_list: Creation complete after 9s [id=projects/prj-b-cicd-fgbs/locations/northamerica-northeast1/repositories/tf-runners/roles/artifactregistry.reader/serviceAccount:terraform-runner-workflow-sa@prj-b-cicd-fgbs.iam.gserviceaccount.com]
module.bootstrap_projects_remove_editor["cicd"].google_project_iam_binding.iam_remove["roles/editor"]: Creating...
module.tf_cloud_builder.google_artifact_registry_repository_iam_member.push_images: Creation complete after 9s [id=projects/prj-b-cicd-fgbs/locations/northamerica-northeast1/repositories/tf-runners/roles/artifactregistry.writer/serviceAccount:tf-cb-builder-sa@prj-b-cicd-fgbs.iam.gserviceaccount.com]
module.bootstrap_projects_remove_editor["seed"].google_project_iam_binding.iam_remove["roles/editor"]: Creating...
module.tf_private_pool.module.peered_network[0].module.vpc.google_compute_network.network: Still creating... [10s elapsed]

module.tf_cloud_builder.google_storage_bucket_iam_member.member: Creation complete after 4s [id=b/bkt-prj-b-cicd-fgbs-tf-cloudbuilder-build-logs/roles/storage.admin/serviceAccount:tf-cb-builder-sa@prj-b-cicd-fgbs.iam.gserviceaccount.com]
module.tf_cloud_builder.google_cloud_scheduler_job.trigger_workflow: Creation complete after 9s [id=projects/prj-b-cicd-fgbs/locations/northamerica-northeast1/jobs/trigger-terraform-runner-workflow]
module.bootstrap_projects_remove_editor["cicd"].google_project_iam_binding.iam_remove["roles/editor"]: Creation complete after 8s [id=prj-b-cicd-fgbs/roles/editor]
module.bootstrap_projects_remove_editor["seed"].google_project_iam_binding.iam_remove["roles/editor"]: Creation complete after 7s [id=prj-b-seed-cf20/roles/editor]
google_artifact_registry_repository_iam_member.terraform_sa_artifact_registry_reader["bootstrap"]: Still creating... [10s elapsed]
google_artifact_registry_repository_iam_member.terraform_sa_artifact_registry_reader["net"]: Still creating... [10s elapsed]
google_artifact_registry_repository_iam_member.terraform_sa_artifact_registry_reader["proj"]: Still creating... [10s elapsed]
google_artifact_registry_repository_iam_member.terraform_sa_artifact_registry_reader["env"]: Still creating... [10s elapsed]
google_artifact_registry_repository_iam_member.terraform_sa_artifact_registry_reader["org"]: Still creating... [10s elapsed]
module.tf_private_pool.module.peered_network[0].module.vpc.google_compute_network.network: Still creating... [20s elapsed]
google_artifact_registry_repository_iam_member.terraform_sa_artifact_registry_reader["bootstrap"]: Still creating... [20s elapsed]
google_artifact_registry_repository_iam_member.terraform_sa_artifact_registry_reader["net"]: Still creating... [20s elapsed]
google_artifact_registry_repository_iam_member.terraform_sa_artifact_registry_reader["proj"]: Still creating... [20s elapsed]
google_artifact_registry_repository_iam_member.terraform_sa_artifact_registry_reader["env"]: Still creating... [20s elapsed]
google_artifact_registry_repository_iam_member.terraform_sa_artifact_registry_reader["org"]: Still creating... [20s elapsed]
google_artifact_registry_repository_iam_member.terraform_sa_artifact_registry_reader["bootstrap"]: Creation complete after 22s [id=projects/prj-b-cicd-fgbs/locations/northamerica-northeast1/repositories/tf-runners/roles/artifactregistry.reader/serviceAccount:sa-terraform-bootstrap@prj-b-seed-cf20.iam.gserviceaccount.com]
module.tf_private_pool.module.peered_network[0].module.vpc.google_compute_network.network: Still creating... [30s elapsed]
google_artifact_registry_repository_iam_member.terraform_sa_artifact_registry_reader["net"]: Creation complete after 21s [id=projects/prj-b-cicd-fgbs/locations/northamerica-northeast1/repositories/tf-runners/roles/artifactregistry.reader/serviceAccount:sa-terraform-net@prj-b-seed-cf20.iam.gserviceaccount.com]
google_artifact_registry_repository_iam_member.terraform_sa_artifact_registry_reader["proj"]: Creation complete after 22s [id=projects/prj-b-cicd-fgbs/locations/northamerica-northeast1/repositories/tf-runners/roles/artifactregistry.reader/serviceAccount:sa-terraform-proj@prj-b-seed-cf20.iam.gserviceaccount.com]
google_artifact_registry_repository_iam_member.terraform_sa_artifact_registry_reader["env"]: Creation complete after 22s [id=projects/prj-b-cicd-fgbs/locations/northamerica-northeast1/repositories/tf-runners/roles/artifactregistry.reader/serviceAccount:sa-terraform-env@prj-b-seed-cf20.iam.gserviceaccount.com]
google_artifact_registry_repository_iam_member.terraform_sa_artifact_registry_reader["org"]: Creation complete after 22s [id=projects/prj-b-cicd-fgbs/locations/northamerica-northeast1/repositories/tf-runners/roles/artifactregistry.reader/serviceAccount:sa-terraform-org@prj-b-seed-cf20.iam.gserviceaccount.com]
module.tf_private_pool.module.peered_network[0].module.vpc.google_compute_network.network: Creation complete after 33s [id=projects/prj-b-cicd-fgbs/global/networks/vpc-b-cbpools]
module.tf_private_pool.google_dns_policy.default_policy[0]: Creating...
module.tf_private_pool.google_compute_global_address.worker_pool_range[0]: Creating...
module.tf_private_pool.module.peered_network[0].module.subnets.google_compute_subnetwork.subnetwork["northamerica-northeast1/sb-b-cbpools-northamerica-northeast1"]: Creating...
module.tf_private_pool.google_dns_policy.default_policy[0]: Creation complete after 1s [id=projects/prj-b-cicd-fgbs/policies/dp-b-cbpools-default-policy]

module.tf_private_pool.module.peered_network[0].module.subnets.google_compute_subnetwork.subnetwork["northamerica-northeast1/sb-b-cbpools-northamerica-northeast1"]: Still creating... [10s elapsed]
module.tf_private_pool.google_compute_global_address.worker_pool_range[0]: Creation complete after 11s [id=projects/prj-b-cicd-fgbs/global/addresses/ga-b-cbpools-worker-pool-range]
module.tf_private_pool.google_service_networking_connection.worker_pool_conn[0]: Creating...
module.tf_private_pool.module.firewall_rules[0].google_compute_firewall.rules["fw-b-cbpools-100-i-a-all-all-all-service-networking"]: Creating...
module.tf_private_pool.module.peered_network[0].module.subnets.google_compute_subnetwork.subnetwork["northamerica-northeast1/sb-b-cbpools-northamerica-northeast1"]: Still creating... [20s elapsed]
module.tf_private_pool.google_service_networking_connection.worker_pool_conn[0]: Still creating... [10s elapsed]
module.tf_private_pool.module.firewall_rules[0].google_compute_firewall.rules["fw-b-cbpools-100-i-a-all-all-all-service-networking"]: Still creating... [10s elapsed]
module.tf_private_pool.module.firewall_rules[0].google_compute_firewall.rules["fw-b-cbpools-100-i-a-all-all-all-service-networking"]: Creation complete after 12s [id=projects/prj-b-cicd-fgbs/global/firewalls/fw-b-cbpools-100-i-a-all-all-all-service-networking]

east1"]: Creation complete after 24s [id=projects/prj-b-cicd-fgbs/regions/northamerica-northeast1/subnetworks/sb-b-cbpools-northamerica-northeast1]
module.tf_private_pool.google_service_networking_connection.worker_pool_conn[0]: Still creating... [20s elapsed]

module.tf_private_pool.google_service_networking_connection.worker_pool_conn[0]: Still creating... [40s elapsed]
module.tf_private_pool.google_service_networking_connection.worker_pool_conn[0]: Still creating... [50s elapsed]
module.tf_private_pool.google_service_networking_connection.worker_pool_conn[0]: Still creating... [1m0s elapsed]
module.tf_private_pool.google_service_networking_connection.worker_pool_conn[0]: Still creating... [1m10s elapsed]

module.tf_private_pool.google_service_networking_connection.worker_pool_conn[0]: Still creating... [1m20s elapsed]
module.tf_private_pool.google_service_networking_connection.worker_pool_conn[0]: Still creating... [1m30s elapsed]
module.tf_private_pool.google_service_networking_connection.worker_pool_conn[0]: Still creating... [1m40s elapsed]
module.tf_private_pool.google_service_networking_connection.worker_pool_conn[0]: Still creating... [1m50s elapsed]
module.tf_private_pool.google_service_networking_connection.worker_pool_conn[0]: Still creating... [2m0s elapsed]
module.tf_private_pool.google_service_networking_connection.worker_pool_conn[0]: Still creating... [2m10s elapsed]
module.tf_private_pool.google_service_networking_connection.worker_pool_conn[0]: Creation complete after 2m13s [id=projects%2Fprj-b-cicd-fgbs%2Fglobal%2Fnetworks%2Fvpc-b-cbpools:servicenetworking.googleapis.com]
module.tf_private_pool.google_compute_network_peering_routes_config.peering_routes[0]: Creating...
module.tf_private_pool.google_cloudbuild_worker_pool.private_pool: Creating...

module.tf_private_pool.google_compute_network_peering_routes_config.peering_routes[0]: Still creating... [10s elapsed]
module.tf_private_pool.google_compute_network_peering_routes_config.peering_routes[0]: Creation complete after 12s [id=projects/prj-b-cicd-fgbs/global/networks/vpc-b-cbpools/networkPeerings/servicenetworking-googleapis-com]
╷
│ Error: Error creating WorkerPool: operation received error: error code "8", message: generic::resource_exhausted: project 991985440030 has insufficient quota to create a new worker pool; see https://cloud.google.com/build/quotas, details: []
│  details: map[]
│ 
│   with module.tf_private_pool.google_cloudbuild_worker_pool.private_pool,
│   on modules/cb-private-pool/main.tf line 30, in resource "google_cloudbuild_worker_pool" "private_pool":
│   30: resource "google_cloudbuild_worker_pool" "private_pool" {
│ 
╵
michael@cloudshell:~/tef-oldev3/pbmm-on-gcp-onboarding/0-bootstrap (tef-oldev3)
obriensystems commented 6 months ago

raised #434

Screenshot 2024-05-09 at 23 48 33 Screenshot 2024-05-09 at 23 52 24

increasing quota

obriensystems commented 6 months ago

rerun apply tomorrow


Plan: 71 to add, 0 to change, 0 to destroy.

Changes to Outputs:
  + cloud_build_private_worker_pool_id = (known after apply)
  + gcs_bucket_cloudbuild_artifacts    = {
      + bootstrap = (known after apply)
      + env       = (known after apply)
      + net       = (known after apply)
      + org       = (known after apply)
      + proj      = (known after apply)
    }
  + gcs_bucket_cloudbuild_logs         = {
      + bootstrap = (known after apply)
      + env       = (known after apply)
      + net       = (known after apply)
      + org       = (known after apply)
      + proj      = (known after apply)
    }
fmichaelobrien commented 6 months ago

Did some testing last night on a clean main for my oldev org - as well as local CSR cloning - both good.  From the script in https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/blob/main/0-bootstrap/scripts/push-to-repo.sh#L32 It looks like it may be strictly related to authentication with your account - as there is no TF SA involved (but do need to verify your terraform-runner-workflow-sa@prj-b-cicd-fgbs.iam.gserviceaccount.com). The key is that your local gcloud csr clone (uses your auth not a git clone token or ssh key) - fails in cloud shell.  Once we fix cloning a csr repo locally in cloud shell - we should be good for terraform.

details in https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/issues/431

local clone

michael@cloudshell:~/tef-oldev3 (tef-oldev2)$ gcloud config set project tef-oldev3
Updated property [core/project].
michael@cloudshell:~/tef-oldev3 (tef-oldev3)$ ls
pbmm-on-gcp-onboarding  terraform
michael@cloudshell:~/tef-oldev3 (tef-oldev3)$ mkdir _test_repo
michael@cloudshell:~/tef-oldev3 (tef-oldev3)$ cd _test_repo
michael@cloudshell:~/tef-oldev3/_test_repo (tef-oldev3)$ gcloud source repos clone gcp-policies --project=prj-b-cicd-fgbs
Cloning into '/home/michael/tef-oldev3/_test_repo/gcp-policies'...
warning: You appear to have cloned an empty repository.
Project [prj-b-cicd-fgbs] repository [gcp-policies] was cloned to [/home/michael/tef-oldev3/_test_repo/gcp-policies].```

TF clone

module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + '[' 3 -lt 3 ']' module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + CSR_PROJECT_ID=prj-b-cicd-fgbs module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + CSR_NAME=tf-cloudbuilder module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + DOCKERFILE_PATH=./Dockerfile module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): ++ mktemp -d module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + tmp_dir=/tmp/tmp.UPb5Ov3BbM module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + gcloud source repos clone tf-cloudbuilder /tmp/tmp.UPb5Ov3BbM --project prj-b-cicd-fgbs module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): Cloning into '/tmp/tmp.UPb5Ov3BbM'... module.tf_cloud_builder.google_service_account.workflow_sa[0]: Creation complete after 0s [id=projects/prj-b-cicd-fgbs/serviceAccounts/terraform-runner-workflow-sa@prj-b-cicd-fgbs.iam.gserviceaccount.com] module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/cloudscheduler.admin"]: Creating... module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): warning: You appear to have cloned an empty repository. module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): Project [prj-b-cicd-fgbs] repository [tf-cloudbuilder] was cloned to [/tmp/tmp.UPb5Ov3BbM]. module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + cp ./Dockerfile /tmp/tmp.UPb5Ov3BbM module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + pushd /tmp/tmp.UPb5Ov3BbM module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): /tmp/tmp.UPb5Ov3BbM ~/tef-oldev3/pbmm-on-gcp-onboarding/0-bootstrap

obriensystems commented 6 months ago

Finish retest 0-bootstrap

Plan: 71 to add, 0 to change, 0 to destroy.

Changes to Outputs:
  + cloud_build_private_worker_pool_id                = (known after apply)
  + gcs_bucket_cloudbuild_artifacts                   = {
      + bootstrap = (known after apply)
      + env       = (known after apply)
      + net       = (known after apply)
      + org       = (known after apply)
      + proj      = (known after apply)
    }
  + gcs_bucket_cloudbuild_logs                        = {
      + bootstrap = (known after apply)
      + env       = (known after apply)
      + net       = (known after apply)
      + org       = (known after apply)
      + proj      = (known after apply)
    }

────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

Saved the plan to: bootstrap.tfplan

To perform exactly these actions, run the following command to apply:
    terraform apply "bootstrap.tfplan"
michael@cloudshell:~/tef-oldev3/pbmm-on-gcp-onboarding/0-bootstrap (tef-oldev)$ terraform plan -input=false -out bootstrap.tfplan

michael@cloudshell:~/tef-oldev3/pbmm-on-gcp-onboarding/0-bootstrap (tef-oldev)$ terraform apply bootstrap.tfplan 
module.tf_private_pool.google_cloudbuild_worker_pool.private_pool: Creating...
module.tf_private_pool.google_cloudbuild_worker_pool.private_pool: Still creating... [10s elapsed]
module.tf_private_pool.google_cloudbuild_worker_pool.private_pool: Still creating... [20s elapsed]

module.tf_private_pool.google_cloudbuild_worker_pool.private_pool: Still creating... [30s elapsed]
module.tf_private_pool.google_cloudbuild_worker_pool.private_pool: Still creating... [40s elapsed]
module.tf_private_pool.google_cloudbuild_worker_pool.private_pool: Still creating... [50s elapsed]
module.tf_private_pool.google_cloudbuild_worker_pool.private_pool: Still creating... [1m0s elapsed]
module.tf_private_pool.google_cloudbuild_worker_pool.private_pool: Creation complete after 1m2s [id=projects/prj-b-cicd-fgbs/locations/northamerica-northeast1/workerPools/private-pool-ymbi]
module.tf_cloud_builder.google_cloudbuild_trigger.build_trigger: Creating...
module.tf_cloud_builder.google_cloudbuild_trigger.build_trigger: Creation complete after 0s [id=projects/prj-b-cicd-fgbs/locations/northamerica-northeast1/triggers/eb087ec7-a803-4058-b0d4-e467b689f7e7]
time_sleep.cloud_builder: Creating...
module.tf_workspace["bootstrap"].data.google_project.cloudbuild_project[0]: Reading...
module.tf_workspace["net"].data.google_project.cloudbuild_project[0]: Reading...
module.tf_workspace["org"].data.google_project.cloudbuild_project[0]: Reading...
module.tf_workspace["proj"].data.google_project.cloudbuild_project[0]: Reading...
module.tf_workspace["env"].data.google_project.cloudbuild_project[0]: Reading...
module.tf_workspace["net"].google_service_account_iam_member.cb_sa_self["roles/iam.serviceAccountUser"]: Creating...
module.tf_workspace["net"].google_project_iam_member.cb_sa_logging: Creating...
module.tf_workspace["net"].google_storage_bucket_iam_member.state_admin: Creating...
module.tf_workspace["org"].google_storage_bucket_iam_member.state_admin: Creating...
module.tf_workspace["org"].data.google_project.cloudbuild_project[0]: Read complete after 0s [id=projects/prj-b-cicd-fgbs]
module.tf_workspace["env"].data.google_project.cloudbuild_project[0]: Read complete after 0s [id=projects/prj-b-cicd-fgbs]
module.tf_workspace["proj"].data.google_project.cloudbuild_project[0]: Read complete after 0s [id=projects/prj-b-cicd-fgbs]
module.tf_workspace["bootstrap"].data.google_project.cloudbuild_project[0]: Read complete after 0s [id=projects/prj-b-cicd-fgbs]
module.tf_workspace["net"].data.google_project.cloudbuild_project[0]: Read complete after 0s [id=projects/prj-b-cicd-fgbs]
module.tf_workspace["org"].google_service_account_iam_member.cb_sa_self["roles/iam.serviceAccountTokenCreator"]: Creating...
module.tf_workspace["proj"].google_service_account_iam_member.cb_sa_self["roles/iam.serviceAccountTokenCreator"]: Creating...
module.tf_workspace["proj"].google_service_account_iam_member.cb_sa_self["roles/iam.serviceAccountUser"]: Creating...
module.tf_workspace["bootstrap"].google_service_account_iam_member.cb_sa_self["roles/iam.serviceAccountTokenCreator"]: Creating...
module.tf_workspace["net"].google_service_account_iam_member.cb_sa_self["roles/iam.serviceAccountTokenCreator"]: Creating...
module.tf_workspace["proj"].google_service_account_iam_member.cb_sa_self["roles/iam.serviceAccountTokenCreator"]: Creation complete after 4s [id=projects/prj-b-seed-cf20/serviceAccounts/sa-terraform-proj@prj-b-seed-cf20.iam.gserviceaccount.com/roles/iam.serviceAccountTokenCreator/serviceAccount:sa-terraform-proj@prj-b-seed-cf20.iam.gserviceaccount.com]
module.tf_workspace["org"].google_service_account_iam_member.cb_sa_self["roles/iam.serviceAccountTokenCreator"]: Creation complete after 4s [id=projects/prj-b-seed-cf20/serviceAccounts/sa-terraform-org@prj-b-seed-cf20.iam.gserviceaccount.com/roles/iam.serviceAccountTokenCreator/serviceAccount:sa-terraform-org@prj-b-seed-cf20.iam.gserviceaccount.com]
module.tf_workspace["bootstrap"].google_service_account_iam_member.cb_sa_self["roles/iam.serviceAccountTokenCreator"]: Creation complete after 4s [id=projects/prj-b-seed-cf20/serviceAccounts/sa-terraform-bootstrap@prj-b-seed-cf20.iam.gserviceaccount.com/roles/iam.serviceAccountTokenCreator/serviceAccount:sa-terraform-bootstrap@prj-b-seed-cf20.iam.gserviceaccount.com]
module.tf_workspace["proj"].google_storage_bucket_iam_member.state_admin: Creating...
module.tf_workspace["env"].google_storage_bucket_iam_member.state_admin: Creating...
module.tf_workspace["bootstrap"].google_service_account_iam_member.cb_sa_self["roles/iam.serviceAccountUser"]: Creating...

module.tf_workspace["net"].google_project_iam_member.cb_sa_logging: Creation complete after 8s [id=prj-b-cicd-fgbs/roles/logging.logWriter/serviceAccount:sa-terraform-net@prj-b-seed-cf20.iam.gserviceaccount.com]
module.tf_workspace["bootstrap"].google_project_iam_member.cb_sa_logging: Creating...
module.tf_workspace["net"].google_service_account_iam_member.cb_sa_self["roles/iam.serviceAccountTokenCreator"]: Creation complete after 7s [id=projects/prj-b-seed-cf20/serviceAccounts/sa-terraform-net@prj-b-seed-cf20.iam.gserviceaccount.com/roles/iam.serviceAccountTokenCreator/serviceAccount:sa-terraform-net@prj-b-seed-cf20.iam.gserviceaccount.com]
module.tf_workspace["net"].google_service_account_iam_member.cb_sa_self["roles/iam.serviceAccountUser"]: Creation complete after 8s [id=projects/prj-b-seed-cf20/serviceAccounts/sa-terraform-net@prj-b-seed-cf20.iam.gserviceaccount.com/roles/iam.serviceAccountUser/serviceAccount:sa-terraform-net@prj-b-seed-cf20.iam.gserviceaccount.com]
module.tf_workspace["org"].google_project_iam_member.cb_sa_logging: Creating...
module.tf_workspace["org"].google_storage_bucket_iam_member.state_admin: Creation complete after 8s [id=b/bkt-prj-b-seed-tfstate-8bea/roles/storage.admin/serviceAccount:sa-terraform-org@prj-b-seed-cf20.iam.gserviceaccount.com]
module.tf_workspace["net"].google_storage_bucket_iam_member.state_admin: Creation complete after 8s [id=b/bkt-prj-b-seed-tfstate-8bea/roles/storage.admin/serviceAccount:sa-terraform-net@prj-b-seed-cf20.iam.gserviceaccount.com]
module.tf_workspace["env"].google_service_account_iam_member.cb_sa_self["roles/iam.serviceAccountTokenCreator"]: Creating...
module.tf_workspace["env"].google_service_account_iam_member.cb_sa_self["roles/iam.serviceAccountUser"]: Creating...
module.tf_workspace["org"].google_service_account_iam_member.cb_sa_self["roles/iam.serviceAccountUser"]: Creating...
module.tf_workspace["proj"].google_service_account_iam_member.cb_sa_self["roles/iam.serviceAccountUser"]: Creation complete after 8s [id=projects/prj-b-seed-cf20/serviceAccounts/sa-terraform-proj@prj-b-seed-cf20.iam.gserviceaccount.com/roles/iam.serviceAccountUser/serviceAccount:sa-terraform-proj@prj-b-seed-cf20.iam.gserviceaccount.com]
module.tf_workspace["env"].google_project_iam_member.cb_sa_logging: Creating...
module.tf_workspace["bootstrap"].google_service_account_iam_member.cb_sa_self["roles/iam.serviceAccountUser"]: Creation complete after 4s [id=projects/prj-b-seed-cf20/serviceAccounts/sa-terraform-bootstrap@prj-b-seed-cf20.iam.gserviceaccount.com/roles/iam.serviceAccountUser/serviceAccount:sa-terraform-bootstrap@prj-b-seed-cf20.iam.gserviceaccount.com]
module.tf_workspace["proj"].google_storage_bucket_iam_member.state_admin: Creation complete after 4s [id=b/bkt-prj-b-seed-cf20-gcp-projects-tfstate/roles/storage.admin/serviceAccount:sa-terraform-proj@prj-b-seed-cf20.iam.gserviceaccount.com]
module.tf_workspace["bootstrap"].google_storage_bucket_iam_member.state_admin: Creating...
module.tf_workspace["proj"].google_project_iam_member.cb_sa_logging: Creating...
time_sleep.cloud_builder: Still creating... [10s elapsed]
module.tf_workspace["org"].google_service_account_iam_member.cb_sa_self["roles/iam.serviceAccountUser"]: Creation complete after 4s [id=projects/prj-b-seed-cf20/serviceAccounts/sa-terraform-org@prj-b-seed-cf20.iam.gserviceaccount.com/roles/iam.serviceAccountUser/serviceAccount:sa-terraform-org@prj-b-seed-cf20.iam.gserviceaccount.com]
module.tf_workspace["env"].google_service_account_iam_member.cb_sa_self["roles/iam.serviceAccountUser"]: Creation complete after 4s [id=projects/prj-b-seed-cf20/serviceAccounts/sa-terraform-env@prj-b-seed-cf20.iam.gserviceaccount.com/roles/iam.serviceAccountUser/serviceAccount:sa-terraform-env@prj-b-seed-cf20.iam.gserviceaccount.com]
module.tf_workspace["env"].google_sourcerepo_repository_iam_member.member[0]: Creating...
module.tf_workspace["proj"].google_sourcerepo_repository_iam_member.member[0]: Creating...
module.tf_workspace["env"].google_storage_bucket_iam_member.state_admin: Still creating... [10s elapsed]
module.tf_workspace["proj"].google_project_iam_member.cb_sa_logging: Creation complete after 6s [id=prj-b-cicd-fgbs/roles/logging.logWriter/serviceAccount:sa-terraform-proj@prj-b-seed-cf20.iam.gserviceaccount.com]
module.tf_workspace["org"].google_project_iam_member.cb_sa_logging: Creation complete after 7s [id=prj-b-cicd-fgbs/roles/logging.logWriter/serviceAccount:sa-terraform-org@prj-b-seed-cf20.iam.gserviceaccount.com]
module.tf_workspace["org"].google_sourcerepo_repository_iam_member.member[0]: Creating...
module.tf_workspace["bootstrap"].google_project_iam_member.cb_sa_logging: Creation complete after 7s [id=prj-b-cicd-fgbs/roles/logging.logWriter/serviceAccount:sa-terraform-bootstrap@prj-b-seed-cf20.iam.gserviceaccount.com]
module.tf_workspace["bootstrap"].google_sourcerepo_repository_iam_member.member[0]: Creating...
module.tf_workspace["env"].google_project_iam_member.cb_sa_logging: Creation complete after 7s [id=prj-b-cicd-fgbs/roles/logging.logWriter/serviceAccount:sa-terraform-env@prj-b-seed-cf20.iam.gserviceaccount.com]
module.tf_workspace["net"].google_sourcerepo_repository_iam_member.member[0]: Creating...
module.tf_workspace["org"].google_project_iam_member.pool_user[0]: Creating...
module.tf_workspace["env"].google_service_account_iam_member.cb_sa_self["roles/iam.serviceAccountTokenCreator"]: Creation complete after 8s [id=projects/prj-b-seed-cf20/serviceAccounts/sa-terraform-env@prj-b-seed-cf20.iam.gserviceaccount.com/roles/iam.serviceAccountTokenCreator/serviceAccount:sa-terraform-env@prj-b-seed-cf20.iam.gserviceaccount.com]
module.tf_workspace["env"].google_storage_bucket_iam_member.state_admin: Creation complete after 11s [id=b/bkt-prj-b-seed-tfstate-8bea/roles/storage.admin/serviceAccount:sa-terraform-env@prj-b-seed-cf20.iam.gserviceaccount.com]
module.tf_workspace["org"].google_service_account_iam_member.cb_service_agent_impersonate[0]: Creating...
module.tf_workspace["bootstrap"].google_storage_bucket_iam_member.state_admin: Creation complete after 7s [id=b/bkt-prj-b-seed-tfstate-8bea/roles/storage.admin/serviceAccount:sa-terraform-bootstrap@prj-b-seed-cf20.iam.gserviceaccount.com]
module.tf_workspace["env"].google_service_account_iam_member.cb_service_agent_impersonate[0]: Creating...
module.tf_workspace["env"].google_project_iam_member.pool_user[0]: Creating...
module.tf_workspace["proj"].google_sourcerepo_repository_iam_member.member[0]: Creation complete after 4s [id=projects/prj-b-cicd-fgbs/repos/gcp-projects/roles/viewer/serviceAccount:sa-terraform-proj@prj-b-seed-cf20.iam.gserviceaccount.com]
module.tf_workspace["proj"].google_project_iam_member.pool_user[0]: Creating...
module.tf_workspace["env"].google_sourcerepo_repository_iam_member.member[0]: Creation complete after 5s [id=projects/prj-b-cicd-fgbs/repos/gcp-environments/roles/viewer/serviceAccount:sa-terraform-env@prj-b-seed-cf20.iam.gserviceaccount.com]
module.tf_workspace["proj"].google_service_account_iam_member.cb_service_agent_impersonate[0]: Creating...

module.tf_workspace["net"].google_project_iam_member.cb_sa_logging: Creation complete after 8s [id=prj-b-cicd-fgbs/roles/logging.logWriter/serviceAccount:sa-terraform-net@prj-b-seed-cf20.iam.gserviceaccount.com]
module.tf_workspace["bootstrap"].google_project_iam_member.cb_sa_logging: Creating...
module.tf_workspace["net"].google_service_account_iam_member.cb_sa_self["roles/iam.serviceAccountTokenCreator"]: Creation complete after 7s [id=projects/prj-b-seed-cf20/serviceAccounts/sa-terraform-net@prj-b-seed-cf20.iam.gserviceaccount.com/roles/iam.serviceAccountTokenCreator/serviceAccount:sa-terraform-net@prj-b-seed-cf20.iam.gserviceaccount.com]
module.tf_workspace["net"].google_service_account_iam_member.cb_sa_self["roles/iam.serviceAccountUser"]: Creation complete after 8s [id=projects/prj-b-seed-cf20/serviceAccounts/sa-terraform-net@prj-b-seed-cf20.iam.gserviceaccount.com/roles/iam.serviceAccountUser/serviceAccount:sa-terraform-net@prj-b-seed-cf20.iam.gserviceaccount.com]
module.tf_workspace["org"].google_project_iam_member.cb_sa_logging: Creating...
module.tf_workspace["org"].google_storage_bucket_iam_member.state_admin: Creation complete after 8s [id=b/bkt-prj-b-seed-tfstate-8bea/roles/storage.admin/serviceAccount:sa-terraform-org@prj-b-seed-cf20.iam.gserviceaccount.com]
module.tf_workspace["net"].google_storage_bucket_iam_member.state_admin: Creation complete after 8s [id=b/bkt-prj-b-seed-tfstate-8bea/roles/storage.admin/serviceAccount:sa-terraform-net@prj-b-seed-cf20.iam.gserviceaccount.com]
module.tf_workspace["env"].google_service_account_iam_member.cb_sa_self["roles/iam.serviceAccountTokenCreator"]: Creating...
module.tf_workspace["env"].google_service_account_iam_member.cb_sa_self["roles/iam.serviceAccountUser"]: Creating...
module.tf_workspace["org"].google_service_account_iam_member.cb_sa_self["roles/iam.serviceAccountUser"]: Creating...
module.tf_workspace["proj"].google_service_account_iam_member.cb_sa_self["roles/iam.serviceAccountUser"]: Creation complete after 8s [id=projects/prj-b-seed-cf20/serviceAccounts/sa-terraform-proj@prj-b-seed-cf20.iam.gserviceaccount.com/roles/iam.serviceAccountUser/serviceAccount:sa-terraform-proj@prj-b-seed-cf20.iam.gserviceaccount.com]
module.tf_workspace["env"].google_project_iam_member.cb_sa_logging: Creating...
module.tf_workspace["bootstrap"].google_service_account_iam_member.cb_sa_self["roles/iam.serviceAccountUser"]: Creation complete after 4s [id=projects/prj-b-seed-cf20/serviceAccounts/sa-terraform-bootstrap@prj-b-seed-cf20.iam.gserviceaccount.com/roles/iam.serviceAccountUser/serviceAccount:sa-terraform-bootstrap@prj-b-seed-cf20.iam.gserviceaccount.com]
module.tf_workspace["proj"].google_storage_bucket_iam_member.state_admin: Creation complete after 4s [id=b/bkt-prj-b-seed-cf20-gcp-projects-tfstate/roles/storage.admin/serviceAccount:sa-terraform-proj@prj-b-seed-cf20.iam.gserviceaccount.com]
module.tf_workspace["bootstrap"].google_storage_bucket_iam_member.state_admin: Creating...
module.tf_workspace["proj"].google_project_iam_member.cb_sa_logging: Creating...
time_sleep.cloud_builder: Still creating... [10s elapsed]
module.tf_workspace["org"].google_service_account_iam_member.cb_sa_self["roles/iam.serviceAccountUser"]: Creation complete after 4s [id=projects/prj-b-seed-cf20/serviceAccounts/sa-terraform-org@prj-b-seed-cf20.iam.gserviceaccount.com/roles/iam.serviceAccountUser/serviceAccount:sa-terraform-org@prj-b-seed-cf20.iam.gserviceaccount.com]
module.tf_workspace["env"].google_service_account_iam_member.cb_sa_self["roles/iam.serviceAccountUser"]: Creation complete after 4s [id=projects/prj-b-seed-cf20/serviceAccounts/sa-terraform-env@prj-b-seed-cf20.iam.gserviceaccount.com/roles/iam.serviceAccountUser/serviceAccount:sa-terraform-env@prj-b-seed-cf20.iam.gserviceaccount.com]
module.tf_workspace["env"].google_sourcerepo_repository_iam_member.member[0]: Creating...
module.tf_workspace["proj"].google_sourcerepo_repository_iam_member.member[0]: Creating...
module.tf_workspace["env"].google_storage_bucket_iam_member.state_admin: Still creating... [10s elapsed]
module.tf_workspace["proj"].google_project_iam_member.cb_sa_logging: Creation complete after 6s [id=prj-b-cicd-fgbs/roles/logging.logWriter/serviceAccount:sa-terraform-proj@prj-b-seed-cf20.iam.gserviceaccount.com]
module.tf_workspace["org"].google_project_iam_member.cb_sa_logging: Creation complete after 7s [id=prj-b-cicd-fgbs/roles/logging.logWriter/serviceAccount:sa-terraform-org@prj-b-seed-cf20.iam.gserviceaccount.com]
module.tf_workspace["org"].google_sourcerepo_repository_iam_member.member[0]: Creating...
module.tf_workspace["bootstrap"].google_project_iam_member.cb_sa_logging: Creation complete after 7s [id=prj-b-cicd-fgbs/roles/logging.logWriter/serviceAccount:sa-terraform-bootstrap@prj-b-seed-cf20.iam.gserviceaccount.com]
module.tf_workspace["bootstrap"].google_sourcerepo_repository_iam_member.member[0]: Creating...
module.tf_workspace["env"].google_project_iam_member.cb_sa_logging: Creation complete after 7s [id=prj-b-cicd-fgbs/roles/logging.logWriter/serviceAccount:sa-terraform-env@prj-b-seed-cf20.iam.gserviceaccount.com]
module.tf_workspace["net"].google_sourcerepo_repository_iam_member.member[0]: Creating...
module.tf_workspace["org"].google_project_iam_member.pool_user[0]: Creating...
module.tf_workspace["env"].google_service_account_iam_member.cb_sa_self["roles/iam.serviceAccountTokenCreator"]: Creation complete after 8s [id=projects/prj-b-seed-cf20/serviceAccounts/sa-terraform-env@prj-b-seed-cf20.iam.gserviceaccount.com/roles/iam.serviceAccountTokenCreator/serviceAccount:sa-terraform-env@prj-b-seed-cf20.iam.gserviceaccount.com]
module.tf_workspace["env"].google_storage_bucket_iam_member.state_admin: Creation complete after 11s [id=b/bkt-prj-b-seed-tfstate-8bea/roles/storage.admin/serviceAccount:sa-terraform-env@prj-b-seed-cf20.iam.gserviceaccount.com]
module.tf_workspace["org"].google_service_account_iam_member.cb_service_agent_impersonate[0]: Creating...
module.tf_workspace["bootstrap"].google_storage_bucket_iam_member.state_admin: Creation complete after 7s [id=b/bkt-prj-b-seed-tfstate-8bea/roles/storage.admin/serviceAccount:sa-terraform-bootstrap@prj-b-seed-cf20.iam.gserviceaccount.com]
module.tf_workspace["env"].google_service_account_iam_member.cb_service_agent_impersonate[0]: Creating...
module.tf_workspace["env"].google_project_iam_member.pool_user[0]: Creating...
module.tf_workspace["proj"].google_sourcerepo_repository_iam_member.member[0]: Creation complete after 4s [id=projects/prj-b-cicd-fgbs/repos/gcp-projects/roles/viewer/serviceAccount:sa-terraform-proj@prj-b-seed-cf20.iam.gserviceaccount.com]
module.tf_workspace["proj"].google_project_iam_member.pool_user[0]: Creating...
module.tf_workspace["env"].google_sourcerepo_repository_iam_member.member[0]: Creation complete after 5s [id=projects/prj-b-cicd-fgbs/repos/gcp-environments/roles/viewer/serviceAccount:sa-terraform-env@prj-b-seed-cf20.iam.gserviceaccount.com]
module.tf_workspace["proj"].google_service_account_iam_member.cb_service_agent_impersonate[0]: Creating...

module.tf_workspace["bootstrap"].google_sourcerepo_repository_iam_member.member[0]: Creation complete after 5s [id=projects/prj-b-cicd-fgbs/repos/gcp-bootstrap/roles/viewer/serviceAccount:sa-terraform-bootstrap@prj-b-seed-cf20.iam.gserviceaccount.com]
module.tf_workspace["org"].google_sourcerepo_repository_iam_member.member[0]: Creation complete after 5s [id=projects/prj-b-cicd-fgbs/repos/gcp-org/roles/viewer/serviceAccount:sa-terraform-org@prj-b-seed-cf20.iam.gserviceaccount.com]
module.tf_workspace["bootstrap"].google_service_account_iam_member.cb_service_agent_impersonate[0]: Creating...
module.tf_workspace["net"].google_sourcerepo_repository_iam_member.member[0]: Creation complete after 4s [id=projects/prj-b-cicd-fgbs/repos/gcp-networks/roles/viewer/serviceAccount:sa-terraform-net@prj-b-seed-cf20.iam.gserviceaccount.com]
module.tf_workspace["bootstrap"].google_project_iam_member.pool_user[0]: Creating...
module.tf_workspace["env"].google_service_account_iam_member.cb_service_agent_impersonate[0]: Creation complete after 4s [id=projects/prj-b-seed-cf20/serviceAccounts/sa-terraform-env@prj-b-seed-cf20.iam.gserviceaccount.com/roles/iam.serviceAccountTokenCreator/serviceAccount:service-991985440030@gcp-sa-cloudbuild.iam.gserviceaccount.com]
module.tf_workspace["net"].google_service_account_iam_member.cb_service_agent_impersonate[0]: Creating...
module.tf_workspace["org"].google_service_account_iam_member.cb_service_agent_impersonate[0]: Creation complete after 4s [id=projects/prj-b-seed-cf20/serviceAccounts/sa-terraform-org@prj-b-seed-cf20.iam.gserviceaccount.com/roles/iam.serviceAccountTokenCreator/serviceAccount:service-991985440030@gcp-sa-cloudbuild.iam.gserviceaccount.com]
time_sleep.cloud_builder: Still creating... [20s elapsed]
module.tf_workspace["net"].google_project_iam_member.pool_user[0]: Creating...
module.tf_workspace["env"].module.log_bucket.google_storage_bucket.bucket: Creating...
module.tf_workspace["proj"].google_service_account_iam_member.cb_service_agent_impersonate[0]: Creation complete after 4s [id=projects/prj-b-seed-cf20/serviceAccounts/sa-terraform-proj@prj-b-seed-cf20.iam.gserviceaccount.com/roles/iam.serviceAccountTokenCreator/serviceAccount:service-991985440030@gcp-sa-cloudbuild.iam.gserviceaccount.com]
module.tf_workspace["bootstrap"].module.log_bucket.google_storage_bucket.bucket: Creating...
module.tf_workspace["env"].module.log_bucket.google_storage_bucket.bucket: Creation complete after 1s [id=bkt-prj-b-cicd-fgbs-gcp-environments-build-logs]
module.tf_workspace["org"].module.log_bucket.google_storage_bucket.bucket: Creating...
module.tf_workspace["bootstrap"].module.log_bucket.google_storage_bucket.bucket: Creation complete after 1s [id=bkt-prj-b-cicd-fgbs-gcp-bootstrap-build-logs]
module.tf_workspace["net"].module.log_bucket.google_storage_bucket.bucket: Creating...
module.tf_workspace["org"].module.log_bucket.google_storage_bucket.bucket: Creation complete after 1s [id=bkt-prj-b-cicd-fgbs-gcp-org-build-logs]
module.tf_workspace["proj"].module.log_bucket.google_storage_bucket.bucket: Creating...
module.tf_workspace["net"].module.log_bucket.google_storage_bucket.bucket: Creation complete after 1s [id=bkt-prj-b-cicd-fgbs-gcp-networks-build-logs]
module.tf_workspace["net"].module.artifacts_bucket.google_storage_bucket.bucket: Creating...
module.tf_workspace["proj"].module.log_bucket.google_storage_bucket.bucket: Creation complete after 1s [id=bkt-prj-b-cicd-fgbs-gcp-projects-build-logs]
module.tf_workspace["proj"].module.artifacts_bucket.google_storage_bucket.bucket: Creating...
module.tf_workspace["net"].module.artifacts_bucket.google_storage_bucket.bucket: Creation complete after 1s [id=bkt-prj-b-cicd-fgbs-gcp-networks-build-artifacts]
module.tf_workspace["bootstrap"].google_service_account_iam_member.cb_service_agent_impersonate[0]: Creation complete after 4s [id=projects/prj-b-seed-cf20/serviceAccounts/sa-terraform-bootstrap@prj-b-seed-cf20.iam.gserviceaccount.com/roles/iam.serviceAccountTokenCreator/serviceAccount:service-991985440030@gcp-sa-cloudbuild.iam.gserviceaccount.com]
module.tf_workspace["env"].module.artifacts_bucket.google_storage_bucket.bucket: Creating...
module.tf_workspace["proj"].module.artifacts_bucket.google_storage_bucket.bucket: Creation complete after 1s [id=bkt-prj-b-cicd-fgbs-gcp-projects-build-artifacts]
module.tf_workspace["bootstrap"].module.artifacts_bucket.google_storage_bucket.bucket: Creating...
module.tf_workspace["net"].google_service_account_iam_member.cb_service_agent_impersonate[0]: Creation complete after 4s [id=projects/prj-b-seed-cf20/serviceAccounts/sa-terraform-net@prj-b-seed-cf20.iam.gserviceaccount.com/roles/iam.serviceAccountTokenCreator/serviceAccount:service-991985440030@gcp-sa-cloudbuild.iam.gserviceaccount.com]
module.tf_workspace["org"].module.artifacts_bucket.google_storage_bucket.bucket: Creating...
module.tf_workspace["env"].google_storage_bucket_iam_member.log_admin: Creating...
module.tf_workspace["env"].module.artifacts_bucket.google_storage_bucket.bucket: Creation complete after 1s [id=bkt-prj-b-cicd-fgbs-gcp-environments-build-artifacts]
module.tf_workspace["bootstrap"].module.artifacts_bucket.google_storage_bucket.bucket: Creation complete after 1s [id=bkt-prj-b-cicd-fgbs-gcp-bootstrap-build-artifacts]
module.tf_workspace["bootstrap"].google_storage_bucket_iam_member.log_admin: Creating...
module.tf_workspace["net"].google_storage_bucket_iam_member.log_admin: Creating...
module.tf_workspace["org"].google_project_iam_member.pool_user[0]: Still creating... [10s elapsed]
module.tf_workspace["env"].google_project_iam_member.pool_user[0]: Still creating... [10s elapsed]
module.tf_workspace["org"].module.artifacts_bucket.google_storage_bucket.bucket: Creation complete after 2s [id=bkt-prj-b-cicd-fgbs-gcp-org-build-artifacts]
module.tf_workspace["proj"].google_storage_bucket_iam_member.log_admin: Creating...
module.tf_workspace["proj"].google_project_iam_member.pool_user[0]: Still creating... [10s elapsed]
module.tf_workspace["proj"].google_project_iam_member.pool_user[0]: Creation complete after 11s [id=prj-b-cicd-fgbs/roles/cloudbuild.workerPoolUser/serviceAccount:991985440030@cloudbuild.gserviceaccount.com]
module.tf_workspace["org"].google_storage_bucket_iam_member.log_admin: Creating...
module.tf_workspace["org"].google_project_iam_member.pool_user[0]: Creation complete after 12s [id=prj-b-cicd-fgbs/roles/cloudbuild.workerPoolUser/serviceAccount:991985440030@cloudbuild.gserviceaccount.com]
module.tf_workspace["net"].google_storage_bucket_iam_member.artifacts_admin: Creating...
module.tf_workspace["env"].google_project_iam_member.pool_user[0]: Creation complete after 12s [id=prj-b-cicd-fgbs/roles/cloudbuild.workerPoolUser/serviceAccount:991985440030@cloudbuild.gserviceaccount.com]
module.tf_workspace["env"].google_storage_bucket_iam_member.artifacts_admin: Creating...
module.tf_workspace["env"].google_storage_bucket_iam_member.log_admin: Creation complete after 4s [id=b/bkt-prj-b-cicd-fgbs-gcp-environments-build-logs/roles/storage.admin/serviceAccount:sa-terraform-env@prj-b-seed-cf20.iam.gserviceaccount.com]
module.tf_workspace["net"].google_project_iam_member.pool_user[0]: Creation complete after 8s [id=prj-b-cicd-fgbs/roles/cloudbuild.workerPoolUser/serviceAccount:991985440030@cloudbuild.gserviceaccount.com]
module.tf_workspace["bootstrap"].google_storage_bucket_iam_member.artifacts_admin: Creating...
module.tf_workspace["bootstrap"].google_project_iam_member.pool_user[0]: Creation complete after 8s [id=prj-b-cicd-fgbs/roles/cloudbuild.workerPoolUser/serviceAccount:991985440030@cloudbuild.gserviceaccount.com]
module.tf_workspace["proj"].google_storage_bucket_iam_member.artifacts_admin: Creating...
module.tf_workspace["org"].google_storage_bucket_iam_member.artifacts_admin: Creating...
module.tf_workspace["net"].google_storage_bucket_iam_member.log_admin: Creation complete after 4s [id=b/bkt-prj-b-cicd-fgbs-gcp-networks-build-logs/roles/storage.admin/serviceAccount:sa-terraform-net@prj-b-seed-cf20.iam.gserviceaccount.com]
module.tf_workspace["bootstrap"].google_cloudbuild_trigger.triggers["apply"]: Creating...
module.tf_workspace["bootstrap"].google_storage_bucket_iam_member.log_admin: Creation complete after 4s [id=b/bkt-prj-b-cicd-fgbs-gcp-bootstrap-build-logs/roles/storage.admin/serviceAccount:sa-terraform-bootstrap@prj-b-seed-cf20.iam.gserviceaccount.com]
module.tf_workspace["proj"].google_cloudbuild_trigger.triggers["apply"]: Creating...
module.tf_workspace["bootstrap"].google_cloudbuild_trigger.triggers["apply"]: Creation complete after 0s [id=projects/prj-b-cicd-fgbs/locations/northamerica-northeast1/triggers/60451a79-8743-444a-ac0d-df43a062dc47]
module.tf_workspace["proj"].google_cloudbuild_trigger.triggers["apply"]: Creation complete after 0s [id=projects/prj-b-cicd-fgbs/locations/northamerica-northeast1/triggers/5d37e7bc-798b-4308-ba8f-5c528de1bc26]
module.tf_workspace["bootstrap"].google_cloudbuild_trigger.triggers["plan"]: Creating...
module.tf_workspace["net"].google_cloudbuild_trigger.triggers["plan"]: Creating...
module.tf_workspace["net"].google_cloudbuild_trigger.triggers["plan"]: Creation complete after 0s [id=projects/prj-b-cicd-fgbs/locations/northamerica-northeast1/triggers/e9ff8b36-6eae-4cf3-b0a9-cf4ca4c53a25]
module.tf_workspace["bootstrap"].google_cloudbuild_trigger.triggers["plan"]: Creation complete after 0s [id=projects/prj-b-cicd-fgbs/locations/northamerica-northeast1/triggers/dfac23ee-a1bc-49bb-abc0-b45d039b2a10]
module.tf_workspace["env"].google_cloudbuild_trigger.triggers["plan"]: Creating...
time_sleep.cloud_builder: Still creating... [30s elapsed]
time_sleep.cloud_builder: Creation complete after 30s [id=2024-05-14T14:04:53Z]
module.tf_workspace["net"].google_cloudbuild_trigger.triggers["apply"]: Creating...
module.tf_workspace["org"].google_cloudbuild_trigger.triggers["apply"]: Creating...
module.tf_workspace["env"].google_cloudbuild_trigger.triggers["plan"]: Creation complete after 0s [id=projects/prj-b-cicd-fgbs/locations/northamerica-northeast1/triggers/d1e4b14c-640f-4ea7-8da3-782729fda2ea]
module.tf_workspace["proj"].google_storage_bucket_iam_member.log_admin: Creation complete after 5s [id=b/bkt-prj-b-cicd-fgbs-gcp-projects-build-logs/roles/storage.admin/serviceAccount:sa-terraform-proj@prj-b-seed-cf20.iam.gserviceaccount.com]
module.tf_workspace["env"].google_cloudbuild_trigger.triggers["apply"]: Creating...
module.tf_workspace["org"].google_cloudbuild_trigger.triggers["apply"]: Creation complete after 1s [id=projects/prj-b-cicd-fgbs/locations/northamerica-northeast1/triggers/d80f6aae-68fd-4a58-8fc7-944f883baf6f]
module.tf_workspace["net"].google_cloudbuild_trigger.triggers["apply"]: Creation complete after 1s [id=projects/prj-b-cicd-fgbs/locations/northamerica-northeast1/triggers/7ec30681-8994-47be-849f-4eb7168d40d8]
module.tf_workspace["proj"].google_cloudbuild_trigger.triggers["plan"]: Creating...
module.tf_workspace["env"].google_cloudbuild_trigger.triggers["apply"]: Creation complete after 0s [id=projects/prj-b-cicd-fgbs/locations/northamerica-northeast1/triggers/5810239d-0cfa-4e19-9832-7554ae2a8fa4]
module.build_terraform_image.null_resource.module_depends_on[0]: Creating...
module.tf_workspace["org"].google_cloudbuild_trigger.triggers["plan"]: Creating...
module.build_terraform_image.null_resource.module_depends_on[0]: Creation complete after 0s [id=1319307347153994581]
module.tf_workspace["proj"].google_cloudbuild_trigger.triggers["plan"]: Creation complete after 0s [id=projects/prj-b-cicd-fgbs/locations/northamerica-northeast1/triggers/b2c171ff-0dc6-435b-92df-059c73c6040f]
module.tf_workspace["org"].google_cloudbuild_trigger.triggers["plan"]: Creation complete after 0s [id=projects/prj-b-cicd-fgbs/locations/northamerica-northeast1/triggers/03bc6aa1-8ba1-4949-a32a-d53e3619eb76]
module.build_terraform_image.null_resource.run_command[0]: Creating...
module.build_terraform_image.null_resource.run_destroy_command[0]: Creating...
module.build_terraform_image.null_resource.run_destroy_command[0]: Creation complete after 0s [id=3198878574336107300]
module.build_terraform_image.null_resource.run_command[0]: Provisioning with 'local-exec'...
module.build_terraform_image.null_resource.run_command[0] (local-exec): Executing: ["/bin/sh" "-c" "PATH=/google-cloud-sdk/bin:$PATH\ngcloud beta builds triggers run  eb087ec7-a803-4058-b0d4-e467b689f7e7 --branch main --region northamerica-northeast1 --project prj-b-cicd-fgbs\n"]
module.tf_workspace["org"].google_storage_bucket_iam_member.log_admin: Creation complete after 5s [id=b/bkt-prj-b-cicd-fgbs-gcp-org-build-logs/roles/storage.admin/serviceAccount:sa-terraform-org@prj-b-seed-cf20.iam.gserviceaccount.com]
module.tf_workspace["net"].google_storage_bucket_iam_member.artifacts_admin: Creation complete after 4s [id=b/bkt-prj-b-cicd-fgbs-gcp-networks-build-artifacts/roles/storage.admin/serviceAccount:sa-terraform-net@prj-b-seed-cf20.iam.gserviceaccount.com]
module.tf_workspace["env"].google_storage_bucket_iam_member.artifacts_admin: Creation complete after 4s [id=b/bkt-prj-b-cicd-fgbs-gcp-environments-build-artifacts/roles/storage.admin/serviceAccount:sa-terraform-env@prj-b-seed-cf20.iam.gserviceaccount.com]
module.tf_workspace["bootstrap"].google_storage_bucket_iam_member.artifacts_admin: Creation complete after 4s [id=b/bkt-prj-b-cicd-fgbs-gcp-bootstrap-build-artifacts/roles/storage.admin/serviceAccount:sa-terraform-bootstrap@prj-b-seed-cf20.iam.gserviceaccount.com]
module.tf_workspace["org"].google_storage_bucket_iam_member.artifacts_admin: Creation complete after 5s [id=b/bkt-prj-b-cicd-fgbs-gcp-org-build-artifacts/roles/storage.admin/serviceAccount:sa-terraform-org@prj-b-seed-cf20.iam.gserviceaccount.com]
module.tf_workspace["proj"].google_storage_bucket_iam_member.artifacts_admin: Creation complete after 5s [id=b/bkt-prj-b-cicd-fgbs-gcp-projects-build-artifacts/roles/storage.admin/serviceAccount:sa-terraform-proj@prj-b-seed-cf20.iam.gserviceaccount.com]
module.build_terraform_image.null_resource.run_command[0] (local-exec): metadata:
module.build_terraform_image.null_resource.run_command[0] (local-exec):   '@type': type.googleapis.com/google.devtools.cloudbuild.v1.BuildOperationMetadata
module.build_terraform_image.null_resource.run_command[0] (local-exec):   build:
module.build_terraform_image.null_resource.run_command[0] (local-exec):     artifacts:
module.build_terraform_image.null_resource.run_command[0] (local-exec):       images:
module.build_terraform_image.null_resource.run_command[0] (local-exec):       - northamerica-northeast1-docker.pkg.dev/prj-b-cicd-fgbs/tf-runners/terraform:v1.3.0
module.build_terraform_image.null_resource.run_command[0] (local-exec):       - northamerica-northeast1-docker.pkg.dev/prj-b-cicd-fgbs/tf-runners/terraform:v1
module.build_terraform_image.null_resource.run_command[0] (local-exec):       - northamerica-northeast1-docker.pkg.dev/prj-b-cicd-fgbs/tf-runners/terraform:v1.3
module.build_terraform_image.null_resource.run_command[0] (local-exec):     buildTriggerId: eb087ec7-a803-4058-b0d4-e467b689f7e7
module.build_terraform_image.null_resource.run_command[0] (local-exec):     createTime: '2024-05-14T14:04:56.125638Z'
module.build_terraform_image.null_resource.run_command[0] (local-exec):     id: b8fc3169-a4b7-42ac-b756-3e5e9c05f1c6
module.build_terraform_image.null_resource.run_command[0] (local-exec):     images:
module.build_terraform_image.null_resource.run_command[0] (local-exec):     - northamerica-northeast1-docker.pkg.dev/prj-b-cicd-fgbs/tf-runners/terraform:v1.3.0
module.build_terraform_image.null_resource.run_command[0] (local-exec):     - northamerica-northeast1-docker.pkg.dev/prj-b-cicd-fgbs/tf-runners/terraform:v1
module.build_terraform_image.null_resource.run_command[0] (local-exec):     - northamerica-northeast1-docker.pkg.dev/prj-b-cicd-fgbs/tf-runners/terraform:v1.3
module.build_terraform_image.null_resource.run_command[0] (local-exec):     logUrl: https://console.cloud.google.com/cloud-build/builds;region=northamerica-northeast1/b8fc3169-a4b7-42ac-b756-3e5e9c05f1c6?project=991985440030
module.build_terraform_image.null_resource.run_command[0] (local-exec):     logsBucket: gs://bkt-prj-b-cicd-fgbs-tf-cloudbuilder-build-logs
module.build_terraform_image.null_resource.run_command[0] (local-exec):     name: projects/991985440030/locations/northamerica-northeast1/builds/b8fc3169-a4b7-42ac-b756-3e5e9c05f1c6
module.build_terraform_image.null_resource.run_command[0] (local-exec):     options:
module.build_terraform_image.null_resource.run_command[0] (local-exec):       dynamicSubstitutions: true
module.build_terraform_image.null_resource.run_command[0] (local-exec):       logging: LEGACY
module.build_terraform_image.null_resource.run_command[0] (local-exec):       pool:
module.build_terraform_image.null_resource.run_command[0] (local-exec):         name: projects/prj-b-cicd-fgbs/locations/northamerica-northeast1/workerPools/private-pool-ymbi
module.build_terraform_image.null_resource.run_command[0] (local-exec):       substitutionOption: ALLOW_LOOSE
module.build_terraform_image.null_resource.run_command[0] (local-exec):     projectId: prj-b-cicd-fgbs
module.build_terraform_image.null_resource.run_command[0] (local-exec):     queueTtl: 3600s
module.build_terraform_image.null_resource.run_command[0] (local-exec):     serviceAccount: projects/prj-b-cicd-fgbs/serviceAccounts/tf-cb-builder-sa@prj-b-cicd-fgbs.iam.gserviceaccount.com
module.build_terraform_image.null_resource.run_command[0] (local-exec):     source:
module.build_terraform_image.null_resource.run_command[0] (local-exec):       repoSource:
module.build_terraform_image.null_resource.run_command[0] (local-exec):         commitSha: c1c0f290f526108abd98b512bc04093ff0e18684
module.build_terraform_image.null_resource.run_command[0] (local-exec):         projectId: prj-b-cicd-fgbs
module.build_terraform_image.null_resource.run_command[0] (local-exec):         repoName: tf-cloudbuilder
module.build_terraform_image.null_resource.run_command[0] (local-exec):     sourceProvenance:
module.build_terraform_image.null_resource.run_command[0] (local-exec):       resolvedRepoSource:
module.build_terraform_image.null_resource.run_command[0] (local-exec):         commitSha: c1c0f290f526108abd98b512bc04093ff0e18684
module.build_terraform_image.null_resource.run_command[0] (local-exec):         projectId: prj-b-cicd-fgbs
module.build_terraform_image.null_resource.run_command[0] (local-exec):         repoName: tf-cloudbuilder
module.build_terraform_image.null_resource.run_command[0] (local-exec):     status: QUEUED
module.build_terraform_image.null_resource.run_command[0] (local-exec):     steps:
module.build_terraform_image.null_resource.run_command[0] (local-exec):     - args:
module.build_terraform_image.null_resource.run_command[0] (local-exec):       - build
module.build_terraform_image.null_resource.run_command[0] (local-exec):       - --tag=northamerica-northeast1-docker.pkg.dev/prj-b-cicd-fgbs/tf-runners/terraform:v1.3.0
module.build_terraform_image.null_resource.run_command[0] (local-exec):       - --tag=northamerica-northeast1-docker.pkg.dev/prj-b-cicd-fgbs/tf-runners/terraform:v1
module.build_terraform_image.null_resource.run_command[0] (local-exec):       - --tag=northamerica-northeast1-docker.pkg.dev/prj-b-cicd-fgbs/tf-runners/terraform:v1.3
module.build_terraform_image.null_resource.run_command[0] (local-exec):       - --build-arg=TERRAFORM_VERSION=1.3.0
module.build_terraform_image.null_resource.run_command[0] (local-exec):       - .
module.build_terraform_image.null_resource.run_command[0] (local-exec):       name: gcr.io/cloud-builders/docker
module.build_terraform_image.null_resource.run_command[0] (local-exec):     - args:
module.build_terraform_image.null_resource.run_command[0] (local-exec):       - version
module.build_terraform_image.null_resource.run_command[0] (local-exec):       name: northamerica-northeast1-docker.pkg.dev/prj-b-cicd-fgbs/tf-runners/terraform:v1.3.0
module.build_terraform_image.null_resource.run_command[0] (local-exec):     substitutions:
module.build_terraform_image.null_resource.run_command[0] (local-exec):       BRANCH_NAME: main
module.build_terraform_image.null_resource.run_command[0] (local-exec):       COMMIT_SHA: c1c0f290f526108abd98b512bc04093ff0e18684
module.build_terraform_image.null_resource.run_command[0] (local-exec):       REF_NAME: main
module.build_terraform_image.null_resource.run_command[0] (local-exec):       REPO_NAME: tf-cloudbuilder
module.build_terraform_image.null_resource.run_command[0] (local-exec):       REVISION_ID: c1c0f290f526108abd98b512bc04093ff0e18684
module.build_terraform_image.null_resource.run_command[0] (local-exec):       SHORT_SHA: c1c0f29
module.build_terraform_image.null_resource.run_command[0] (local-exec):       TRIGGER_BUILD_CONFIG_PATH: ''
module.build_terraform_image.null_resource.run_command[0] (local-exec):       TRIGGER_NAME: tf-cloud-builder-build
module.build_terraform_image.null_resource.run_command[0] (local-exec):       _TERRAFORM_FULL_VERSION: 1.3.0
module.build_terraform_image.null_resource.run_command[0] (local-exec):       _TERRAFORM_MAJOR_VERSION: '1'
module.build_terraform_image.null_resource.run_command[0] (local-exec):       _TERRAFORM_MINOR_VERSION: '1.3'
module.build_terraform_image.null_resource.run_command[0] (local-exec):     tags:
module.build_terraform_image.null_resource.run_command[0] (local-exec):     - trigger-eb087ec7-a803-4058-b0d4-e467b689f7e7
module.build_terraform_image.null_resource.run_command[0] (local-exec):     timeout: 1200s
module.build_terraform_image.null_resource.run_command[0] (local-exec): name: operations/build/prj-b-cicd-fgbs/YjhmYzMxNjktYTRiNy00MmFjLWI3NTYtM2U1ZTljMDVmMWM2
module.build_terraform_image.null_resource.run_command[0]: Creation complete after 2s [id=2105418832736276878]

Apply complete! Resources: 71 added, 0 changed, 0 destroyed.

Outputs:

bootstrap_step_terraform_service_account_email = "sa-terraform-bootstrap@prj-b-seed-cf20.iam.gserviceaccount.com"
cloud_build_peered_network_id = "projects/prj-b-cicd-fgbs/global/networks/vpc-b-cbpools"
cloud_build_private_worker_pool_id = "projects/prj-b-cicd-fgbs/locations/northamerica-northeast1/workerPools/private-pool-ymbi"
cloud_build_worker_peered_ip_range = "192.168.0.0/24"
cloud_build_worker_range_id = "projects/prj-b-cicd-fgbs/global/addresses/ga-b-cbpools-worker-pool-range"
cloud_builder_artifact_repo = "projects/prj-b-cicd-fgbs/locations/northamerica-northeast1/repositories/tf-runners"
cloudbuild_project_id = "prj-b-cicd-fgbs"
common_config = {
  "billing_account" = "019283-6F1AB5-7AD576"
  "bootstrap_folder_name" = "folders/236258101664"
  "default_region" = "northamerica-northeast1"
  "folder_prefix" = "fldr"
  "org_id" = "583675367868"
  "parent_folder" = "444651735300"
  "parent_id" = "folders/444651735300"
  "project_prefix" = "prj"
}
csr_repos = {
  "gcp-bootstrap" = {
    "id" = "projects/prj-b-cicd-fgbs/repos/gcp-bootstrap"
    "name" = "gcp-bootstrap"
    "project" = "prj-b-cicd-fgbs"
    "url" = "https://source.developers.google.com/p/prj-b-cicd-fgbs/r/gcp-bootstrap"
  }
  "gcp-environments" = {
    "id" = "projects/prj-b-cicd-fgbs/repos/gcp-environments"
    "name" = "gcp-environments"
    "project" = "prj-b-cicd-fgbs"
    "url" = "https://source.developers.google.com/p/prj-b-cicd-fgbs/r/gcp-environments"
  }
  "gcp-networks" = {
    "id" = "projects/prj-b-cicd-fgbs/repos/gcp-networks"
    "name" = "gcp-networks"
    "project" = "prj-b-cicd-fgbs"
    "url" = "https://source.developers.google.com/p/prj-b-cicd-fgbs/r/gcp-networks"
  }
  "gcp-org" = {
    "id" = "projects/prj-b-cicd-fgbs/repos/gcp-org"
    "name" = "gcp-org"
    "project" = "prj-b-cicd-fgbs"
    "url" = "https://source.developers.google.com/p/prj-b-cicd-fgbs/r/gcp-org"
  }
  "gcp-policies" = {
    "id" = "projects/prj-b-cicd-fgbs/repos/gcp-policies"
    "name" = "gcp-policies"
    "project" = "prj-b-cicd-fgbs"
    "url" = "https://source.developers.google.com/p/prj-b-cicd-fgbs/r/gcp-policies"
  }
  "gcp-projects" = {
    "id" = "projects/prj-b-cicd-fgbs/repos/gcp-projects"
    "name" = "gcp-projects"
    "project" = "prj-b-cicd-fgbs"
    "url" = "https://source.developers.google.com/p/prj-b-cicd-fgbs/r/gcp-projects"
  }
  "tf-cloudbuilder" = {
    "id" = "projects/prj-b-cicd-fgbs/repos/tf-cloudbuilder"
    "name" = "tf-cloudbuilder"
    "project" = "prj-b-cicd-fgbs"
    "url" = "https://source.developers.google.com/p/prj-b-cicd-fgbs/r/tf-cloudbuilder"
  }
}
environment_step_terraform_service_account_email = "sa-terraform-env@prj-b-seed-cf20.iam.gserviceaccount.com"
gcs_bucket_cloudbuild_artifacts = {
  "bootstrap" = "bkt-prj-b-cicd-fgbs-gcp-bootstrap-build-artifacts"
  "env" = "bkt-prj-b-cicd-fgbs-gcp-environments-build-artifacts"
  "net" = "bkt-prj-b-cicd-fgbs-gcp-networks-build-artifacts"
  "org" = "bkt-prj-b-cicd-fgbs-gcp-org-build-artifacts"
  "proj" = "bkt-prj-b-cicd-fgbs-gcp-projects-build-artifacts"
}
gcs_bucket_cloudbuild_logs = {
  "bootstrap" = "bkt-prj-b-cicd-fgbs-gcp-bootstrap-build-logs"
  "env" = "bkt-prj-b-cicd-fgbs-gcp-environments-build-logs"
  "net" = "bkt-prj-b-cicd-fgbs-gcp-networks-build-logs"
  "org" = "bkt-prj-b-cicd-fgbs-gcp-org-build-logs"
  "proj" = "bkt-prj-b-cicd-fgbs-gcp-projects-build-logs"
}
gcs_bucket_tfstate = "bkt-prj-b-seed-tfstate-8bea"
networks_step_terraform_service_account_email = "sa-terraform-net@prj-b-seed-cf20.iam.gserviceaccount.com"
optional_groups = tomap({
  "gcp_global_secrets_admin" = "gcp_global_secrets_admin3@obrienlabs.dev"
  "gcp_kms_admin" = "gcp_kms_admin3@obrienlabs.dev"
  "gcp_network_viewer" = "gcp_network_viewer3@obrienlabs.dev"
  "gcp_scc_admin" = "gcp_scc_admin3@obrienlabs.dev"
  "gcp_security_reviewer" = "gcp_security_reviewer3@obrienlabs.dev"
})
organization_step_terraform_service_account_email = "sa-terraform-org@prj-b-seed-cf20.iam.gserviceaccount.com"
projects_gcs_bucket_tfstate = "bkt-prj-b-seed-cf20-gcp-projects-tfstate"
projects_step_terraform_service_account_email = "sa-terraform-proj@prj-b-seed-cf20.iam.gserviceaccount.com"
required_groups = tomap({
  "audit_data_users" = "gcp-audit-data3@obrienlabs.dev"
  "billing_data_users" = "gcp-billing-data3@obrienlabs.dev"
  "group_billing_admins" = "gcp-billing-admins3@obrienlabs.dev"
  "group_org_admins" = "gcp-organization-admins3@obrienlabs.dev"
  "monitoring_workspace_users" = "gcp-monitoring-workspace3@obrienlabs.dev"
})
seed_project_id = "prj-b-seed-cf20"
michael@cloudshell:~/tef-oldev3/pbmm-on-gcp-onboarding/0-bootstrap (tef-oldev)$ 
Screenshot 2024-05-14 at 10 06 02

ran on older bootstrap project - but all good

Screenshot 2024-05-14 at 10 08 11
obriensystems commented 6 months ago

addresses #434

fmichaelobrien commented 6 months ago

An example of an attempted gcloud csr clone will kick in the authentication dialog

Screenshot 2024-05-14 at 12 42 48
michael@cloudshell:~/tef-oldev3/_test_repo (clouddeploy-ol)$ gcloud source repos clone gcp-environments --project=prj-b-cicd-khce
ERROR: (gcloud.source.repos.clone) You do not currently have an active account selected.
Please run:

  $ gcloud auth login

to obtain new credentials.

If you have already logged in with a different account, run:

  $ gcloud config set account ACCOUNT

to select an already authenticated account to use.

michael@cloudshell:~/tef-oldev3/_test_repo (clouddeploy-ol)$ gcloud auth login

You are already authenticated with gcloud when running
inside the Cloud Shell and so do not need to run this
command. Do you wish to proceed anyway?

Do you want to continue (Y/n)?  y

Go to the following link in your browser, and complete the sign-in prompts:

    https://accounts.google.com/o/oauth2/auth?response_type=code&client_id=32555940559.apps.googleusercontent.com&redirect_uri=https%3A%2F%2Fsdk.cloud.google.com%2Fauthcode.html&scope=openid+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fuserinfo.email+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fcloud-platform+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fappengine.admin+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fsqlservice.login+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fcompute+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Faccounts.reauth&state=aG2dXYAYft3Th6tRhLrlPta9R2SFnH&prompt=consent&token_usage=remote&access_type=offline&code_challenge=o5hpUDq-NnEEBvYal1yeuN_sDodjpFN03ngTn8wr3Es&code_challenge_method=S256

Once finished, enter the verification code provided in your browser: 4/0AdLIrYdzyTUtvEi5ypXe28QTkjUE_7b1amWrJV4u_TH8QI9cl9DXDVjFnZ-UawcrZuz9KA

You are now logged in as [michael@obrienlabs.dev].
Your current project is [tef-oldev].  You can change this setting by running:
  $ gcloud config set project PROJECT_ID

michael@cloudshell:~/tef-oldev3/_test_repo (tef-oldev)$ gcloud source repos clone gcp-environments --project=prj-b-cicd-khce
Cloning into '/home/michael/tef-oldev3/_test_repo/gcp-environments'...
warning: You appear to have cloned an empty repository.
Project [prj-b-cicd-khce] repository [gcp-environments] was cloned to [/home/michael/tef-oldev3/_test_repo/gcp-environments].
michael@cloudshell:~/tef-oldev3/_test_repo (tef-oldev)$ ls 
gcp-environments  gcp-policies

Client after re-authentication sa is admin and csr admin in iam, and also set in the repo permissions block copy

" ERROR: (gcloud.source.repos.clone) Command '['git', 'clone', 'https://source.developers.google.com/p/prj-b-cicd-orcl/r/gcp-networks', '/home/user-account/lz-tef-dev-2/temp/gcp-networks', '--config', 'credential.https://source.developers.google.com/.helper=', '--config', 'credential.https://source.developers.google.com/.helper=!gcloud auth git-helper --account=user@gcp..ca --ignore-unknown $@']' returned non-zero exit status 128.

from #429

close but not related (local sdk) https://groups.google.com/g/google-cloud-dev/c/fpaCcI6Bulg

obriensystems commented 6 months ago

SSH workaround for CSR

ichael@cloudshell:~/tef-oldev3/_test_repo$ ssh-keygen -t rsa -C michael@obrienlabs.dev
Generating public/private rsa key pair.
Enter file in which to save the key (/home/michael/.ssh/id_rsa): /home/michael/.ssh/csr_id_rsa

get key
michael@cloudshell:~/tef-oldev3/_test_repo (tef-oldev3)$ cat ~/.ssh/csr_id_rsa.pub 
ssh-rsa AAAAB3NzaC1yc2EA....ev

add key to CSR

Screenshot 2024-05-15 at 14 51 19

Clone

need ssh-add ~/.ssh/csr_id_rsa

michael@cloudshell:~/tef-oldev3/_test_repo (tef-oldev3)$ vi ~/.ssh/config
michael@cloudshell:~/tef-oldev3/_test_repo (tef-oldev3)$ cat ~/.ssh/config 
Host csr
    Hostname source.developers.google.com
    IdentityFile ~/.ssh/csr_id_rsa
    IdentitiesOnly yes # see NOTES below
    AddKeysToAgent yes
michael@cloudshell:~/tef-oldev3/_test_repo (tef-oldev3)$ git clone ssh://michael@obrienlabs.dev@source.developers.google.com:2022/p/prj-b-cicd-fgbs/r/gcp-networks 
Cloning into 'gcp-networks'...
michael@obrienlabs.dev@source.developers.google.com: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
obriensystems commented 6 months ago

Use the default id_rsa key name

michael@cloudshell:~/tef-oldev3/_test_repo (tef-oldev3)$ ls ~/.ssh
config  csr_id_rsa  csr_id_rsa.pub  google_compute_engine  google_compute_engine.pub  google_compute_known_hosts  known_hosts  obrienlabs_org_github
michael@cloudshell:~/tef-oldev3/_test_repo (tef-oldev3)$ ssh-keygen -m PEM -t rsa -P ""
Generating public/private rsa key pair.
Enter file in which to save the key (/home/michael/.ssh/id_rsa): 
Your identification has been saved in /home/michael/.ssh/id_rsa
Your public key has been saved in /home/michael/.ssh/id_rsa.pub

michael@cloudshell:~/tef-oldev3/_test_repo (tef-oldev3)$ cat ~/.ssh/id_rsa
id_rsa      id_rsa.pub  
michael@cloudshell:~/tef-oldev3/_test_repo (tef-oldev3)$ cat ~/.ssh/id_rsa.pub 
ssh-rsa AAAA...1616-default

register key with CSR

Screenshot 2024-05-15 at 15 06 41
michael@cloudshell:~/tef-oldev3/_test_repo (tef-oldev3)$ git clone ssh://michael@obrienlabs.dev@source.developers.google.com:2022/p/prj-b-cicd-fgbs/r/gcp-networks 
Cloning into 'gcp-networks'...
warning: You appear to have cloned an empty repository.
michael@cloudshell:~/tef-oldev3/_test_repo (tef-oldev3)$ ls gcp-networks/
michael@cloudshell:~/tef-oldev3/_test_repo (tef-oldev3)$ ls
gcp-environments  gcp-networks  gcp-policies
michael@cloudshell:~/tef-oldev3/_test_repo (tef-oldev3)$ cd gcp-networks/
michael@cloudshell:~/tef-oldev3/_test_repo/gcp-networks (tef-oldev3)$ git status
On branch master
No commits yet
nothing to commit (create/copy files and use "git add" to track)

procedure on #440

fmichaelobrien commented 5 months ago

gcloud api is the same as another working org SSH works for client - avoid using github for now

also verify ssh-agent - working on my cloudshell

gcloud version
admin_@cloudshell:~$ eval `ssh-agent`
Agent pid 1370
admin_@cloudshell:~$ ssh-add ~/github-f....n_202112 
Identity added: /home/admin_/github-f...ien_202112 (f...com)
obriensystems commented 5 months ago

SSH main retrofit - option to cloud repos API authentication

using ssh-agent

michael@cloudshell:~/tef-oldev4 (clouddeploy-ol)$ mkdir _431_ssh_testing
michael@cloudshell:~/tef-oldev4 (clouddeploy-ol)$ eval `ssh-agent`
Agent pid 1357
michael@cloudshell:~/tef-oldev4 (clouddeploy-ol)$ ls ~/.ssh
config      csr_id_rsa.pub         google_compute_engine.pub   id_rsa      known_hosts
csr_id_rsa  google_compute_engine  google_compute_known_hosts  id_rsa.pub  obrienlabs_org_github
michael@cloudshell:~/tef-oldev4 (clouddeploy-ol)$ gcloud config set project tef-oldev4
Updated property [core/project].
michael@cloudshell:~/tef-oldev4 (tef-oldev4)$ 

branch https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/tree/gh431-csr-ssh test ssh-add

ichael@cloudshell:~/tef-oldev4 (tef-oldev4)$ git clone git@github.com:GoogleCloudPlatform/pbmm-on-gcp-onboarding.git
Cloning into 'pbmm-on-gcp-onboarding'...
git@github.com: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
michael@cloudshell:~/tef-oldev4 (tef-oldev4)$ ssh-add ~/.ssh/obrienlabs_org_github 
Identity added: /home/michael/.ssh/obrienlabs_org_github (michael@obrienlabs.org)
michael@cloudshell:~/tef-oldev4 (tef-oldev4)$ git clone git@github.com:GoogleCloudPlatform/pbmm-on-gcp-onboarding.git
Cloning into 'pbmm-on-gcp-onboarding'...
remote: Enumerating objects: 6783, done.
remote: Counting objects: 100% (2527/2527), done.
remote: Compressing objects: 100% (892/892), done.
remote: Total 6783 (delta 1757), reused 2181 (delta 1598), pack-reused 4256
Receiving objects: 100% (6783/6783), 31.86 MiB | 28.05 MiB/s, done.
Resolving deltas: 100% (4179/4179), done.
michael@cloudshell:~/tef-oldev4 (tef-oldev4)$ 

edit tfvars

org_id = "58...8" # format "000000000000"

billing_account = "0...76" # format "000000-000000-000000"

// For enabling the automatic groups creation, uncoment the
// variables and update the values with the group names
groups = {
  create_required_groups = true # Change to true to create the required_groups
  create_optional_groups = true # Change to true to create the optional_groups
  billing_project        = "te..v4"  # Fill with bootstrap project id (the one you are starting with) to create required or optional groups
  required_groups = {
    group_org_admins           = "gcp-organization-admins4@obrienlabs.dev" # example "gcp-organization-admins@example.com"
    group_billing_admins       = "gcp-billing-admins4@obrienlabs.dev" # example "gcp-billing-admins@example.com"
    billing_data_users         = "gcp-billing-data4@obrienlabs.dev" # example "gcp-billing-data@example.com"
    audit_data_users           = "gcp-audit-data4@obrienlabs.dev" # example "gcp-audit-data@example.com"
    monitoring_workspace_users = "gcp-monitoring-workspace4@obrienlabs.dev" # example "gcp-monitoring-workspace@example.com"
  }
  optional_groups = {
     gcp_security_reviewer      = "gcp_security_reviewer4@obrienlabs.dev" #"gcp_security_reviewer_local_test@example.com"
     gcp_network_viewer         = "gcp_network_viewer4@obrienlabs.dev" #"gcp_network_viewer_local_test@example.com"
     gcp_scc_admin              = "gcp_scc_admin4@obrienlabs.dev" #"gcp_scc_admin_local_test@example.com"
     gcp_global_secrets_admin   = "gcp_global_secrets_admin4@obrienlabs.dev" #"gcp_global_secrets_admin_local_test@example.com"
     gcp_kms_admin              = "gcp_kms_admin4@obrienlabs.dev" #"gcp_kms_admin_local_test@example.com"
   }
}

default_region = "northamerica-northeast1"
#default_region = "northamerica-northeast2"

# Optional - for an organization with existing projects or for development/validation.
# Uncomment this variable to place all the example foundation resources under
# the provided folder instead of the root organization.
# The variable value is the numeric folder ID
# The folder must already exist.
parent_folder = "4..37"

downgrade terraform

michael@cloudshell:~/tef-oldev4 (tef-oldev4)$ cp terraform /usr/bin/terraform 
cp: cannot create regular file '/usr/bin/terraform': Permission denied
michael@cloudshell:~/tef-oldev4 (tef-oldev4)$ sudo cp terraform /usr/bin/terraform 
michael@cloudshell:~/tef-oldev4 (tef-oldev4)$ cd pbmm-on-gcp-onboarding/0-bootstrap/
michael@cloudshell:~/tef-oldev4/pbmm-on-gcp-onboarding/0-bootstrap (tef-oldev4)$ terraform version
Terraform v1.3.10
on linux_amd64

Your version of Terraform is out of date! The latest version
is 1.8.3. You can update by downloading from https://www.terraform.io/downloads.html

terraform init and plan

Plan: 271 to add, 0 to change, 0 to destroy.

Changes to Outputs:
  + bootstrap_step_terraform_service_account_email    = (known after apply)
  + cloud_build_peered_network_id                     = (known after apply)
  + cloud_build_private_worker_pool_id                = (known after apply)
  + cloud_build_worker_peered_ip_range                = "192.168.0.0/24"
  + cloud_build_worker_range_id                       = (known after apply)
  + cloud_builder_artifact_repo                       = (known after apply)
  + cloudbuild_project_id                             = (known after apply)
  + common_config                                     = {
      + billing_account       = "019283-6F1AB5-7AD576"
      + bootstrap_folder_name = (known after apply)
      + default_region        = "northamerica-northeast1"
      + folder_prefix         = "fldr"
      + org_id                = "583675367868"
      + parent_folder         = "479872525237"
      + parent_id             = "folders/479872525237"
      + project_prefix        = "prj"
    }
  + csr_repos                                         = {
      + gcp-bootstrap    = {
          + id      = (known after apply)
          + name    = "gcp-bootstrap"
          + project = (known after apply)
          + url     = (known after apply)
        }
      + gcp-environments = {
          + id      = (known after apply)
          + name    = "gcp-environments"
          + project = (known after apply)
          + url     = (known after apply)
        }
      + gcp-networks     = {
          + id      = (known after apply)
          + name    = "gcp-networks"
          + project = (known after apply)
          + url     = (known after apply)
        }
      + gcp-org          = {
          + id      = (known after apply)
          + name    = "gcp-org"
          + project = (known after apply)
          + url     = (known after apply)
        }
      + gcp-policies     = {
          + id      = (known after apply)
          + name    = "gcp-policies"
          + project = (known after apply)
          + url     = (known after apply)
        }
      + gcp-projects     = {
          + id      = (known after apply)
          + name    = "gcp-projects"
          + project = (known after apply)
          + url     = (known after apply)
        }
      + tf-cloudbuilder  = {
          + id      = (known after apply)
          + name    = "tf-cloudbuilder"
          + project = (known after apply)
          + url     = (known after apply)
        }
    }
  + environment_step_terraform_service_account_email  = (known after apply)
  + gcs_bucket_cloudbuild_artifacts                   = {
      + bootstrap = (known after apply)
      + env       = (known after apply)
      + net       = (known after apply)
      + org       = (known after apply)
      + proj      = (known after apply)
    }
  + gcs_bucket_cloudbuild_logs                        = {
      + bootstrap = (known after apply)
      + env       = (known after apply)
      + net       = (known after apply)
      + org       = (known after apply)
      + proj      = (known after apply)
    }
  + gcs_bucket_tfstate                                = (known after apply)
  + networks_step_terraform_service_account_email     = (known after apply)
  + optional_groups                                   = {
      + "gcp_global_secrets_admin" = "gcp_global_secrets_admin4@obrienlabs.dev"
      + "gcp_kms_admin"            = "gcp_kms_admin4@obrienlabs.dev"
      + "gcp_network_viewer"       = "gcp_network_viewer4@obrienlabs.dev"
      + "gcp_scc_admin"            = "gcp_scc_admin4@obrienlabs.dev"
      + "gcp_security_reviewer"    = "gcp_security_reviewer4@obrienlabs.dev"
    }
  + organization_step_terraform_service_account_email = (known after apply)
  + projects_gcs_bucket_tfstate                       = (known after apply)
  + projects_step_terraform_service_account_email     = (known after apply)
  + required_groups                                   = {
      + "audit_data_users"           = "gcp-audit-data4@obrienlabs.dev"
      + "billing_data_users"         = "gcp-billing-data4@obrienlabs.dev"
      + "group_billing_admins"       = "gcp-billing-admins4@obrienlabs.dev"
      + "group_org_admins"           = "gcp-organization-admins4@obrienlabs.dev"
      + "monitoring_workspace_users" = "gcp-monitoring-workspace4@obrienlabs.dev"
    }
  + seed_project_id                                   = (known after apply)

─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

Saved the plan to: bootstrap.tfplan

To perform exactly these actions, run the following command to apply:
    terraform apply "bootstrap.tfplan"

1036

expected eventually consistent error on group creation after 1 min

michael@cloudshell:~/tef-oldev4/pbmm-on-gcp-onboarding/0-bootstrap (tef-oldev4)$ terraform apply bootstrap.tfplan
module.bootstrap_csr_repo.null_resource.run_destroy_command[0]: Creating...
module.bootstrap_csr_repo.null_resource.run_destroy_command[0]: Creation complete after 0s [id=4012333594955662454]
module.seed_bootstrap.module.seed_project.module.project-factory.random_id.random_project_id_suffix: Creating...
module.tf_private_pool.random_string.suffix: Creating...
module.seed_bootstrap.random_id.suffix: Creating...
random_string.suffix: Creating...
module.tf_private_pool.random_string.suffix: Creation complete after 0s [id=gdo0]
module.seed_bootstrap.module.seed_project.module.project-factory.random_id.random_project_id_suffix: Creation complete after 0s [id=Sa8]
random_string.suffix: Creation complete after 0s [id=pdn7]
module.seed_bootstrap.random_id.suffix: Creation complete after 0s [id=leg]
module.seed_bootstrap.google_folder_iam_member.org_admin_serviceusage_consumer[0]: Creating...
module.seed_bootstrap.google_organization_iam_member.org_admins_group["roles/resourcemanager.organizationAdmin"]: Creating...
module.seed_bootstrap.google_organization_iam_member.org_admins_group["roles/billing.user"]: Creating...
module.seed_bootstrap.google_folder_iam_member.tmp_project_creator[0]: Creating...
module.seed_bootstrap.google_organization_iam_member.org_billing_admin: Creating...
module.required_group["audit_data_users"].google_cloud_identity_group.group: Creating...
module.seed_bootstrap.google_organization_iam_binding.billing_creator: Creating...
module.seed_bootstrap.google_folder_iam_member.org_admin_service_account_user[0]: Creating...
google_folder.bootstrap: Creating...
module.optional_group["gcp_scc_admin"].google_cloud_identity_group.group: Creating...
module.required_group["group_billing_admins"].google_cloud_identity_group.group: Creating...
module.required_group["billing_data_users"].google_cloud_identity_group.group: Creating...
module.required_group["group_org_admins"].google_cloud_identity_group.group: Creating...
module.required_group["monitoring_workspace_users"].google_cloud_identity_group.group: Creating...
module.seed_bootstrap.google_folder_iam_member.org_admin_service_account_user[0]: Creation complete after 6s [id=folders/479872525237/roles/iam.serviceAccountUser/group:gcp-organization-admins4@obrienlabs.dev]
module.optional_group["gcp_security_reviewer"].google_cloud_identity_group.group: Creating...
module.required_group["audit_data_users"].google_cloud_identity_group.group: Creation complete after 9s [id=groups/0111kx3o0i0vokv]
module.optional_group["gcp_network_viewer"].google_cloud_identity_group.group: Creating...
module.optional_group["gcp_scc_admin"].google_cloud_identity_group.group: Creation complete after 10s [id=groups/01pxezwc2yon24n]
module.optional_group["gcp_global_secrets_admin"].google_cloud_identity_group.group: Creating...
module.required_group["group_billing_admins"].google_cloud_identity_group.group: Creation complete after 9s [id=groups/02koq6561p3tqsw]
module.optional_group["gcp_kms_admin"].google_cloud_identity_group.group: Creating...
module.required_group["billing_data_users"].google_cloud_identity_group.group: Creation complete after 9s [id=groups/017dp8vu47j024a]
module.seed_bootstrap.google_organization_iam_member.org_billing_admin: Still creating... [10s elapsed]
google_folder.bootstrap: Still creating... [10s elapsed]
module.seed_bootstrap.google_organization_iam_binding.billing_creator: Still creating... [10s elapsed]
module.required_group["monitoring_workspace_users"].google_cloud_identity_group.group: Creation complete after 9s [id=groups/04bvk7pj2i03lqt]
module.seed_bootstrap.google_organization_iam_binding.billing_creator: Creation complete after 11s [id=583675367868/roles/billing.creator]
module.required_group["group_org_admins"].google_cloud_identity_group.group: Creation complete after 9s [id=groups/04bvk7pj0j31gy7]
module.seed_bootstrap.google_organization_iam_member.org_billing_admin: Creation complete after 11s [id=583675367868/roles/billing.admin/group:gcp-billing-admins4@obrienlabs.dev]
google_folder.bootstrap: Creation complete after 12s [id=folders/976224166955]
module.optional_group["gcp_security_reviewer"].google_cloud_identity_group.group: Creation complete after 9s [id=groups/03fwokq00h2vpb9]
module.optional_group["gcp_network_viewer"].google_cloud_identity_group.group: Creation complete after 9s [id=groups/02p2csry3d9cins]
module.optional_group["gcp_kms_admin"].google_cloud_identity_group.group: Creation complete after 8s [id=groups/00sqyw640obco6o]
module.optional_group["gcp_global_secrets_admin"].google_cloud_identity_group.group: Creation complete after 8s [id=groups/01ci93xb3cor196]
╷
│ Error: Error applying IAM policy for folder "folders/479872525237": Error setting IAM policy for folder "folders/479872525237": googleapi: Error 400: Group gcp-organization-admins4@obrienlabs.dev does not exist., badRequest
│ 
│   with module.seed_bootstrap.google_folder_iam_member.tmp_project_creator[0],
│   on .terraform/modules/seed_bootstrap/main.tf line 47, in resource "google_folder_iam_member" "tmp_project_creator":
│   47: resource "google_folder_iam_member" "tmp_project_creator" {
│ 
╵
╷
│ Error: Error applying IAM policy for organization "583675367868": Error setting IAM policy for organization "583675367868": googleapi: Error 400: Group gcp-organization-admins4@obrienlabs.dev does not exist., badRequest
│ 
│   with module.seed_bootstrap.google_organization_iam_member.org_admins_group["roles/billing.user"],
│   on .terraform/modules/seed_bootstrap/main.tf line 184, in resource "google_organization_iam_member" "org_admins_group":
│  184: resource "google_organization_iam_member" "org_admins_group" {
│ 
╵
╷
│ Error: Error applying IAM policy for organization "583675367868": Error setting IAM policy for organization "583675367868": googleapi: Error 400: Group gcp-organization-admins4@obrienlabs.dev does not exist., badRequest
│ 
│   with module.seed_bootstrap.google_organization_iam_member.org_admins_group["roles/resourcemanager.organizationAdmin"],
│   on .terraform/modules/seed_bootstrap/main.tf line 184, in resource "google_organization_iam_member" "org_admins_group":
│  184: resource "google_organization_iam_member" "org_admins_group" {
│ 
╵
╷
│ Error: Error applying IAM policy for folder "folders/479872525237": Error setting IAM policy for folder "folders/479872525237": googleapi: Error 400: Group gcp-organization-admins4@obrienlabs.dev does not exist., badRequest
│ 
│   with module.seed_bootstrap.google_folder_iam_member.org_admin_serviceusage_consumer[0],
│   on .terraform/modules/seed_bootstrap/main.tf line 267, in resource "google_folder_iam_member" "org_admin_serviceusage_consumer":
│  267: resource "google_folder_iam_member" "org_admin_serviceusage_consumer" {
│ 
╵

groups are there

Screenshot 2024-05-23 at 10 41 04

restarting 1042

plan and apply 2

Plan: 252 to add, 0 to change, 0 to destroy.

michael@cloudshell:~/tef-oldev4/pbmm-on-gcp-onboarding/0-bootstrap (tef-oldev4)$ terraform apply bootstrap.tfplan
module.seed_bootstrap.google_organization_iam_member.org_admins_group["roles/billing.user"]: Creating...
module.seed_bootstrap.google_organization_iam_member.org_admins_group["roles/resourcemanager.organizationAdmin"]: Creating...
module.seed_bootstrap.google_folder_iam_member.tmp_project_creator[0]: Creating...
module.seed_bootstrap.google_folder_iam_member.org_admin_serviceusage_consumer[0]: Creating...
module.seed_bootstrap.google_folder_iam_member.tmp_project_creator[0]: Creation complete after 4s [id=folders/479872525237/roles/resourcemanager.projectCreator/group:gcp-organization-admins4@obrienlabs.dev]
module.seed_bootstrap.module.seed_project.module.project-factory.google_project.main: Creating...
module.seed_bootstrap.google_organization_iam_member.org_admins_group["roles/billing.user"]: Creation complete after 5s [id=583675367868/roles/billing.user/group:gcp-organization-admins4@obrienlabs.dev]
module.seed_bootstrap.google_organization_iam_member.org_admins_group["roles/resourcemanager.organizationAdmin"]: Creation complete after 9s [id=583675367868/roles/resourcemanager.organizationAdmin/group:gcp-organization-admins4@obrienlabs.dev]
module.seed_bootstrap.google_folder_iam_member.org_admin_serviceusage_consumer[0]: Creation complete after 9s [id=folders/479872525237/roles/serviceusage.serviceUsageConsumer/group:gcp-organization-admins4@obrienlabs.dev]
module.seed_bootstrap.module.seed_project.module.project-factory.google_project.main: Still creating... [10s elapsed]

good

raised https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/issues/443

Screenshot 2024-05-23 at 10 50 50

continuing

module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/storage.admin"]: Creating...
module.bootstrap_csr_repo.null_resource.run_command[0]: Creating...
google_sourcerepo_repository_iam_member.member["net"]: Creating...
module.bootstrap_csr_repo.null_resource.run_command[0]: Provisioning with 'local-exec'...
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): Executing: ["/bin/sh" "-c" "PATH=/google-cloud-sdk/bin:$PATH\n./scripts/push-to-repo.sh prj-b-cicd-pdn7 tf-cloudbuilder ./Dockerfile\n"]
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + '[' 3 -lt 3 ']'
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + CSR_PROJECT_ID=prj-b-cicd-pdn7
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + CSR_NAME=tf-cloudbuilder
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + DOCKERFILE_PATH=./Dockerfile
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): ++ mktemp -d
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + tmp_dir=/tmp/tmp.03EaYa6qc3
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + gcloud source repos clone tf-cloudbuilder /tmp/tmp.03EaYa6qc3 --project prj-b-cicd-pdn7
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): Cloning into '/tmp/tmp.03EaYa6qc3'...
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): warning: You appear to have cloned an empty repository.
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): Project [prj-b-cicd-pdn7] repository [tf-cloudbuilder] was cloned to [/tmp/tmp.03EaYa6qc3].
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + cp ./Dockerfile /tmp/tmp.03EaYa6qc3
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + pushd /tmp/tmp.03EaYa6qc3
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): /tmp/tmp.03EaYa6qc3 ~/tef-oldev4/pbmm-on-gcp-onboarding/0-bootstrap
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + git config credential.helper gcloud.sh
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + git config init.defaultBranch main
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + git config user.email terraform-robot@example.com
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + git config user.name 'TF Robot'
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + git checkout main
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): error: pathspec 'main' did not match any file(s) known to git
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + git checkout -b main
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): Switched to a new branch 'main'
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + git add Dockerfile
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + git commit -m 'Initialize tf dockerfile repo'
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): [main (root-commit) 6419d5b] Initialize tf dockerfile repo
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec):  1 file changed, 39 insertions(+)
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec):  create mode 100644 Dockerfile
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + git push origin main -f
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/cloudscheduler.admin"]: Creation complete after 8s [id=prj-b-cicd-pdn7/roles/cloudscheduler.admin/serviceAccount:sa-terraform-bootstrap@prj-b-seed-49af.iam.gserviceaccount.com]
google_sourcerepo_repository_iam_member.member["env"]: Creating...
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): remote: Waiting for private key checker: 1/1 objects left
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): To https://source.developers.google.com/p/prj-b-cicd-pdn7/r/tf-cloudbuilder
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec):  * [new branch]      main -> main
module.bootstrap_csr_repo.null_resource.run_command[0]: Creation complete after 8s [id=5532690325071743313]
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/source.admin"]: Creation complete after 8s [id=prj-b-cicd-pdn7/roles/source.admin/serviceAccount:sa-terraform-bootstrap@prj-b-seed-49af.iam.gserviceaccount.com]
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/iam.workloadIdentityPoolAdmin"]: Creating...
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/resourcemanager.projectDeleter"]: Creating...
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/compute.networkAdmin"]: Creation complete after 8s [id=prj-b-cicd-pdn7/roles/compute.networkAdmin/serviceAccount:sa-terraform-bootstrap@prj-b-seed-49af.iam.gserviceaccount.com]
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/iam.serviceAccountAdmin"]: Creating...
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/workflows.admin"]: Creation complete after 8s [id=prj-b-cicd-pdn7/roles/workflows.admin/serviceAccount:sa-terraform-bootstrap@prj-b-seed-49af.iam.gserviceaccount.com]
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/cloudbuild.workerPoolOwner"]: Creating...
google_sourcerepo_repository_iam_member.member["proj"]: Creation complete after 8s [id=projects/prj-b-cicd-pdn7/repos/gcp-policies/roles/viewer/serviceAccount:sa-terraform-proj@prj-b-seed-49af.iam.gserviceaccount.com]
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/dns.admin"]: Creating...
google_sourcerepo_repository_iam_member.member["net"]: Creation complete after 9s [id=projects/prj-b-cicd-pdn7/repos/gcp-policies/roles/viewer/serviceAccount:sa-terraform-net@prj-b-seed-49af.iam.gserviceaccount.com]
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/storage.admin"]: Creation complete after 9s [id=prj-b-cicd-pdn7/roles/storage.admin/serviceAccount:sa-terraform-bootstrap@prj-b-seed-49af.iam.gserviceaccount.com]
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/artifactregistry.admin"]: Creating...
google_sourcerepo_repository_iam_member.member["bootstrap"]: Creating...
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/cloudbuild.builds.editor"]: Creation complete after 9s [id=prj-b-cicd-pdn7/roles/cloudbuild.builds.editor/serviceAccount:sa-terraform-bootstrap@prj-b-seed-49af.iam.gserviceaccount.com]
google_sourcerepo_repository_iam_member.member["org"]: Creating...
module.tf_private_pool.module.peered_network[0].module.vpc.google_compute_network.network: Still creating... [10s elapsed]
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/artifactregistry.admin"]: Creation complete after 7s [id=prj-b-cicd-pdn7/roles/artifactregistry.admin/serviceAccount:sa-terraform-bootstrap@prj-b-seed-49af.iam.gserviceaccount.com]
module.tf_cloud_builder.google_service_account.cb_sa[0]: Creating...
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/cloudbuild.workerPoolOwner"]: Creation complete after 8s [id=prj-b-cicd-pdn7/roles/cloudbuild.workerPoolOwner/serviceAccount:sa-terraform-bootstrap@prj-b-seed-49af.iam.gserviceaccount.com]
module.tf_cloud_builder.google_artifact_registry_repository.tf-image-repo: Creating...
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/iam.serviceAccountAdmin"]: Creation complete after 8s [id=prj-b-cicd-pdn7/roles/iam.serviceAccountAdmin/serviceAccount:sa-terraform-bootstrap@prj-b-seed-49af.iam.gserviceaccount.com]
module.tf_cloud_builder.google_service_account.workflow_sa[0]: Creating...
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/iam.workloadIdentityPoolAdmin"]: Creation complete after 8s [id=prj-b-cicd-pdn7/roles/iam.workloadIdentityPoolAdmin/serviceAccount:sa-terraform-bootstrap@prj-b-seed-49af.iam.gserviceaccount.com]
module.tf_cloud_builder.module.bucket.google_storage_bucket.bucket: Creating...
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/resourcemanager.projectDeleter"]: Creation complete after 8s [id=prj-b-cicd-pdn7/roles/resourcemanager.projectDeleter/serviceAccount:sa-terraform-bootstrap@prj-b-seed-49af.iam.gserviceaccount.com]
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/dns.admin"]: Creation complete after 9s [id=prj-b-cicd-pdn7/roles/dns.admin/serviceAccount:sa-terraform-bootstrap@prj-b-seed-49af.iam.gserviceaccount.com]
module.bootstrap_projects_remove_editor["seed"].google_project_iam_binding.iam_remove["roles/editor"]: Creating...
module.bootstrap_projects_remove_editor["cicd"].google_project_iam_binding.iam_remove["roles/editor"]: Creating...
module.tf_cloud_builder.google_service_account.cb_sa[0]: Creation complete after 1s [id=projects/prj-b-cicd-pdn7/serviceAccounts/tf-cb-builder-sa@prj-b-cicd-pdn7.iam.gserviceaccount.com]
module.tf_cloud_builder.google_sourcerepo_repository_iam_member.member[0]: Creating...
module.tf_cloud_builder.google_service_account.workflow_sa[0]: Creation complete after 1s [id=projects/prj-b-cicd-pdn7/serviceAccounts/terraform-runner-workflow-sa@prj-b-cicd-pdn7.iam.gserviceaccount.com]
module.tf_cloud_builder.google_project_iam_member.logs_writer: Creating...
module.tf_cloud_builder.module.bucket.google_storage_bucket.bucket: Creation complete after 1s [id=bkt-prj-b-cicd-pdn7-tf-cloudbuilder-build-logs]
module.tf_cloud_builder.google_project_iam_member.invoke_workflow_scheduler: Creating...
google_sourcerepo_repository_iam_member.member["env"]: Still creating... [10s elapsed]
google_sourcerepo_repository_iam_member.member["bootstrap"]: Still creating... [10s elapsed]
google_sourcerepo_repository_iam_member.member["org"]: Still creating... [10s elapsed]
google_sourcerepo_repository_iam_member.member["env"]: Creation complete after 12s [id=projects/prj-b-cicd-pdn7/repos/gcp-policies/roles/viewer/serviceAccount:sa-terraform-env@prj-b-seed-49af.iam.gserviceaccount.com]
module.tf_cloud_builder.google_project_iam_member.trigger_builds: Creating...
google_sourcerepo_repository_iam_member.member["bootstrap"]: Creation complete after 11s [id=projects/prj-b-cicd-pdn7/repos/gcp-policies/roles/viewer/serviceAccount:sa-terraform-bootstrap@prj-b-seed-49af.iam.gserviceaccount.com]
module.tf_cloud_builder.google_workflows_workflow.builder: Creating...
google_sourcerepo_repository_iam_member.member["org"]: Creation complete after 11s [id=projects/prj-b-cicd-pdn7/repos/gcp-policies/roles/viewer/serviceAccount:sa-terraform-org@prj-b-seed-49af.iam.gserviceaccount.com]
module.tf_cloud_builder.google_service_account_iam_member.use_cb_sa: Creating...
module.tf_private_pool.module.peered_network[0].module.vpc.google_compute_network.network: Still creating... [20s elapsed]
module.tf_cloud_builder.google_sourcerepo_repository_iam_member.member[0]: Creation complete after 4s [id=projects/prj-b-cicd-pdn7/repos/tf-cloudbuilder/roles/viewer/serviceAccount:tf-cb-builder-sa@prj-b-cicd-pdn7.iam.gserviceaccount.com]
module.tf_cloud_builder.google_storage_bucket_iam_member.member: Creating...
module.tf_cloud_builder.google_workflows_workflow.builder: Creation complete after 3s [id=projects/prj-b-cicd-pdn7/locations/northamerica-northeast1/workflows/terraform-runner-workflow]
module.tf_cloud_builder.google_cloud_scheduler_job.trigger_workflow: Creating...
module.tf_cloud_builder.google_service_account_iam_member.use_cb_sa: Creation complete after 4s [id=projects/prj-b-cicd-pdn7/serviceAccounts/tf-cb-builder-sa@prj-b-cicd-pdn7.iam.gserviceaccount.com/roles/iam.serviceAccountUser/serviceAccount:terraform-runner-workflow-sa@prj-b-cicd-pdn7.iam.gserviceaccount.com]
module.tf_cloud_builder.google_project_iam_member.invoke_workflow_scheduler: Creation complete after 7s [id=prj-b-cicd-pdn7/roles/workflows.invoker/serviceAccount:terraform-runner-workflow-sa@prj-b-cicd-pdn7.iam.gserviceaccount.com]
module.bootstrap_projects_remove_editor["seed"].google_project_iam_binding.iam_remove["roles/editor"]: Creation complete after 7s [id=prj-b-seed-49af/roles/editor]
module.tf_cloud_builder.google_storage_bucket_iam_member.member: Creation complete after 4s [id=b/bkt-prj-b-cicd-pdn7-tf-cloudbuilder-build-logs/roles/storage.admin/serviceAccount:tf-cb-builder-sa@prj-b-cicd-pdn7.iam.gserviceaccount.com]
module.tf_cloud_builder.google_artifact_registry_repository.tf-image-repo: Still creating... [10s elapsed]
module.bootstrap_projects_remove_editor["cicd"].google_project_iam_binding.iam_remove["roles/editor"]: Still creating... [10s elapsed]
module.tf_cloud_builder.google_artifact_registry_repository.tf-image-repo: Creation complete after 11s [id=projects/prj-b-cicd-pdn7/locations/northamerica-northeast1/repositories/tf-runners]
module.tf_cloud_builder.google_project_iam_member.logs_writer: Still creating... [10s elapsed]
module.tf_cloud_builder.google_artifact_registry_repository_iam_member.push_images: Creating...
module.tf_cloud_builder.google_artifact_registry_repository_iam_member.workflow_list: Creating...
google_artifact_registry_repository_iam_member.terraform_sa_artifact_registry_reader["env"]: Creating...
google_artifact_registry_repository_iam_member.terraform_sa_artifact_registry_reader["bootstrap"]: Creating...
google_artifact_registry_repository_iam_member.terraform_sa_artifact_registry_reader["org"]: Creating...

raised https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/issues/444

obriensystems commented 5 months ago

getting expected cb worker pool quota error

Screenshot 2024-05-23 at 11 15 11
module.tf_private_pool.google_compute_network_peering_routes_config.peering_routes[0]: Creating...
module.tf_private_pool.google_cloudbuild_worker_pool.private_pool: Creating...
module.tf_private_pool.google_compute_network_peering_routes_config.peering_routes[0]: Still creating... [10s elapsed]
module.tf_private_pool.google_compute_network_peering_routes_config.peering_routes[0]: Creation complete after 11s [id=projects/prj-b-cicd-pdn7/global/networks/vpc-b-cbpools/networkPeerings/servicenetworking-googleapis-com]
╷
│ Error: Error creating WorkerPool: operation received error: error code "8", message: generic::resource_exhausted: project 633332953071 has insufficient quota to create a new worker pool; see https://cloud.google.com/build/quotas, details: []
│  details: map[]
│ 
│   with module.tf_private_pool.google_cloudbuild_worker_pool.private_pool,
│   on modules/cb-private-pool/main.tf line 30, in resource "google_cloudbuild_worker_pool" "private_pool":
│   30: resource "google_cloudbuild_worker_pool" "private_pool" {
│ 
╵

switching region

#default_region = "northamerica-northeast1"
default_region = "northamerica-northeast2"

cannot because of kms lifecycle

google_billing_account_iam_member.billing_admin_user["org"]: Refreshing state... [id=019283-6F1AB5-7AD576/roles/billing.admin/serviceAccount:sa-terraform-org@prj-b-seed-49af.iam.gserviceaccount.com]
╷
│ Error: Instance cannot be destroyed
│ 
│   on .terraform/modules/seed_bootstrap.kms/main.tf line 27:
│   27: resource "google_kms_crypto_key" "key" {
│ 
│ Resource module.seed_bootstrap.module.kms[0].google_kms_crypto_key.key[0] has lifecycle.prevent_destroy set, but the plan calls for this resource to be destroyed.
│ To avoid this error and continue with the plan, either disable lifecycle.prevent_destroy or reduce the scope of the plan using the -target flag.
╵

switch only on the pool in cb.tf

module "tf_private_pool" {
  source = "./modules/cb-private-pool"

  project_id = module.tf_source.cloudbuild_project_id

  private_worker_pool = {
    #region                   = var.default_region,
    region                   = "northamerica-northeast2",
    enable_network_peering   = true,
    create_peered_network    = true,
    peered_network_subnet_ip = "10.3.0.0/24"
    peering_address          = "192.168.0.0"
    peering_prefix_length    = 24
  }

  vpn_configuration = {
    enable_vpn = false
  }
}

1102

  # module.tf_private_pool.module.peered_network[0].module.subnets.google_compute_subnetwork.subnetwork["northamerica-northeast1/sb-b-cbpools-northamerica-northeast1"] will be destroyed
  # (because key ["northamerica-northeast1/sb-b-cbpools-northamerica-northeast1"] is not in for_each map)
  - resource "google_compute_subnetwork" "subnetwork" {
      - creation_timestamp         = "2024-05-23T07:54:59.580-07:00" -> null
      - description                = "Peered subnet for Cloud Build private pool" -> null
      - gateway_address            = "10.3.0.1" -> null
      - id                         = "projects/prj-b-cicd-pdn7/regions/northamerica-northeast1/subnetworks/sb-b-cbpools-northamerica-northeast1" -> null
      - ip_cidr_range              = "10.3.0.0/24" -> null
      - name                       = "sb-b-cbpools-northamerica-northeast1" -> null
      - network                    = "https://www.googleapis.com/compute/v1/projects/prj-b-cicd-pdn7/global/networks/vpc-b-cbpools" -> null
      - private_ip_google_access   = true -> null
      - private_ipv6_google_access = "DISABLE_GOOGLE_ACCESS" -> null
      - project                    = "prj-b-cicd-pdn7" -> null
      - purpose                    = "PRIVATE" -> null
      - region                     = "northamerica-northeast1" -> null
      - secondary_ip_range         = [] -> null
      - self_link                  = "https://www.googleapis.com/compute/v1/projects/prj-b-cicd-pdn7/regions/northamerica-northeast1/subnetworks/sb-b-cbpools-northamerica-northeast1" -> null
      - stack_type                 = "IPV4_ONLY" -> null

      - log_config {
          - aggregation_interval = "INTERVAL_5_SEC" -> null
          - filter_expr          = "true" -> null
          - flow_sampling        = 0.5 -> null
          - metadata             = "INCLUDE_ALL_METADATA" -> null
          - metadata_fields      = [] -> null
        }
    }

  # module.tf_private_pool.module.peered_network[0].module.subnets.google_compute_subnetwork.subnetwork["northamerica-northeast2/sb-b-cbpools-northamerica-northeast2"] will be created
  + resource "google_compute_subnetwork" "subnetwork" {
      + creation_timestamp         = (known after apply)
      + description                = "Peered subnet for Cloud Build private pool"
      + external_ipv6_prefix       = (known after apply)
      + fingerprint                = (known after apply)
      + gateway_address            = (known after apply)
      + id                         = (known after apply)
      + internal_ipv6_prefix       = (known after apply)
      + ip_cidr_range              = "10.3.0.0/24"
      + ipv6_cidr_range            = (known after apply)
      + name                       = "sb-b-cbpools-northamerica-northeast2"
      + network                    = "vpc-b-cbpools"
      + private_ip_google_access   = true
      + private_ipv6_google_access = (known after apply)
      + project                    = "prj-b-cicd-pdn7"
      + purpose                    = (known after apply)
      + region                     = "northamerica-northeast2"
      + secondary_ip_range         = (known after apply)
      + self_link                  = (known after apply)
      + stack_type                 = (known after apply)

      + log_config {
          + aggregation_interval = "INTERVAL_5_SEC"
          + filter_expr          = "true"
          + flow_sampling        = 0.5
          + metadata             = "INCLUDE_ALL_METADATA"
        }
    }

Plan: 72 to add, 0 to change, 1 to destroy.

Changes to Outputs:
  + cloud_build_private_worker_pool_id = (known after apply)
  + gcs_bucket_cloudbuild_artifacts    = {
      + bootstrap = (known after apply)
      + env       = (known after apply)
      + net       = (known after apply)
      + org       = (known after apply)
      + proj      = (known after apply)
    }
  + gcs_bucket_cloudbuild_logs         = {
      + bootstrap = (known after apply)
      + env       = (known after apply)
      + net       = (known after apply)
      + org       = (known after apply)
      + proj      = (known after apply)
    }

─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

Saved the plan to: bootstrap.tfplan

To perform exactly these actions, run the following command to apply:
    terraform apply "bootstrap.tfplan"
m

72 add, 1 cb pool to destroy

michael@cloudshell:~/tef-oldev4/pbmm-on-gcp-onboarding/0-bootstrap (tef-oldev4)$ terraform apply bootstrap.tfplan
module.tf_private_pool.module.peered_network[0].module.subnets.google_compute_subnetwork.subnetwork["northamerica-northeast1/sb-b-cbpools-northamerica-northeast1"]: Destroying... [id=projects/prj-b-cicd-pdn7/regions/northamerica-northeast1/subnetworks/sb-b-cbpools-northamerica-northeast1]
module.tf_private_pool.google_cloudbuild_worker_pool.private_pool: Creating...
module.tf_private_pool.module.peered_network[0].module.subnets.google_compute_subnetwork.subnetwork["northamerica-northeast2/sb-b-cbpools-northamerica-northeast2"]: Creating...
module.tf_private_pool.module.peered_network[0].module.subnets.google_compute_subnetwork.subnetwork["northamerica-northeast1/sb-b-cbpools-northamerica-northeast1"]: Still destroying... [id=projects/prj-b-cicd-pdn7/regions/northa...s/sb-b-cbpools-northamerica-northeast1, 10s elapsed]
module.tf_private_pool.google_cloudbuild_worker_pool.private_pool: Still creating... [10s elapsed]
module.tf_private_pool.module.peered_network[0].module.subnets.google_compute_subnetwork.subnetwork["northamerica-northeast2/sb-b-cbpools-northamerica-northeast2"]: Still creating... [10s elapsed]
module.tf_private_pool.module.peered_network[0].module.subnets.google_compute_subnetwork.subnetwork["northamerica-northeast1/sb-b-cbpools-northamerica-northeast1"]: Still destroying... [id=projects/prj-b-cicd-pdn7/regions/northa...s/sb-b-cbpools-northamerica-northeast1, 20s elapsed]
module.tf_private_pool.google_cloudbuild_worker_pool.private_pool: Still creating... [20s elapsed]

till destroying... [id=projects/prj-b-cicd-pdn7/regions/northa...s/sb-b-cbpools-northamerica-northeast1, 30s elapsed]
module.tf_private_pool.google_cloudbuild_worker_pool.private_pool: Still creating... [30s elapsed]
module.tf_private_pool.module.peered_network[0].module.subnets.google_compute_subnetwork.subnetwork["northamerica-northeast1/sb-b-cbpools-northamerica-northeast1"]: Destruction complete after 32s
module.tf_private_pool.google_cloudbuild_worker_pool.private_pool: Still creating... [40s elapsed]
module.tf_private_pool.google_cloudbuild_worker_pool.private_pool: Still creating... [50s elapsed]
module.tf_private_pool.google_cloudbuild_worker_pool.private_pool: Still creating... [1m0s elapsed]

not that simple - change the cidr

module.build_terraform_image.null_resource.run_destroy_command[0]: Creation complete after 1s [id=252802963512947118]
module.build_terraform_image.null_resource.run_command[0]: Provisioning with 'local-exec'...
module.build_terraform_image.null_resource.run_command[0] (local-exec): Executing: ["/bin/sh" "-c" "PATH=/google-cloud-sdk/bin:$PATH\ngcloud beta builds triggers run  60d999c2-4ef0-4eb4-99f8-2b899aebbf6d --branch main --region northamerica-northeast1 --project prj-b-cicd-pdn7\n"]
module.tf_workspace["org"].google_storage_bucket_iam_member.log_admin: Creation complete after 4s [id=b/bkt-prj-b-cicd-pdn7-gcp-org-build-logs/roles/storage.admin/serviceAccount:sa-terraform-org@prj-b-seed-49af.iam.gserviceaccount.com]
module.tf_workspace["bootstrap"].google_storage_bucket_iam_member.log_admin: Creation complete after 4s [id=b/bkt-prj-b-cicd-pdn7-gcp-bootstrap-build-logs/roles/storage.admin/serviceAccount:sa-terraform-bootstrap@prj-b-seed-49af.iam.gserviceaccount.com]
module.tf_workspace["env"].google_storage_bucket_iam_member.log_admin: Creation complete after 4s [id=b/bkt-prj-b-cicd-pdn7-gcp-environments-build-logs/roles/storage.admin/serviceAccount:sa-terraform-env@prj-b-seed-49af.iam.gserviceaccount.com]
module.build_terraform_image.null_resource.run_command[0] (local-exec): ERROR: (gcloud.beta.builds.triggers.run) INVALID_ARGUMENT: Request contains an invalid argument.
╷
│ Error: local-exec provisioner error
│ 
│   with module.build_terraform_image.null_resource.run_command[0],
│   on .terraform/modules/build_terraform_image/main.tf line 232, in resource "null_resource" "run_command":
│  232:   provisioner "local-exec" {
│ 
│ Error running command 'PATH=/google-cloud-sdk/bin:$PATH
│ gcloud beta builds triggers run  60d999c2-4ef0-4eb4-99f8-2b899aebbf6d --branch main --region northamerica-northeast1 --project prj-b-cicd-pdn7
│ ': exit status 1. Output: ERROR: (gcloud.beta.builds.triggers.run) INVALID_ARGUMENT: Request contains an invalid argument.
│ 
╵
╷
│ Error: Error waiting to create Subnetwork: Error waiting for Creating Subnetwork: Invalid IPCidrRange: 10.3.0.0/24 conflicts with existing subnetwork 'sb-b-cbpools-northamerica-northeast1' in region 'northamerica-northeast1'.
│ 
│ 
│   with module.tf_private_pool.module.peered_network[0].module.subnets.google_compute_subnetwork.subnetwork["northamerica-northeast2/sb-b-cbpools-northamerica-northeast2"],
│   on .terraform/modules/tf_private_pool.peered_network/modules/subnets/main.tf line 28, in resource "google_compute_subnetwork" "subnetwork":
│   28: resource "google_compute_subnetwork" "subnetwork" {
│ 

replan/apply - to check that the older cb pool subnet is deleted - as creation/deletion was done in parallel

module.tf_workspace["env"].google_cloudbuild_trigger.triggers["plan"]: Refreshing state... [id=projects/prj-b-cicd-pdn7/locations/northamerica-northeast1/triggers/6d51efc4-f603-46e6-a1cd-18e9783ac364]
module.tf_workspace["bootstrap"].google_cloudbuild_trigger.triggers["plan"]: Refreshing state... [id=projects/prj-b-cicd-pdn7/locations/northamerica-northeast1/triggers/46b00b06-dcf7-45e8-a251-a5dcd5124aee]

Note: Objects have changed outside of Terraform

Terraform detected the following changes made outside of Terraform since the last "terraform apply" which may have affected this plan:

  # module.tf_workspace["bootstrap"].module.artifacts_bucket.google_storage_bucket.bucket has changed
  ~ resource "google_storage_bucket" "bucket" {
        id                          = "bkt-prj-b-cicd-pdn7-gcp-bootstrap-build-artifacts"
      + labels                      = {}
        name                        = "bkt-prj-b-cicd-pdn7-gcp-bootstrap-build-artifacts"
        # (14 unchanged attributes hidden)

        # (2 unchanged blocks hidden)
    }

  # module.tf_workspace["bootstrap"].module.log_bucket.google_storage_bucket.bucket has changed
  ~ resource "google_storage_bucket" "bucket" {
        id                          = "bkt-prj-b-cicd-pdn7-gcp-bootstrap-build-logs"
      + labels                      = {}
        name                        = "bkt-prj-b-cicd-pdn7-gcp-bootstrap-build-logs"
        # (14 unchanged attributes hidden)

        # (2 unchanged blocks hidden)
    }

  # module.tf_workspace["env"].module.artifacts_bucket.google_storage_bucket.bucket has changed
  ~ resource "google_storage_bucket" "bucket" {
        id                          = "bkt-prj-b-cicd-pdn7-gcp-environments-build-artifacts"
      + labels                      = {}
        name                        = "bkt-prj-b-cicd-pdn7-gcp-environments-build-artifacts"
        # (14 unchanged attributes hidden)

        # (2 unchanged blocks hidden)
    }

  # module.tf_workspace["env"].module.log_bucket.google_storage_bucket.bucket has changed
  ~ resource "google_storage_bucket" "bucket" {
        id                          = "bkt-prj-b-cicd-pdn7-gcp-environments-build-logs"
      + labels                      = {}
        name                        = "bkt-prj-b-cicd-pdn7-gcp-environments-build-logs"
        # (14 unchanged attributes hidden)

        # (2 unchanged blocks hidden)
    }

  # module.tf_workspace["net"].module.artifacts_bucket.google_storage_bucket.bucket has changed
  ~ resource "google_storage_bucket" "bucket" {
        id                          = "bkt-prj-b-cicd-pdn7-gcp-networks-build-artifacts"
      + labels                      = {}
        name                        = "bkt-prj-b-cicd-pdn7-gcp-networks-build-artifacts"
        # (14 unchanged attributes hidden)

        # (2 unchanged blocks hidden)
    }

  # module.tf_workspace["net"].module.log_bucket.google_storage_bucket.bucket has changed
  ~ resource "google_storage_bucket" "bucket" {
        id                          = "bkt-prj-b-cicd-pdn7-gcp-networks-build-logs"
      + labels                      = {}
        name                        = "bkt-prj-b-cicd-pdn7-gcp-networks-build-logs"
        # (14 unchanged attributes hidden)

        # (2 unchanged blocks hidden)
    }

  # module.tf_workspace["org"].module.artifacts_bucket.google_storage_bucket.bucket has changed
  ~ resource "google_storage_bucket" "bucket" {
        id                          = "bkt-prj-b-cicd-pdn7-gcp-org-build-artifacts"
      + labels                      = {}
        name                        = "bkt-prj-b-cicd-pdn7-gcp-org-build-artifacts"
        # (14 unchanged attributes hidden)

        # (2 unchanged blocks hidden)
    }

  # module.tf_workspace["org"].module.log_bucket.google_storage_bucket.bucket has changed
  ~ resource "google_storage_bucket" "bucket" {
        id                          = "bkt-prj-b-cicd-pdn7-gcp-org-build-logs"
      + labels                      = {}
        name                        = "bkt-prj-b-cicd-pdn7-gcp-org-build-logs"
        # (14 unchanged attributes hidden)

        # (2 unchanged blocks hidden)
    }

  # module.tf_workspace["proj"].module.artifacts_bucket.google_storage_bucket.bucket has changed
  ~ resource "google_storage_bucket" "bucket" {
        id                          = "bkt-prj-b-cicd-pdn7-gcp-projects-build-artifacts"
      + labels                      = {}
        name                        = "bkt-prj-b-cicd-pdn7-gcp-projects-build-artifacts"
        # (14 unchanged attributes hidden)

        # (2 unchanged blocks hidden)
    }

  # module.tf_workspace["proj"].module.log_bucket.google_storage_bucket.bucket has changed
  ~ resource "google_storage_bucket" "bucket" {
        id                          = "bkt-prj-b-cicd-pdn7-gcp-projects-build-logs"
      + labels                      = {}
        name                        = "bkt-prj-b-cicd-pdn7-gcp-projects-build-logs"
        # (14 unchanged attributes hidden)

        # (2 unchanged blocks hidden)
    }

Unless you have made equivalent changes to your configuration, or ignored the relevant attributes using ignore_changes, the following plan may include actions to
undo or respond to these changes.

─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create
-/+ destroy and then create replacement

Terraform will perform the following actions:

  # module.build_terraform_image.null_resource.run_command[0] is tainted, so must be replaced
-/+ resource "null_resource" "run_command" {
      ~ id       = "6471300985274719899" -> (known after apply)
        # (1 unchanged attribute hidden)
    }

  # module.tf_private_pool.module.peered_network[0].module.subnets.google_compute_subnetwork.subnetwork["northamerica-northeast2/sb-b-cbpools-northamerica-northeast2"] will be created
  + resource "google_compute_subnetwork" "subnetwork" {
      + creation_timestamp         = (known after apply)
      + description                = "Peered subnet for Cloud Build private pool"
      + external_ipv6_prefix       = (known after apply)
      + fingerprint                = (known after apply)
      + gateway_address            = (known after apply)
      + id                         = (known after apply)
      + internal_ipv6_prefix       = (known after apply)
      + ip_cidr_range              = "10.3.0.0/24"
      + ipv6_cidr_range            = (known after apply)
      + name                       = "sb-b-cbpools-northamerica-northeast2"
      + network                    = "vpc-b-cbpools"
      + private_ip_google_access   = true
      + private_ipv6_google_access = (known after apply)
      + project                    = "prj-b-cicd-pdn7"
      + purpose                    = (known after apply)
      + region                     = "northamerica-northeast2"
      + secondary_ip_range         = (known after apply)
      + self_link                  = (known after apply)
      + stack_type                 = (known after apply)

      + log_config {
          + aggregation_interval = "INTERVAL_5_SEC"
          + filter_expr          = "true"
          + flow_sampling        = 0.5
          + metadata             = "INCLUDE_ALL_METADATA"
        }
    }

Plan: 2 to add, 0 to change, 1 to destroy.

─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

Saved the plan to: bootstrap.tfplan

To perform exactly these actions, run the following command to apply:
    terraform apply "bootstrap.tfplan"

apply

michael@cloudshell:~/tef-oldev4/pbmm-on-gcp-onboarding/0-bootstrap (tef-oldev4)$ terraform apply bootstrap.tfplan
Acquiring state lock. This may take a few moments...
module.build_terraform_image.null_resource.run_command[0]: Destroying... [id=6471300985274719899]
module.build_terraform_image.null_resource.run_command[0]: Destruction complete after 0s
module.tf_private_pool.module.peered_network[0].module.subnets.google_compute_subnetwork.subnetwork["northamerica-northeast2/sb-b-cbpools-northamerica-northeast2"]: Creating...

module.build_terraform_image.null_resource.run_command[0]: Creating...
module.build_terraform_image.null_resource.run_command[0]: Provisioning with 'local-exec'...
module.build_terraform_image.null_resource.run_command[0] (local-exec): Executing: ["/bin/sh" "-c" "PATH=/google-cloud-sdk/bin:$PATH\ngcloud beta builds triggers run  60d999c2-4ef0-4eb4-99f8-2b899aebbf6d --branch main --region northamerica-northeast1 --project prj-b-cicd-pdn7\n"]
module.build_terraform_image.null_resource.run_command[0] (local-exec): ERROR: (gcloud.beta.builds.triggers.run) INVALID_ARGUMENT: Request contains an invalid argument.
module.tf_private_pool.module.peered_network[0].module.subnets.google_compute_subnetwork.subnetwork["northamerica-northeast2/sb-b-cbpools-northamerica-northeast2"]: Still creating... [10s elapsed]
module.tf_private_pool.module.peered_network[0].module.subnets.google_compute_subnetwork.subnetwork["northamerica-northeast2/sb-b-cbpools-northamerica-northeast2"]: Still creating... [20s elapsed]
module.tf_private_pool.module.peered_network[0].module.subnets.google_compute_subnetwork.subnetwork["northamerica-northeast2/sb-b-cbpools-northamerica-northeast2"]: Still creating... [30s elapsed]
module.tf_private_pool.module.peered_network[0].module.subnets.google_compute_subnetwork.subnetwork["northamerica-northeast2/sb-b-cbpools-northamerica-northeast2"]: Creation complete after 33s [id=projects/prj-b-cicd-pdn7/regions/northamerica-northeast2/subnetworks/sb-b-cbpools-northamerica-northeast2]
╷
│ Error: local-exec provisioner error
│ 
│   with module.build_terraform_image.null_resource.run_command[0],
│   on .terraform/modules/build_terraform_image/main.tf line 232, in resource "null_resource" "run_command":
│  232:   provisioner "local-exec" {
│ 
│ Error running command 'PATH=/google-cloud-sdk/bin:$PATH
│ gcloud beta builds triggers run  60d999c2-4ef0-4eb4-99f8-2b899aebbf6d --branch main --region northamerica-northeast1 --project prj-b-cicd-pdn7
│ ': exit status 1. Output: ERROR: (gcloud.beta.builds.triggers.run) INVALID_ARGUMENT: Request contains an invalid argument.
│ 

I broke the triggers - they point to the older regions

fix the trigger

module "tf_cloud_builder" {
  source  = "terraform-google-modules/bootstrap/google//modules/tf_cloudbuild_builder"
  version = "~> 7.0"

  project_id                   = module.tf_source.cloudbuild_project_id
  dockerfile_repo_uri          = module.tf_source.csr_repos[local.cloudbuilder_repo].url
  gar_repo_location            = var.default_region
  workflow_region              = var.default_region
  terraform_version            = local.terraform_version
  build_timeout                = "1200s"
  cb_logs_bucket_force_destroy = var.bucket_force_destroy
  #trigger_location             = var.default_region
  trigger_location             = "northamerica-northeast2"
  enable_worker_pool           = true
  worker_pool_id               = module.tf_private_pool.private_worker_pool_id
  bucket_name                  = "${var.bucket_prefix}-${module.tf_source.cloudbuild_project_id}-tf-cloudbuilder-build-logs"
}

module.tf_workspace["env"].google_cloudbuild_trigger.triggers["apply"]: Refreshing state... [id=projects/prj-b-cicd-pdn7/locations/northamerica-northeast1/triggers/fdf3de03-cf9a-4318-890c-14c8f89e6366]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
-/+ destroy and then create replacement
 <= read (data resources)

Terraform will perform the following actions:

  # module.build_terraform_image.null_resource.run_command[0] is tainted, so must be replaced
-/+ resource "null_resource" "run_command" {
      ~ id       = "1011748001341205614" -> (known after apply)
      ~ triggers = {
          ~ "arguments"             = "c1764eb7969f9e9bd0ec6883c2d2e540" -> (known after apply)
          ~ "create_cmd_body"       = "beta builds triggers run  60d999c2-4ef0-4eb4-99f8-2b899aebbf6d --branch main --region northamerica-northeast1 --project prj-b-cicd-pdn7" -> (known after apply)
            # (4 unchanged elements hidden)
        }
    }

  # module.tf_cloud_builder.google_cloudbuild_trigger.build_trigger must be replaced
-/+ resource "google_cloudbuild_trigger" "build_trigger" {
      ~ create_time     = "2024-05-23T15:04:39.495860665Z" -> (known after apply)
      - disabled        = false -> null
      ~ id              = "projects/prj-b-cicd-pdn7/locations/northamerica-northeast1/triggers/60d999c2-4ef0-4eb4-99f8-2b899aebbf6d" -> (known after apply)
      - ignored_files   = [] -> null
      - included_files  = [] -> null
      ~ location        = "northamerica-northeast1" -> "northamerica-northeast2" # forces replacement
        name            = "tf-cloud-builder-build"
      - tags            = [] -> null
      ~ trigger_id      = "60d999c2-4ef0-4eb4-99f8-2b899aebbf6d" -> (known after apply)
        # (4 unchanged attributes hidden)

      ~ approval_config {
          ~ approval_required = false -> (known after apply)
        }

      ~ build {
          - substitutions = {} -> null
          - tags          = [] -> null
            # (3 unchanged attributes hidden)

          ~ options {
              - disk_size_gb           = 0 -> null
              - dynamic_substitutions  = false -> null
              - env                    = [] -> null
              - secret_env             = [] -> null
              - source_provenance_hash = [] -> null
                # (1 unchanged attribute hidden)
            }

          ~ step {
              - allow_exit_codes = [] -> null
              - allow_failure    = false -> null
              - env              = [] -> null
                name             = "gcr.io/cloud-builders/docker"
              - secret_env       = [] -> null
              - wait_for         = [] -> null
                # (1 unchanged attribute hidden)
            }
          ~ step {
              - allow_exit_codes = [] -> null
              - allow_failure    = false -> null
              - env              = [] -> null
                name             = "northamerica-northeast1-docker.pkg.dev/prj-b-cicd-pdn7/tf-runners/terraform:v${_TERRAFORM_FULL_VERSION}"
              - secret_env       = [] -> null
              - wait_for         = [] -> null
                # (1 unchanged attribute hidden)
            }
        }

      ~ source_to_build {
            # (3 unchanged attributes hidden)
        }
    }

  # module.tf_workspace["bootstrap"].data.google_project.cloudbuild_project[0] will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "google_project" "cloudbuild_project" {
      + auto_create_network = (known after apply)
      + billing_account     = (known after apply)
      + effective_labels    = (known after apply)
      + folder_id           = (known after apply)
      + id                  = (known after apply)
      + labels              = (known after apply)
      + name                = (known after apply)
      + number              = (known after apply)
      + org_id              = (known after apply)
      + project_id          = "prj-b-cicd-pdn7"
      + skip_delete         = (known after apply)
      + terraform_labels    = (known after apply)
    }

  # module.tf_workspace["bootstrap"].google_project_iam_member.pool_user[0] must be replaced
-/+ resource "google_project_iam_member" "pool_user" {
      ~ etag    = "BwYZIGBgggM=" -> (known after apply)
      ~ id      = "prj-b-cicd-pdn7/roles/cloudbuild.workerPoolUser/serviceAccount:633332953071@cloudbuild.gserviceaccount.com" -> (known after apply)
      ~ member  = "serviceAccount:633332953071@cloudbuild.gserviceaccount.com" -> (known after apply) # forces replacement
        # (2 unchanged attributes hidden)
    }

  # module.tf_workspace["bootstrap"].google_service_account_iam_member.cb_service_agent_impersonate[0] must be replaced
-/+ resource "google_service_account_iam_member" "cb_service_agent_impersonate" {
      ~ etag               = "BwYZIF+6Q0w=" -> (known after apply)
      ~ id                 = "projects/prj-b-seed-49af/serviceAccounts/sa-terraform-bootstrap@prj-b-seed-49af.iam.gserviceaccount.com/roles/iam.serviceAccountTokenCreator/serviceAccount:service-633332953071@gcp-sa-cloudbuild.iam.gserviceaccount.com" -> (known after apply)
      ~ member             = "serviceAccount:service-633332953071@gcp-sa-cloudbuild.iam.gserviceaccount.com" -> (known after apply) # forces replacement
        # (2 unchanged attributes hidden)
    }

  # module.tf_workspace["env"].data.google_project.cloudbuild_project[0] will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "google_project" "cloudbuild_project" {
      + auto_create_network = (known after apply)
      + billing_account     = (known after apply)
      + effective_labels    = (known after apply)
      + folder_id           = (known after apply)
      + id                  = (known after apply)
      + labels              = (known after apply)
      + name                = (known after apply)
      + number              = (known after apply)
      + org_id              = (known after apply)
      + project_id          = "prj-b-cicd-pdn7"
      + skip_delete         = (known after apply)
      + terraform_labels    = (known after apply)
    }

  # module.tf_workspace["env"].google_project_iam_member.pool_user[0] must be replaced
-/+ resource "google_project_iam_member" "pool_user" {
      ~ etag    = "BwYZIGBgggM=" -> (known after apply)
      ~ id      = "prj-b-cicd-pdn7/roles/cloudbuild.workerPoolUser/serviceAccount:633332953071@cloudbuild.gserviceaccount.com" -> (known after apply)
      ~ member  = "serviceAccount:633332953071@cloudbuild.gserviceaccount.com" -> (known after apply) # forces replacement
        # (2 unchanged attributes hidden)
    }

  # module.tf_workspace["env"].google_service_account_iam_member.cb_service_agent_impersonate[0] must be replaced
-/+ resource "google_service_account_iam_member" "cb_service_agent_impersonate" {
      ~ etag               = "BwYZIF/pUjU=" -> (known after apply)
      ~ id                 = "projects/prj-b-seed-49af/serviceAccounts/sa-terraform-env@prj-b-seed-49af.iam.gserviceaccount.com/roles/iam.serviceAccountTokenCreator/serviceAccount:service-633332953071@gcp-sa-cloudbuild.iam.gserviceaccount.com" -> (known after apply)
      ~ member             = "serviceAccount:service-633332953071@gcp-sa-cloudbuild.iam.gserviceaccount.com" -> (known after apply) # forces replacement
        # (2 unchanged attributes hidden)
    }

  # module.tf_workspace["net"].data.google_project.cloudbuild_project[0] will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "google_project" "cloudbuild_project" {
      + auto_create_network = (known after apply)
      + billing_account     = (known after apply)
      + effective_labels    = (known after apply)
      + folder_id           = (known after apply)
      + id                  = (known after apply)
      + labels              = (known after apply)
      + name                = (known after apply)
      + number              = (known after apply)
      + org_id              = (known after apply)
      + project_id          = "prj-b-cicd-pdn7"
      + skip_delete         = (known after apply)
      + terraform_labels    = (known after apply)
    }

  # module.tf_workspace["net"].google_project_iam_member.pool_user[0] must be replaced
-/+ resource "google_project_iam_member" "pool_user" {
      ~ etag    = "BwYZIGBgggM=" -> (known after apply)
      ~ id      = "prj-b-cicd-pdn7/roles/cloudbuild.workerPoolUser/serviceAccount:633332953071@cloudbuild.gserviceaccount.com" -> (known after apply)
      ~ member  = "serviceAccount:633332953071@cloudbuild.gserviceaccount.com" -> (known after apply) # forces replacement
        # (2 unchanged attributes hidden)
    }

  # module.tf_workspace["net"].google_service_account_iam_member.cb_service_agent_impersonate[0] must be replaced
-/+ resource "google_service_account_iam_member" "cb_service_agent_impersonate" {
      ~ etag               = "BwYZIF+tqoQ=" -> (known after apply)
      ~ id                 = "projects/prj-b-seed-49af/serviceAccounts/sa-terraform-net@prj-b-seed-49af.iam.gserviceaccount.com/roles/iam.serviceAccountTokenCreator/serviceAccount:service-633332953071@gcp-sa-cloudbuild.iam.gserviceaccount.com" -> (known after apply)
      ~ member             = "serviceAccount:service-633332953071@gcp-sa-cloudbuild.iam.gserviceaccount.com" -> (known after apply) # forces replacement
        # (2 unchanged attributes hidden)
    }

  # module.tf_workspace["org"].data.google_project.cloudbuild_project[0] will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "google_project" "cloudbuild_project" {
      + auto_create_network = (known after apply)
      + billing_account     = (known after apply)
      + effective_labels    = (known after apply)
      + folder_id           = (known after apply)
      + id                  = (known after apply)
      + labels              = (known after apply)
      + name                = (known after apply)
      + number              = (known after apply)
      + org_id              = (known after apply)
      + project_id          = "prj-b-cicd-pdn7"
      + skip_delete         = (known after apply)
      + terraform_labels    = (known after apply)
    }

  # module.tf_workspace["org"].google_project_iam_member.pool_user[0] must be replaced
-/+ resource "google_project_iam_member" "pool_user" {
      ~ etag    = "BwYZIGBgggM=" -> (known after apply)
      ~ id      = "prj-b-cicd-pdn7/roles/cloudbuild.workerPoolUser/serviceAccount:633332953071@cloudbuild.gserviceaccount.com" -> (known after apply)
      ~ member  = "serviceAccount:633332953071@cloudbuild.gserviceaccount.com" -> (known after apply) # forces replacement
        # (2 unchanged attributes hidden)
    }

  # module.tf_workspace["org"].google_service_account_iam_member.cb_service_agent_impersonate[0] must be replaced
-/+ resource "google_service_account_iam_member" "cb_service_agent_impersonate" {
      ~ etag               = "BwYZIF/3fBc=" -> (known after apply)
      ~ id                 = "projects/prj-b-seed-49af/serviceAccounts/sa-terraform-org@prj-b-seed-49af.iam.gserviceaccount.com/roles/iam.serviceAccountTokenCreator/serviceAccount:service-633332953071@gcp-sa-cloudbuild.iam.gserviceaccount.com" -> (known after apply)
      ~ member             = "serviceAccount:service-633332953071@gcp-sa-cloudbuild.iam.gserviceaccount.com" -> (known after apply) # forces replacement
        # (2 unchanged attributes hidden)
    }

  # module.tf_workspace["proj"].data.google_project.cloudbuild_project[0] will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "google_project" "cloudbuild_project" {
      + auto_create_network = (known after apply)
      + billing_account     = (known after apply)
      + effective_labels    = (known after apply)
      + folder_id           = (known after apply)
      + id                  = (known after apply)
      + labels              = (known after apply)
      + name                = (known after apply)
      + number              = (known after apply)
      + org_id              = (known after apply)
      + project_id          = "prj-b-cicd-pdn7"
      + skip_delete         = (known after apply)
      + terraform_labels    = (known after apply)
    }

  # module.tf_workspace["proj"].google_project_iam_member.pool_user[0] must be replaced
-/+ resource "google_project_iam_member" "pool_user" {
      ~ etag    = "BwYZIGBgggM=" -> (known after apply)
      ~ id      = "prj-b-cicd-pdn7/roles/cloudbuild.workerPoolUser/serviceAccount:633332953071@cloudbuild.gserviceaccount.com" -> (known after apply)
      ~ member  = "serviceAccount:633332953071@cloudbuild.gserviceaccount.com" -> (known after apply) # forces replacement
        # (2 unchanged attributes hidden)
    }

  # module.tf_workspace["proj"].google_service_account_iam_member.cb_service_agent_impersonate[0] must be replaced
-/+ resource "google_service_account_iam_member" "cb_service_agent_impersonate" {
      ~ etag               = "BwYZIF/3nKk=" -> (known after apply)
      ~ id                 = "projects/prj-b-seed-49af/serviceAccounts/sa-terraform-proj@prj-b-seed-49af.iam.gserviceaccount.com/roles/iam.serviceAccountTokenCreator/serviceAccount:service-633332953071@gcp-sa-cloudbuild.iam.gserviceaccount.com" -> (known after apply)
      ~ member             = "serviceAccount:service-633332953071@gcp-sa-cloudbuild.iam.gserviceaccount.com" -> (known after apply) # forces replacement
        # (2 unchanged attributes hidden)
    }

Plan: 12 to add, 0 to change, 12 to destroy.

─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

Saved the plan to: bootstrap.tfplan

To perform exactly these actions, run the following command to apply:
    terraform apply "bootstrap.tfplan"

module.tf_workspace["bootstrap"].google_project_iam_member.pool_user[0]: Creation complete after 9s [id=prj-b-cicd-pdn7/roles/cloudbuild.workerPoolUser/serviceAccount:633332953071@cloudbuild.gserviceaccount.com]
module.tf_workspace["net"].google_project_iam_member.pool_user[0]: Creation complete after 9s [id=prj-b-cicd-pdn7/roles/cloudbuild.workerPoolUser/serviceAccount:633332953071@cloudbuild.gserviceaccount.com]
╷
│ Error: local-exec provisioner error
│ 
│   with module.build_terraform_image.null_resource.run_command[0],
│   on .terraform/modules/build_terraform_image/main.tf line 232, in resource "null_resource" "run_command":
│  232:   provisioner "local-exec" {
│ 
│ Error running command 'PATH=/google-cloud-sdk/bin:$PATH
│ gcloud beta builds triggers run  7b0bcc41-c4f0-4170-9e83-c4b9d437f19b --branch main --region northamerica-northeast1 --project prj-b-cicd-pdn7
│ ': exit status 1. Output: ERROR: (gcloud.beta.builds.triggers.run) NOT_FOUND: Requested entity was not found.
│ 
╵

will use us-east4 for now

obriensystems commented 5 months ago
Screenshot 2024-05-23 at 11 16 53
Cloud Build API
Thank you for submitting Case # (ID:51378311) to Google Cloud Platform support for the following quota:
Change Private Pools per region - northamerica-northeast1 from 1 to 2
Your request is being processed and you should receive an email confirmation for your request. Should you need further assistance, you can respond to that email. You can also track the status of this request [here](https://console.cloud.google.com/iam-admin/quotas/qirs?project=prj-b-cicd-pdn7).

Hello, Thank you for contacting Google Cloud Platform Support. This message is to confirm that we've received your quota request for project '633332953071'. Quota increase requests typically take two business days to process.
obriensystems commented 5 months ago

No time - delete both projects, tfstate and restart with us-east4

Screenshot 2024-05-23 at 11 24 43

just in time - quota approved

Hello, Your quota request for project '633332953071' has been approved and your quota has been adjusted accordingly. The following quotas were increased: +-----------------------+--------------------------------+-------------------------+-----------------+ | Name | Dimensions | Location | Requested Limit | +-----------------------+--------------------------------+-------------------------+-----------------+ | PrivatePoolsPerRegion | region=northamerica-northeast1 | northamerica-northeast1 | 2 | +-----------------------+--------------------------------+-------------------------+-----------------+ Please visit https://console.cloud.google.com/iam-admin/quotas?project=633332953071&service=cloudbuild.googleapis.com to review your updated quota.

revert to nane1 default region - plan / apply


Plan: 14 to add, 10 to change, 14 to destroy.

Changes to Outputs:
  ~ cloud_build_private_worker_pool_id = "projects/prj-b-cicd-pdn7/locations/northamerica-northeast2/workerPools/private-pool-gdo0" -> (known after apply)

──────────────────────────────────────

michael@cloudshell:~/tef-oldev4/pbmm-on-gcp-onboarding/0-bootstrap (tef-oldev4)$ terraform apply bootstrap.tfplan
Acquiring state lock. This may take a few moments...
module.build_terraform_image.null_resource.run_command[0]: Destroying... [id=785760541234411319]
module.build_terraform_image.null_resource.run_command[0]: Destruction complete after 0s
module.tf_workspace["proj"].google_service_account_iam_member.cb_service_agent_impersonate[0]: Destroying... [id=projects/prj-b-seed-49af/serviceAccounts/sa-terraform-proj@prj-b-seed-49af.iam.gserviceaccount.com/roles/iam.serviceAccountTokenCreator/serviceAccount:service-633332953071@gcp-sa-cloudbuild.iam.gserviceaccount.com]
module.tf_workspace["org"].google_service_account_iam_member.cb_service_agent_impersonate[0]: Destroying... [id=projects/prj-b-seed-49af/serviceAccounts/sa-terraform-org@prj-b-seed-49af.iam.gserviceaccount.com/roles/iam.serviceAccountTokenCreator/serviceAccount:service-633332953071@gcp-sa-cloudbuild.iam.gserviceaccount.com]
module.tf_workspace["org"].google_project_iam_member.pool_user[0]: Destroying... [id=prj-b-cicd-pdn7/roles/cloudbuild.workerPoolUser/serviceAccount:633332953071@cloudbuild.gserviceaccount.com]
module.tf_workspace["net"].google_project_iam_member.pool_user[0]: Destroying... [id=prj-b-cicd-pdn7/roles/cloudbuild.workerPoolUser/serviceAccount:633332953071@cloudbuild.gserviceaccount.com]
module.tf_workspace["net"].google_service_account_iam_member.cb_service_agent_impersonate[0]: Destroying... [id=projects/prj-b-seed-49af/serviceAccounts/sa-terraform-net@prj-b-seed-49af.iam.gserviceaccount.com/roles/iam.serviceAccountTokenCreator/serviceAccount:service-633332953071@gcp-sa-cloudbuild.iam.gserviceaccount.com]
module.tf_workspace["proj"].google_project_iam_member.pool_user[0]: Destroying... [id=prj-b-cicd-pdn7/roles/cloudbuild.workerPoolUser/serviceAccount:633332953071@cloudbuild.gserviceaccount.com]
module.tf_workspace["bootstrap"].google_project_iam_member.pool_user[0]: Destroying... [id=prj-b-cicd-pdn7/roles/cloudbuild.workerPoolUser/serviceAccount:633332953071@cloudbuild.gserviceaccount.com]
module.tf_private_pool.module.peered_network[0].module.subnets.google_compute_subnetwork.subnetwork["northamerica-northeast2/sb-b-cbpools-northamerica-northeast2"]: Destroying... [id=projects/prj-b-cicd-pdn7/regions/northamerica-northeast2/subnetworks/sb-b-cbpools-northamerica-northeast2]
module.tf_workspace["env"].google_project_iam_member.pool_user[0]: Destroying... [id=prj-b-cicd-pdn7/roles/cloudbuild.workerPoolUser/serviceAccount:633332953071@cloudbuild.gserviceaccount.com]
module.tf_workspace["env"].google_service_account_iam_member.cb_service_agent_impersonate[0]: Destroying... [id=projects/prj-b-seed-49af/serviceAccounts/sa-terraform-env@prj-b-seed-49af.iam.gserviceaccount.com/roles/iam.serviceAccountTokenCreator/serviceAccount:service-633332953071@gcp-sa-cloudbuild.iam.gserviceaccount.com]
module.tf_workspace["net"].google_service_account_iam_member.cb_service_agent_impersonate[0]: Destruction complete after 4s
module.tf_workspace["org"].google_service_account_iam_member.cb_service_agent_impersonate[0]: Destruction complete after 4s
module.tf_workspace["env"].google_service_account_iam_member.cb_service_agent_impersonate[0]: Destruction complete after 4s
module.tf_workspace["proj"].google_service_account_iam_member.cb_service_agent_impersonate[0]: Destruction complete after 5s
module.tf_workspace["bootstrap"].google_service_account_iam_member.cb_service_agent_impersonate[0]: Destroying... [id=projects/prj-b-seed-49af/serviceAccounts/sa-terraform-bootstrap@prj-b-seed-49af.iam.gserviceaccount.com/roles/iam.serviceAccountTokenCreator/serviceAccount:service-633332953071@gcp-sa-cloudbuild.iam.gserviceaccount.com]
module.tf_private_pool.module.peered_network[0].module.subnets.google_compute_subnetwork.subnetwork["northamerica-northeast1/sb-b-cbpools-northamerica-northeast1"]: Creating...
module.tf_workspace["bootstrap"].google_project_iam_member.pool_user[0]: Destruction complete after 8s
module.tf_workspace["org"].google_project_iam_member.pool_user[0]: Destruction complete after 8s
module.tf_workspace["net"].google_project_iam_member.pool_user[0]: Destruction complete after 8s
module.tf_workspace["env"].google_project_iam_member.pool_user[0]: Destruction complete after 8s
module.tf_workspace["proj"].google_project_iam_member.pool_user[0]: Destruction complete after 8s
module.tf_workspace["bootstrap"].google_service_account_iam_member.cb_service_agent_impersonate[0]: Destruction complete after 4s
module.tf_cloud_builder.google_cloudbuild_trigger.build_trigger: Destroying... [id=projects/prj-b-cicd-pdn7/locations/northamerica-northeast2/triggers/7b0bcc41-c4f0-4170-9e83-c4b9d437f19b]
module.tf_cloud_builder.google_cloudbuild_trigger.build_trigger: Destruction complete after 0s
module.tf_private_pool.google_cloudbuild_worker_pool.private_pool: Destroying... [id=projects/prj-b-cicd-pdn7/locations/northamerica-northeast2/workerPools/private-pool-gdo0]
module.tf_private_pool.module.peered_network[0].module.subnets.google_compute_subnetwork.subnetwork["northamerica-northeast2/sb-b-cbpools-northamerica-northeast2"]: Still destroying... [id=projects/prj-b-cicd-pdn7/regions/northa...s/sb-b-cbpools-northamerica-northeast2, 10s elapsed]
module.tf_private_pool.module.peered_network[0].module.subnets.google_compute_subnetwork.subnetwork["northamerica-northeast1/sb-b-cbpools-northamerica-northeast1"]: Still creating... [10s elapsed]
module.tf_private_pool.google_cloudbuild_worker_pool.private_pool: Still destroying... [id=projects/prj-b-cicd-pdn7/locations/nort...rtheast2/workerPools/private-pool-gdo0, 10s elapsed]
module.tf_private_pool.module.peered_network[0].module.subnets.google_compute_subnetwork.subnetwork["northamerica-northeast2/sb-b-cbpools-northamerica-northeast2"]: Still destroying... [id=projects/prj-b-cicd-pdn7/regions/northa...s/sb-b-cbpools-northamerica-northeast2, 20s elapsed]
module.tf_private_pool.google_cloudbuild_worker_pool.private_pool: Still destroying... [id=projects/prj-b-cicd-pdn7/locations/nort...rtheast2/workerPools/private-pool-gdo0, 20s elapsed]
module.tf_private_pool.module.peered_network[0].module.subnets.google_compute_subnetwork.subnetwork["northamerica-northeast2/sb-b-cbpools-northamerica-northeast2"]: Still destroying... [id=projects/prj-b-cicd-pdn7/regions/northa...s/sb-b-cbpools-northamerica-northeast2, 30s elapsed]
module.tf_private_pool.module.peered_network[0].module.subnets.google_compute_subnetwork.subnetwork["northamerica-northeast2/sb-b-cbpools-northamerica-northeast2"]: Destruction complete after 32s
module.tf_private_pool.google_cloudbuild_worker_pool.private_pool: Still destroying... [id=projects/prj-b-cicd-pdn7/locations/nort...rtheast2/workerPools/private-pool-gdo0, 30s elapsed]
module.tf_private_pool.google_cloudbuild_worker_pool.private_pool: Still destroying... [id=projects/prj-b-cicd-pdn7/locations/nort...rtheast2/workerPools/private-pool-gdo0, 40s elapsed]
module.tf_private_pool.google_cloudbuild_worker_pool.private_pool: Destruction complete after 40s
module.tf_private_pool.google_cloudbuild_worker_pool.private_pool: Creating...

michael@cloudshell:~/tef-oldev4/pbmm-on-gcp-onboarding/0-bootstrap (tef-oldev4)$ terraform apply bootstrap.tfplan
Acquiring state lock. This may take a few moments...
module.build_terraform_image.null_resource.run_command[0]: Destroying... [id=785760541234411319]
module.build_terraform_image.null_resource.run_command[0]: Destruction complete after 0s
module.tf_workspace["proj"].google_service_account_iam_member.cb_service_agent_impersonate[0]: Destroying... [id=projects/prj-b-seed-49af/serviceAccounts/sa-terraform-proj@prj-b-seed-49af.iam.gserviceaccount.com/roles/iam.serviceAccountTokenCreator/serviceAccount:service-633332953071@gcp-sa-cloudbuild.iam.gserviceaccount.com]
module.tf_workspace["org"].google_service_account_iam_member.cb_service_agent_impersonate[0]: Destroying... [id=projects/prj-b-seed-49af/serviceAccounts/sa-terraform-org@prj-b-seed-49af.iam.gserviceaccount.com/roles/iam.serviceAccountTokenCreator/serviceAccount:service-633332953071@gcp-sa-cloudbuild.iam.gserviceaccount.com]
module.tf_workspace["org"].google_project_iam_member.pool_user[0]: Destroying... [id=prj-b-cicd-pdn7/roles/cloudbuild.workerPoolUser/serviceAccount:633332953071@cloudbuild.gserviceaccount.com]
module.tf_workspace["net"].google_project_iam_member.pool_user[0]: Destroying... [id=prj-b-cicd-pdn7/roles/cloudbuild.workerPoolUser/serviceAccount:633332953071@cloudbuild.gserviceaccount.com]
module.tf_workspace["net"].google_service_account_iam_member.cb_service_agent_impersonate[0]: Destroying... [id=projects/prj-b-seed-49af/serviceAccounts/sa-terraform-net@prj-b-seed-49af.iam.gserviceaccount.com/roles/iam.serviceAccountTokenCreator/serviceAccount:service-633332953071@gcp-sa-cloudbuild.iam.gserviceaccount.com]
module.tf_workspace["proj"].google_project_iam_member.pool_user[0]: Destroying... [id=prj-b-cicd-pdn7/roles/cloudbuild.workerPoolUser/serviceAccount:633332953071@cloudbuild.gserviceaccount.com]
module.tf_workspace["bootstrap"].google_project_iam_member.pool_user[0]: Destroying... [id=prj-b-cicd-pdn7/roles/cloudbuild.workerPoolUser/serviceAccount:633332953071@cloudbuild.gserviceaccount.com]
module.tf_private_pool.module.peered_network[0].module.subnets.google_compute_subnetwork.subnetwork["northamerica-northeast2/sb-b-cbpools-northamerica-northeast2"]: Destroying... [id=projects/prj-b-cicd-pdn7/regions/northamerica-northeast2/subnetworks/sb-b-cbpools-northamerica-northeast2]
module.tf_workspace["env"].google_project_iam_member.pool_user[0]: Destroying... [id=prj-b-cicd-pdn7/roles/cloudbuild.workerPoolUser/serviceAccount:633332953071@cloudbuild.gserviceaccount.com]
module.tf_workspace["env"].google_service_account_iam_member.cb_service_agent_impersonate[0]: Destroying... [id=projects/prj-b-seed-49af/serviceAccounts/sa-terraform-env@prj-b-seed-49af.iam.gserviceaccount.com/roles/iam.serviceAccountTokenCreator/serviceAccount:service-633332953071@gcp-sa-cloudbuild.iam.gserviceaccount.com]
module.tf_workspace["net"].google_service_account_iam_member.cb_service_agent_impersonate[0]: Destruction complete after 4s
module.tf_workspace["org"].google_service_account_iam_member.cb_service_agent_impersonate[0]: Destruction complete after 4s
module.tf_workspace["env"].google_service_account_iam_member.cb_service_agent_impersonate[0]: Destruction complete after 4s
module.tf_workspace["proj"].google_service_account_iam_member.cb_service_agent_impersonate[0]: Destruction complete after 5s
module.tf_workspace["bootstrap"].google_service_account_iam_member.cb_service_agent_impersonate[0]: Destroying... [id=projects/prj-b-seed-49af/serviceAccounts/sa-terraform-bootstrap@prj-b-seed-49af.iam.gserviceaccount.com/roles/iam.serviceAccountTokenCreator/serviceAccount:service-633332953071@gcp-sa-cloudbuild.iam.gserviceaccount.com]
module.tf_private_pool.module.peered_network[0].module.subnets.google_compute_subnetwork.subnetwork["northamerica-northeast1/sb-b-cbpools-northamerica-northeast1"]: Creating...
module.tf_workspace["bootstrap"].google_project_iam_member.pool_user[0]: Destruction complete after 8s
module.tf_workspace["org"].google_project_iam_member.pool_user[0]: Destruction complete after 8s
module.tf_workspace["net"].google_project_iam_member.pool_user[0]: Destruction complete after 8s
module.tf_workspace["env"].google_project_iam_member.pool_user[0]: Destruction complete after 8s
module.tf_workspace["proj"].google_project_iam_member.pool_user[0]: Destruction complete after 8s
module.tf_workspace["bootstrap"].google_service_account_iam_member.cb_service_agent_impersonate[0]: Destruction complete after 4s
module.tf_cloud_builder.google_cloudbuild_trigger.build_trigger: Destroying... [id=projects/prj-b-cicd-pdn7/locations/northamerica-northeast2/triggers/7b0bcc41-c4f0-4170-9e83-c4b9d437f19b]
module.tf_cloud_builder.google_cloudbuild_trigger.build_trigger: Destruction complete after 0s
module.tf_private_pool.google_cloudbuild_worker_pool.private_pool: Destroying... [id=projects/prj-b-cicd-pdn7/locations/northamerica-northeast2/workerPools/private-pool-gdo0]
module.tf_private_pool.module.peered_network[0].module.subnets.google_compute_subnetwork.subnetwork["northamerica-northeast2/sb-b-cbpools-northamerica-northeast2"]: Still destroying... [id=projects/prj-b-cicd-pdn7/regions/northa...s/sb-b-cbpools-northamerica-northeast2, 10s elapsed]
module.tf_private_pool.module.peered_network[0].module.subnets.google_compute_subnetwork.subnetwork["northamerica-northeast1/sb-b-cbpools-northamerica-northeast1"]: Still creating... [10s elapsed]
module.tf_private_pool.google_cloudbuild_worker_pool.private_pool: Still destroying... [id=projects/prj-b-cicd-pdn7/locations/nort...rtheast2/workerPools/private-pool-gdo0, 10s elapsed]
module.tf_private_pool.module.peered_network[0].module.subnets.google_compute_subnetwork.subnetwork["northamerica-northeast2/sb-b-cbpools-northamerica-northeast2"]: Still destroying... [id=projects/prj-b-cicd-pdn7/regions/northa...s/sb-b-cbpools-northamerica-northeast2, 20s elapsed]
module.tf_private_pool.google_cloudbuild_worker_pool.private_pool: Still destroying... [id=projects/prj-b-cicd-pdn7/locations/nort...rtheast2/workerPools/private-pool-gdo0, 20s elapsed]
module.tf_private_pool.module.peered_network[0].module.subnets.google_compute_subnetwork.subnetwork["northamerica-northeast2/sb-b-cbpools-northamerica-northeast2"]: Still destroying... [id=projects/prj-b-cicd-pdn7/regions/northa...s/sb-b-cbpools-northamerica-northeast2, 30s elapsed]
module.tf_private_pool.module.peered_network[0].module.subnets.google_compute_subnetwork.subnetwork["northamerica-northeast2/sb-b-cbpools-northamerica-northeast2"]: Destruction complete after 32s
module.tf_private_pool.google_cloudbuild_worker_pool.private_pool: Still destroying... [id=projects/prj-b-cicd-pdn7/locations/nort...rtheast2/workerPools/private-pool-gdo0, 30s elapsed]
module.tf_private_pool.google_cloudbuild_worker_pool.private_pool: Still destroying... [id=projects/prj-b-cicd-pdn7/locations/nort...rtheast2/workerPools/private-pool-gdo0, 40s elapsed]
module.tf_private_pool.google_cloudbuild_worker_pool.private_pool: Destruction complete after 40s
module.tf_private_pool.google_cloudbuild_worker_pool.private_pool: Creating...

module.tf_workspace["env"].google_project_iam_member.pool_user[0]: Creation complete after 8s [id=prj-b-cicd-pdn7/roles/cloudbuild.workerPoolUser/serviceAccount:633332953071@cloudbuild.gserviceaccount.com]
module.tf_workspace["net"].google_project_iam_member.pool_user[0]: Creation complete after 7s [id=prj-b-cicd-pdn7/roles/cloudbuild.workerPoolUser/serviceAccount:633332953071@cloudbuild.gserviceaccount.com]
module.tf_workspace["org"].google_project_iam_member.pool_user[0]: Creation complete after 8s [id=prj-b-cicd-pdn7/roles/cloudbuild.workerPoolUser/serviceAccount:633332953071@cloudbuild.gserviceaccount.com]
module.tf_workspace["bootstrap"].google_project_iam_member.pool_user[0]: Creation complete after 8s [id=prj-b-cicd-pdn7/roles/cloudbuild.workerPoolUser/serviceAccount:633332953071@cloudbuild.gserviceaccount.com]
╷
│ Error: Error waiting to create Subnetwork: Error waiting for Creating Subnetwork: Invalid IPCidrRange: 10.3.0.0/24 conflicts with existing subnetwork 'sb-b-cbpools-northamerica-northeast2' in region 'northamerica-northeast2'.
│ 
│ 
│   with module.tf_private_pool.module.peered_network[0].module.subnets.google_compute_subnetwork.subnetwork["northamerica-northeast1/sb-b-cbpools-northamerica-northeast1"],
│   on .terraform/modules/tf_private_pool.peered_network/modules/subnets/main.tf line 28, in resource "google_compute_subnetwork" "subnetwork":
│   28: resource "google_compute_subnetwork" "subnetwork" {
│ 
╵
michael@cloudshell:~/tef-oldev4/pbmm-on-gcp-onboarding/0-bootstrap (tef-oldev4)$ 
Screenshot 2024-05-23 at 11 32 56 Screenshot 2024-05-23 at 11 33 13 Screenshot 2024-05-23 at 11 34 34

no subnets

replan

Plan: 1 to add, 0 to change, 0 to destroy.

michael@cloudshell:~/tef-oldev4/pbmm-on-gcp-onboarding/0-bootstrap (tef-oldev4)$ terraform apply bootstrap.tfplan
Acquiring state lock. This may take a few moments...
module.tf_private_pool.module.peered_network[0].module.subnets.google_compute_subnetwork.subnetwork["northamerica-northeast1/sb-b-cbpools-northamerica-northeast1"]: Creating...
module.tf_private_pool.module.peered_network[0].module.subnets.google_compute_subnetwork.subnetwork["northamerica-northeast1/sb-b-cbpools-northamerica-northeast1"]: Still creating... [10s elapsed]
module.tf_private_pool.module.peered_network[0].module.subnets.google_compute_subnetwork.subnetwork["northamerica-northeast1/sb-b-cbpools-northamerica-northeast1"]: Still creating... [20s elapsed]
module.tf_private_pool.module.peered_network[0].module.subnets.google_compute_subnetwork.subnetwork["northamerica-northeast1/sb-b-cbpools-northamerica-northeast1"]: Still creating... [30s elapsed]
module.tf_private_pool.module.peered_network[0].module.subnets.google_compute_subnetwork.subnetwork["northamerica-northeast1/sb-b-cbpools-northamerica-northeast1"]: Creation complete after 33s [id=projects/prj-b-cicd-pdn7/regions/northamerica-northeast1/subnetworks/sb-b-cbpools-northamerica-northeast1]

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

Outputs:

bootstrap_step_terraform_service_account_email = "sa-terraform-bootstrap@prj-b-seed-49af.iam.gserviceaccount.com"
cloud_build_peered_network_id = "projects/prj-b-cicd-pdn7/global/networks/vpc-b-cbpools"
cloud_build_private_worker_pool_id = "projects/prj-b-cicd-pdn7/locations/northamerica-northeast1/workerPools/private-pool-gdo0"
cloud_build_worker_peered_ip_range = "192.168.0.0/24"
cloud_build_worker_range_id = "projects/prj-b-cicd-pdn7/global/addresses/ga-b-cbpools-worker-pool-range"
cloud_builder_artifact_repo = "projects/prj-b-cicd-pdn7/locations/northamerica-northeast1/repositories/tf-runners"
cloudbuild_project_id = "prj-b-cicd-pdn7"
common_config = {
  "billing_account" = "019283-6F1AB5-7AD576"
  "bootstrap_folder_name" = "folders/976224166955"
  "default_region" = "northamerica-northeast1"
  "folder_prefix" = "fldr"
  "org_id" = "583675367868"
  "parent_folder" = "479872525237"
  "parent_id" = "folders/479872525237"
  "project_prefix" = "prj"
}
csr_repos = {
  "gcp-bootstrap" = {
    "id" = "projects/prj-b-cicd-pdn7/repos/gcp-bootstrap"
    "name" = "gcp-bootstrap"
    "project" = "prj-b-cicd-pdn7"
    "url" = "https://source.developers.google.com/p/prj-b-cicd-pdn7/r/gcp-bootstrap"
  }
  "gcp-environments" = {
    "id" = "projects/prj-b-cicd-pdn7/repos/gcp-environments"
    "name" = "gcp-environments"
    "project" = "prj-b-cicd-pdn7"
    "url" = "https://source.developers.google.com/p/prj-b-cicd-pdn7/r/gcp-environments"
  }
  "gcp-networks" = {
    "id" = "projects/prj-b-cicd-pdn7/repos/gcp-networks"
    "name" = "gcp-networks"
    "project" = "prj-b-cicd-pdn7"
    "url" = "https://source.developers.google.com/p/prj-b-cicd-pdn7/r/gcp-networks"
  }
  "gcp-org" = {
    "id" = "projects/prj-b-cicd-pdn7/repos/gcp-org"
    "name" = "gcp-org"
    "project" = "prj-b-cicd-pdn7"
    "url" = "https://source.developers.google.com/p/prj-b-cicd-pdn7/r/gcp-org"
  }
  "gcp-policies" = {
    "id" = "projects/prj-b-cicd-pdn7/repos/gcp-policies"
    "name" = "gcp-policies"
    "project" = "prj-b-cicd-pdn7"
    "url" = "https://source.developers.google.com/p/prj-b-cicd-pdn7/r/gcp-policies"
  }
  "gcp-projects" = {
    "id" = "projects/prj-b-cicd-pdn7/repos/gcp-projects"
    "name" = "gcp-projects"
    "project" = "prj-b-cicd-pdn7"
    "url" = "https://source.developers.google.com/p/prj-b-cicd-pdn7/r/gcp-projects"
  }
  "tf-cloudbuilder" = {
    "id" = "projects/prj-b-cicd-pdn7/repos/tf-cloudbuilder"
    "name" = "tf-cloudbuilder"
    "project" = "prj-b-cicd-pdn7"
    "url" = "https://source.developers.google.com/p/prj-b-cicd-pdn7/r/tf-cloudbuilder"
  }
}
environment_step_terraform_service_account_email = "sa-terraform-env@prj-b-seed-49af.iam.gserviceaccount.com"
gcs_bucket_cloudbuild_artifacts = {
  "bootstrap" = "bkt-prj-b-cicd-pdn7-gcp-bootstrap-build-artifacts"
  "env" = "bkt-prj-b-cicd-pdn7-gcp-environments-build-artifacts"
  "net" = "bkt-prj-b-cicd-pdn7-gcp-networks-build-artifacts"
  "org" = "bkt-prj-b-cicd-pdn7-gcp-org-build-artifacts"
  "proj" = "bkt-prj-b-cicd-pdn7-gcp-projects-build-artifacts"
}
gcs_bucket_cloudbuild_logs = {
  "bootstrap" = "bkt-prj-b-cicd-pdn7-gcp-bootstrap-build-logs"
  "env" = "bkt-prj-b-cicd-pdn7-gcp-environments-build-logs"
  "net" = "bkt-prj-b-cicd-pdn7-gcp-networks-build-logs"
  "org" = "bkt-prj-b-cicd-pdn7-gcp-org-build-logs"
  "proj" = "bkt-prj-b-cicd-pdn7-gcp-projects-build-logs"
}
gcs_bucket_tfstate = "bkt-prj-b-seed-tfstate-95e8"
networks_step_terraform_service_account_email = "sa-terraform-net@prj-b-seed-49af.iam.gserviceaccount.com"
optional_groups = tomap({
  "gcp_global_secrets_admin" = "gcp_global_secrets_admin4@obrienlabs.dev"
  "gcp_kms_admin" = "gcp_kms_admin4@obrienlabs.dev"
  "gcp_network_viewer" = "gcp_network_viewer4@obrienlabs.dev"
  "gcp_scc_admin" = "gcp_scc_admin4@obrienlabs.dev"
  "gcp_security_reviewer" = "gcp_security_reviewer4@obrienlabs.dev"
})
organization_step_terraform_service_account_email = "sa-terraform-org@prj-b-seed-49af.iam.gserviceaccount.com"
projects_gcs_bucket_tfstate = "bkt-prj-b-seed-49af-gcp-projects-tfstate"
projects_step_terraform_service_account_email = "sa-terraform-proj@prj-b-seed-49af.iam.gserviceaccount.com"
required_groups = tomap({
  "audit_data_users" = "gcp-audit-data4@obrienlabs.dev"
  "billing_data_users" = "gcp-billing-data4@obrienlabs.dev"
  "group_billing_admins" = "gcp-billing-admins4@obrienlabs.dev"
  "group_org_admins" = "gcp-organization-admins4@obrienlabs.dev"
  "monitoring_workspace_users" = "gcp-monitoring-workspace4@obrienlabs.dev"
})
seed_project_id = "prj-b-seed-49af"
Screenshot 2024-05-23 at 11 37 18 Screenshot 2024-05-23 at 11 36 34
obriensystems commented 5 months ago

proceed 0-bootstrap

michael@cloudshell:~/tef-oldev4/pbmm-on-gcp-onboarding/0-bootstrap (tef-oldev4)$ export network_step_sa=$(terraform output -raw networks_step_terraform_service_account_email)
michael@cloudshell:~/tef-oldev4/pbmm-on-gcp-onboarding/0-bootstrap (tef-oldev4)$ export projects_step_sa=$(terraform output -raw projects_step_terraform_service_account_email)
michael@cloudshell:~/tef-oldev4/pbmm-on-gcp-onboarding/0-bootstrap (tef-oldev4)$ export projects_gcs_bucket_tfstate=$(terraform output -raw projects_gcs_bucket_tfstate)
michael@cloudshell:~/tef-oldev4/pbmm-on-gcp-onboarding/0-bootstrap (tef-oldev4)$    echo "network step service account = ${network_step_sa}"
   echo "projects step service account = ${projects_step_sa}"
   echo "projects gcs bucket tfstate = ${projects_gcs_bucket_tfstate}"
network step service account = sa-terraform-net@prj-b-seed-49af.iam.gserviceaccount.com
projects step service account = sa-terraform-proj@prj-b-seed-49af.iam.gserviceaccount.com
projects gcs bucket tfstate = bkt-prj-b-seed-49af-gcp-projects-tfstate

michael@cloudshell:~/tef-oldev4/pbmm-on-gcp-onboarding/0-bootstrap (tef-oldev4)$ export cloudbuild_project_id=$(terraform output -raw cloudbuild_project_id)
michael@cloudshell:~/tef-oldev4/pbmm-on-gcp-onboarding/0-bootstrap (tef-oldev4)$  echo "cloud build project ID = ${cloudbuild_project_id}"
cloud build project ID = prj-b-cicd-pdn7

michael@cloudshell:~/tef-oldev4/pbmm-on-gcp-onboarding/0-bootstrap (tef-oldev4)$ export cloudbuild_project_id=$(terraform output -raw cloudbuild_project_id)
michael@cloudshell:~/tef-oldev4/pbmm-on-gcp-onboarding/0-bootstrap (tef-oldev4)$  echo "cloud build project ID = ${cloudbuild_project_id}"
cloud build project ID = prj-b-cicd-pdn7
michael@cloudshell:~/tef-oldev4/pbmm-on-gcp-onboarding/0-bootstrap (tef-oldev4)$ export backend_bucket=$(terraform output -raw gcs_bucket_tfstate)
michael@cloudshell:~/tef-oldev4/pbmm-on-gcp-onboarding/0-bootstrap (tef-oldev4)$  echo "backend_bucket = ${backend_bucket}"
backend_bucket = bkt-prj-b-seed-tfstate-95e8
michael@cloudshell:~/tef-oldev4/pbmm-on-gcp-onboarding/0-bootstrap (tef-oldev4)$ export backend_bucket_projects=$(terraform output -raw projects_gcs_bucket_tfstate)
michael@cloudshell:~/tef-oldev4/pbmm-on-gcp-onboarding/0-bootstrap (tef-oldev4)$ echo "backend_bucket_projects = ${backend_bucket_projects}"
backend_bucket_projects = bkt-prj-b-seed-49af-gcp-projects-tfstate
michael@cloudshell:~/tef-oldev4/pbmm-on-gcp-onboarding/0-bootstrap (tef-oldev4)$ cp backend.tf.example backend.tf
michael@cloudshell:~/tef-oldev4/pbmm-on-gcp-onboarding/0-bootstrap (tef-oldev4)$ cd ..
michael@cloudshell:~/tef-oldev4/pbmm-on-gcp-onboarding (tef-oldev4)$ for i in `find . -name 'backend.tf'`; do sed -i'' -e "s/UPDATE_ME/${backend_bucket}/" $i; done
michael@cloudshell:~/tef-oldev4/pbmm-on-gcp-onboarding (tef-oldev4)$ for i in `find . -name 'backend.tf'`; do sed -i'' -e "s/UPDATE_PROJECTS_BACKEND/${backend_bucket_projects}/" $i; done
michael@cloudshell:~/tef-oldev4/pbmm-on-gcp-onboarding (tef-oldev4)$ cd 0-bootstrap/
michael@c

michael@cloudshell:~/tef-oldev4/pbmm-on-gcp-onboarding/0-bootstrap (tef-oldev4)$ terraform init
Initializing modules...

Initializing the backend...
Acquiring state lock. This may take a few moments...
Do you want to copy existing state to the new backend?
  Pre-existing state was found while migrating the previous "local" backend to the
  newly configured "gcs" backend. No existing state was found in the newly
  configured "gcs" backend. Do you want to copy this state to the new "gcs"
  backend? Enter "yes" to copy and "no" to start with an empty state.

  Enter a value: yes

Successfully configured the backend "gcs"! Terraform will automatically
use this backend unless the backend configuration changes.

Initializing provider plugins...
- Reusing previous version of hashicorp/google from the dependency lock file
- Reusing previous version of hashicorp/random from the dependency lock file
- Reusing previous version of hashicorp/time from the dependency lock file
- Reusing previous version of hashicorp/google-beta from the dependency lock file
- Reusing previous version of hashicorp/null from the dependency lock file
- Reusing previous version of hashicorp/external from the dependency lock file
- Using previously-installed hashicorp/time v0.11.1
- Using previously-installed hashicorp/google-beta v5.30.0
- Using previously-installed hashicorp/null v3.2.2
- Using previously-installed hashicorp/external v2.3.3
- Using previously-installed hashicorp/google v5.30.0
- Using previously-installed hashicorp/random v3.6.2

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

clone using ssh

michael@cloudshell:~/tef-oldev4/pbmm-on-gcp-onboarding/0-bootstrap (tef-oldev4)$  cd ../..
michael@cloudshell:~/tef-oldev4 (tef-oldev4)$ git clone ssh://michael@obrienlabs.dev@source.developers.google.com:2022/p/prj-b-cicd-pdn7/r/gcp-policies
Cloning into 'gcp-policies'...
warning: You appear to have cloned an empty repository.

michael@cloudshell:~/tef-oldev4 (tef-oldev4)$ cd gcp-policies/
michael@cloudshell:~/tef-oldev4/gcp-policies (tef-oldev4)$ git checkout -b main
Switched to a new branch 'main'
michael@cloudshell:~/tef-oldev4/gcp-policies (tef-oldev4)$ cp -RT ../pbmm-on-gcp-onboarding/policy-library/ .
michael@cloudshell:~/tef-oldev4/gcp-policies (tef-oldev4)$ git add .
michael@cloudshell:~/tef-oldev4/gcp-policies (tef-oldev4)$ git commit -m 'Initialize policy library repo'
[main (root-commit) 7f20cb1] Initialize policy library repo
 112 files changed, 9682 insertions(+)
 create mode 100644 lib/constraints.rego
 create mode 100644 lib/util.rego
 create mode 100644 lib/util_test.rego
 create mode 100644 policies/constraints/appengine_versions.yaml
 create mode 100644 policies/constraints/bigquery_world_readable.yaml
 create mode 100644 policies/constraints/dnssec_prevent_rsasha1_ksk.yaml
 create mode 100644 policies/constraints/dnssec_prevent_rsasha1_zsk.yaml
 create mode 100644 policies/constraints/gke_allow_only_private_cluster.yaml
 create mode 100644 policies/constraints/gke_allowed_node_sa_scope.yaml
 create mode 100644 policies/constraints/gke_container_optimized_os.yaml
 create mode 100644 policies/constraints/gke_dashboard_disable.yaml
 create mode 100644 policies/constraints/gke_disable_default_service_account.yaml
 create mode 100644 policies/constraints/gke_disable_legacy_endpoints.yaml
 create mode 100644 policies/constraints/gke_enable_alias_ip_ranges.yaml
 create mode 100644 policies/constraints/gke_legacy_abac.yaml
 create mode 100644 policies/constraints/gke_master_authorized_networks_enabled.yaml
 create mode 100644 policies/constraints/gke_node_pool_auto_repair.yaml
 create mode 100644 policies/constraints/gke_node_pool_auto_upgrade.yaml
 create mode 100644 policies/constraints/gke_restrict_client_auth_methods.yaml
 create mode 100644 policies/constraints/gke_restrict_pod_traffic.yaml
 create mode 100644 policies/constraints/iam_deny_public.yaml
 create mode 100644 policies/constraints/network_enable_flow_logs.yaml
 create mode 100644 policies/constraints/network_enable_private_google_access.yaml
 create mode 100644 policies/constraints/restrict_fw_rules_rdp_world_open.yaml
 create mode 100644 policies/constraints/restrict_fw_rules_ssh_world_open.yaml
 create mode 100644 policies/constraints/restrict_fw_rules_world_open.yaml
 create mode 100644 policies/constraints/serviceusage_allow_basic_apis.yaml
 create mode 100644 policies/constraints/sql_public_ip.yaml
 create mode 100644 policies/constraints/sql_ssl.yaml
 create mode 100644 policies/constraints/storage_bucket_policy_only.yaml
 create mode 100644 policies/constraints/storage_denylist_public.yaml
 create mode 100644 policies/templates/gcp_allowed_resource_types.yaml
 create mode 100644 policies/templates/gcp_always_violates_v1.yaml
 create mode 100644 policies/templates/gcp_app_service_versions.yaml
 create mode 100644 policies/templates/gcp_appengine_location_v1.yaml
 create mode 100644 policies/templates/gcp_bigquery_cmek_encryption_v1.yaml
 create mode 100644 policies/templates/gcp_bigquery_dataset_world_readable_v1.yaml
 create mode 100644 policies/templates/gcp_bigquery_table_retention_v1.yaml
 create mode 100644 policies/templates/gcp_bq_dataset_location_v1.yaml
 create mode 100644 policies/templates/gcp_cmek_rotation_v1.yaml
 create mode 100644 policies/templates/gcp_cmek_settings_v1.yaml
 create mode 100644 policies/templates/gcp_compute_allowed_networks.yaml
 create mode 100644 policies/templates/gcp_compute_disk_resource_policies_v1.yaml
 create mode 100644 policies/templates/gcp_compute_external_ip_address.yaml
 create mode 100644 policies/templates/gcp_compute_ip_forward.yaml
 create mode 100644 policies/templates/gcp_compute_zone_v1.yaml
 create mode 100644 policies/templates/gcp_dataproc_location_v1.yaml
 create mode 100644 policies/templates/gcp_dnssec_prevent_rsasha1_v1.yaml
 create mode 100644 policies/templates/gcp_dnssec_v1.yaml
 create mode 100644 policies/templates/gcp_enforce_labels_v1.yaml
 create mode 100644 policies/templates/gcp_enforce_naming_v1.yaml
 create mode 100644 policies/templates/gcp_gke_allowed_node_sa_v1.yaml
 create mode 100644 policies/templates/gcp_gke_cluster_location.yaml
 create mode 100644 policies/templates/gcp_gke_cluster_version_v1.yaml
 create mode 100644 policies/templates/gcp_gke_container_optimized_os.yaml
 create mode 100644 policies/templates/gcp_gke_dashboard_v1.yaml
 create mode 100644 policies/templates/gcp_gke_disable_default_service_account_v1.yaml
 create mode 100644 policies/templates/gcp_gke_disable_legacy_endpoints_v1.yaml
 create mode 100644 policies/templates/gcp_gke_enable_alias_ip_ranges.yaml
 create mode 100644 policies/templates/gcp_gke_enable_private_endpoint.yaml
 create mode 100644 policies/templates/gcp_gke_enable_shielded_nodes_v1.yaml
 create mode 100644 policies/templates/gcp_gke_enable_stackdriver_kubernetes_engine_monitoring_v1.yaml
 create mode 100644 policies/templates/gcp_gke_enable_stackdriver_logging_v1.yaml
 create mode 100644 policies/templates/gcp_gke_enable_stackdriver_monitoring_v1.yaml
 create mode 100644 policies/templates/gcp_gke_enable_workload_identity_v1.yaml
 create mode 100644 policies/templates/gcp_gke_legacy_abac_v1.yaml
 create mode 100644 policies/templates/gcp_gke_master_authorized_networks_enabled_v1.yaml
 create mode 100644 policies/templates/gcp_gke_node_auto_repair_v1.yaml
 create mode 100644 policies/templates/gcp_gke_node_auto_upgrade_v1.yaml
 create mode 100644 policies/templates/gcp_gke_private_cluster_v1.yaml
 create mode 100644 policies/templates/gcp_gke_restrict_client_auth_methods_v1.yaml
 create mode 100644 policies/templates/gcp_gke_restrict_pod_traffic_v1.yaml
 create mode 100644 policies/templates/gcp_glb_external_ip_access_constraint_v1.yaml
 create mode 100644 policies/templates/gcp_iam_allow_ban_roles_v1.yaml
 create mode 100644 policies/templates/gcp_iam_allowed_bindings.yaml
 create mode 100644 policies/templates/gcp_iam_allowed_policy_member_domains.yaml
 create mode 100644 policies/templates/gcp_iam_audit_log.yaml
 create mode 100644 policies/templates/gcp_iam_custom_role_permissions_v1.yaml
 create mode 100644 policies/templates/gcp_iam_required_bindings_v1.yaml
 create mode 100644 policies/templates/gcp_iam_restrict_service_account_creation_v1.yaml
 create mode 100644 policies/templates/gcp_iam_restrict_service_account_key_age_v1.yaml
 create mode 100644 policies/templates/gcp_iam_restrict_service_account_key_type_v1.yaml
 create mode 100644 policies/templates/gcp_lb_forwarding_rules.yaml
 create mode 100644 policies/templates/gcp_network_enable_firewall_logs_v1.yaml
 create mode 100644 policies/templates/gcp_network_enable_flow_logs_v1.yaml
 create mode 100644 policies/templates/gcp_network_enable_private_google_access_v1.yaml
 create mode 100644 policies/templates/gcp_network_restrict_default_v1.yaml
 create mode 100644 policies/templates/gcp_network_routing_v1.yaml
 create mode 100644 policies/templates/gcp_resource_value_pattern_v1.yaml
 create mode 100644 policies/templates/gcp_restricted_firewall_rules_v1.yaml
 create mode 100644 policies/templates/gcp_serviceusage_allowed_services_v1.yaml
 create mode 100644 policies/templates/gcp_spanner_location_v1.yaml
 create mode 100644 policies/templates/gcp_sql_allowed_authorized_networks_v1.yaml
 create mode 100644 policies/templates/gcp_sql_backup_v1.yaml
 create mode 100644 policies/templates/gcp_sql_instance_type_v1.yaml
 create mode 100644 policies/templates/gcp_sql_location_v1.yaml
 create mode 100644 policies/templates/gcp_sql_maintenance_window_v1.yaml
 create mode 100644 policies/templates/gcp_sql_public_ip_v1.yaml
 create mode 100644 policies/templates/gcp_sql_ssl_v1.yaml
 create mode 100644 policies/templates/gcp_sql_world_readable_v1.yaml
 create mode 100644 policies/templates/gcp_storage_bucket_policy_only_v1.yaml
 create mode 100644 policies/templates/gcp_storage_bucket_retention_v1.yaml
 create mode 100644 policies/templates/gcp_storage_bucket_world_readable_v1.yaml
 create mode 100644 policies/templates/gcp_storage_cmek_encryption_v1.yaml
 create mode 100644 policies/templates/gcp_storage_location_v1.yaml
 create mode 100644 policies/templates/gcp_storage_logging_v1.yaml
 create mode 100644 policies/templates/gcp_vpc_sc_allowed_regions.yaml
 create mode 100644 policies/templates/gcp_vpc_sc_ensure_access_levels_v1.yaml
 create mode 100644 policies/templates/gcp_vpc_sc_ensure_project_v1.yaml
 create mode 100644 policies/templates/gcp_vpc_sc_ensure_services_v1.yaml
 create mode 100644 policies/templates/gcp_vpc_sc_ip_range_v1.yaml
 create mode 100644 policies/templates/gcp_vpc_sc_project_perimeter.yaml
michael@cloudshell:~/tef-oldev4/gcp-policies (tef-oldev4)$ git push --set-upstream origin main
Enumerating objects: 118, done.
Counting objects: 100% (118/118), done.
Delta compression using up to 4 threads
Compressing objects: 100% (118/118), done.
Writing objects: 100% (118/118), 72.63 KiB | 1.96 MiB/s, done.
Total 118 (delta 87), reused 0 (delta 0), pack-reused 0
remote: Resolving deltas: 100% (87/87)
remote: Waiting for private key checker: 67/112 objects left
To ssh://obrienlabs.dev@source.developers.google.com:2022/p/prj-b-cicd-pdn7/r/gcp-policies
 * [new branch]      main -> main
Branch 'main' set up to track remote branch 'main' from 'origin'.

from

Screenshot 2024-05-23 at 11 45 15

to

Screenshot 2024-05-23 at 11 45 33

2nd repo

michael@cloudshell:~/tef-oldev4/gcp-policies (tef-oldev4)$ cd ..
michael@cloudshell:~/tef-oldev4 (tef-oldev4)$ git clone ssh://michael@obrienlabs.dev@source.developers.google.com:2022/p/prj-b-cicd-pdn7/r/gcp-bootstrap

I forgot to change the API in the script push-to-repo.sh - no SSH test done

#gcloud source repos clone "${CSR_NAME}" "${tmp_dir}" --project "${CSR_PROJECT_ID}"
git clone ssh://michael@obrienlabs.dev@source.developers.google.com:2022/p/${CSR_PROJECT_ID}/r/${CSR_NAME}
obriensystems commented 5 months ago

continued

michael@cloudshell:~/tef-oldev4 (tef-oldev4)$ cd gcp-bootstrap/
michael@cloudshell:~/tef-oldev4/gcp-bootstrap (tef-oldev4)$ git checkout -b plan
Switched to a new branch 'plan'
michael@cloudshell:~/tef-oldev4/gcp-bootstrap (tef-oldev4)$  mkdir -p envs/shared
michael@cloudshell:~/tef-oldev4/gcp-bootstrap (tef-oldev4)$ cp -RT ../pbmm-on-gcp-onboarding/0-bootstrap/ ./envs/shared
cp: error writing './envs/shared/.terraform/providers/registry.terraform.io/hashicorp/google-beta/5.30.0/linux_amd64/terraform-provider-google-beta_v5.30.0_x5': No space left on device

free some 5g space

michael@cloudshell:~/tef-oldev4/gcp-bootstrap (tef-oldev4)$    cp ../pbmm-on-gcp-onboarding/build/cloudbuild-tf-* .
michael@cloudshell:~/tef-oldev4/gcp-bootstrap (tef-oldev4)$    cp ../pbmm-on-gcp-onboarding/build/tf-wrapper.sh .
michael@cloudshell:~/tef-oldev4/gcp-bootstrap (tef-oldev4)$    chmod 755 ./tf-wrapper.sh
michael@cloudshell:~/tef-oldev4/gcp-bootstrap (tef-oldev4)$    git add .
michael@cloudshell:~/tef-oldev4/gcp-bootstrap (tef-oldev4)$    git commit -m 'Initialize bootstrap repo'
[plan (root-commit) 39b7f85] Initialize bootstrap repo
 64 files changed, 10071 insertions(+)
 create mode 100644 cloudbuild-tf-apply.yaml
 create mode 100644 cloudbuild-tf-plan.yaml
 create mode 100644 envs/shared/.gitignore
 create mode 100644 envs/shared/.terraform.lock.hcl
 create mode 100644 envs/shared/Dockerfile
 create mode 100644 envs/shared/README-GitHub.md
 create mode 100644 envs/shared/README-GitLab.md
 create mode 100644 envs/shared/README-Jenkins.md
 create mode 100644 envs/shared/README-Terraform-Cloud.md
 create mode 100644 envs/shared/README.md
 create mode 100644 envs/shared/backend.tf
 create mode 100644 envs/shared/backend.tf.cloud.example
 create mode 100644 envs/shared/backend.tf.example
 create mode 100644 envs/shared/backend.tf.local
 create mode 100644 envs/shared/bootstrap.tfplan
 create mode 100644 envs/shared/cb.tf
 create mode 100644 envs/shared/files/private_key_example.png
 create mode 100644 envs/shared/github.tf.example
 create mode 100644 envs/shared/gitlab.tf.example
 create mode 100644 envs/shared/groups.tf
 create mode 100644 envs/shared/jenkins.tf.example
 create mode 100644 envs/shared/main.tf
 create mode 100644 envs/shared/modules/cb-private-pool/README.md
 create mode 100644 envs/shared/modules/cb-private-pool/main.tf
 create mode 100644 envs/shared/modules/cb-private-pool/network.tf
 create mode 100644 envs/shared/modules/cb-private-pool/outputs.tf
 create mode 100644 envs/shared/modules/cb-private-pool/variables.tf
 create mode 100644 envs/shared/modules/cb-private-pool/versions.tf
 create mode 100644 envs/shared/modules/cb-private-pool/vpn_ha.tf
 create mode 100644 envs/shared/modules/gitlab-oidc/main.tf
 create mode 100644 envs/shared/modules/gitlab-oidc/outputs.tf
 create mode 100644 envs/shared/modules/gitlab-oidc/variables.tf
 create mode 100644 envs/shared/modules/gitlab-oidc/versions.tf
 create mode 100644 envs/shared/modules/jenkins-agent/README.md
 create mode 100755 envs/shared/modules/jenkins-agent/files/jenkins_gce_startup_script.sh
 create mode 100644 envs/shared/modules/jenkins-agent/main.tf
 create mode 100644 envs/shared/modules/jenkins-agent/outputs.tf
 create mode 100644 envs/shared/modules/jenkins-agent/variables.tf
 create mode 100644 envs/shared/modules/jenkins-agent/versions.tf
 create mode 100644 envs/shared/modules/jenkins-agent/vpn_ha.tf
 create mode 100644 envs/shared/modules/parent-iam-member/main.tf
 create mode 100644 envs/shared/modules/parent-iam-member/variables.tf
 create mode 100644 envs/shared/modules/parent-iam-member/versions.tf
 create mode 100644 envs/shared/modules/parent-iam-remove-role/main.tf
 create mode 100644 envs/shared/modules/parent-iam-remove-role/variables.tf
 create mode 100644 envs/shared/modules/parent-iam-remove-role/versions.tf
 create mode 100644 envs/shared/modules/tfc-agent-gke/README.md
 create mode 100644 envs/shared/modules/tfc-agent-gke/main.tf
 create mode 100644 envs/shared/modules/tfc-agent-gke/outputs.tf
 create mode 100644 envs/shared/modules/tfc-agent-gke/variables.tf
 create mode 100644 envs/shared/modules/tfc-agent-gke/versions.tf
 create mode 100644 envs/shared/onprem.md
 create mode 100644 envs/shared/outputs.tf
 create mode 100644 envs/shared/outputs.tf.local
 create mode 100644 envs/shared/provider.tf
 create mode 100644 envs/shared/sa.tf
 create mode 100755 envs/shared/scripts/git_create_branches_helper.sh
 create mode 100755 envs/shared/scripts/push-to-repo.sh
 create mode 100644 envs/shared/terraform-local.tf.example
 create mode 100644 envs/shared/terraform.tfvars
 create mode 100644 envs/shared/terraform_cloud.tf.example
 create mode 100644 envs/shared/variables.tf
 create mode 100644 envs/shared/versions.tf
 create mode 100755 tf-wrapper.sh
michael@cloudshell:~/tef-oldev4/gcp-bootstrap (tef-oldev4)$    git push --set-upstream origin plan
Enumerating objects: 77, done.
Counting objects: 100% (77/77), done.
Delta compression using up to 4 threads
Compressing objects: 100% (76/76), done.
Writing objects: 100% (77/77), 449.07 KiB | 8.64 MiB/s, done.
Total 77 (delta 24), reused 0 (delta 0), pack-reused 0
remote: Resolving deltas: 100% (24/24)
remote: Waiting for private key checker: 53/63 objects left
To ssh://obrienlabs.dev@source.developers.google.com:2022/p/prj-b-cicd-pdn7/r/gcp-bootstrap
 * [new branch]      plan -> plan
Branch 'plan' set up to track remote branch 'plan' from 'origin'.
michael@cloudshell:~/tef-oldev4/gcp-bootstrap (tef-oldev4)$ 
Screenshot 2024-05-23 at 12 01 07

switch region on cloud build

Screenshot 2024-05-23 at 12 02 09 Screenshot 2024-05-23 at 12 02 31
obriensystems commented 5 months ago

retest using tef-oldev4 and us-east4

michael@cloudshell:~/tef-oldev4/pbmm-on-gcp-onboarding/0-bootstrap (tef-oldev4)$ gcloud config set project prj-b-seed-49af
Updated property [core/project].
michael@cloudshell:~/tef-oldev4/pbmm-on-gcp-onboarding/0-bootstrap (prj-b-seed-49af)$ gcloud alpha resource-manager liens list
NAME: p107303979717-l2cdf41e0-625b-4952-9d82-28cd0cd5affb
ORIGIN: project-factory
REASON: Project Factory lien

NAME: p107303979717-lf4b61187-4ae6-4cd1-9b69-e4d8c5106215
ORIGIN: iam.googleapis.com/cross-project-service-accounts
REASON: IAM Cross Project Service Accounts Enabled

michael@cloudshell:~/tef-oldev4/pbmm-on-gcp-onboarding/0-bootstrap (prj-b-seed-49af)$ gcloud alpha resource-manager liens delete p107303979717-lf4b61187-4ae6-4cd1-9b69-e4d8c5106215
Deleted [liens/p107303979717-lf4b61187-4ae6-4cd1-9b69-e4d8c5106215].

and

michael@cloudshell:~/tef-oldev4/pbmm-on-gcp-onboarding/0-bootstrap (prj-b-seed-49af)$ gcloud alpha resource-manager liens list
NAME: p107303979717-l2cdf41e0-625b-4952-9d82-28cd0cd5affb
ORIGIN: project-factory
REASON: Project Factory lien
michael@cloudshell:~/tef-oldev4/pbmm-on-gcp-onboarding/0-bootstrap (prj-b-seed-49af)$ gcloud alpha resource-manager liens delete p107303979717-l2cdf41e0-625b-4952-9d82-28cd0cd5affb
Deleted [liens/p107303979717-l2cdf41e0-625b-4952-9d82-28cd0cd5affb].
obriensystems commented 5 months ago

rerun as 5

#gcloud source repos clone "${CSR_NAME}" "${tmp_dir}" --project "${CSR_PROJECT_ID}"
# no project name until runtime - let it fail once
git clone ssh://michael@obrienlabs.dev@source.developers.google.com:2022/p/prj-b-cicd-pdn7/r/gcp-bootstrap

terraform apply

expected
╷
│ Error: Error applying IAM policy for folder "folders/479872525237": Error setting IAM policy for folder "folders/479872525237": googleapi: Error 400: Group gcp-organization-admins5@obrienlabs.dev does not exist., badRequest
│ 
│   with module.seed_bootstrap.google_folder_iam_member.org_admin_serviceusage_consumer[0],
│   on .terraform/modules/seed_bootstrap/main.tf line 267, in resource "google_folder_iam_member" "org_admin_serviceusage_consumer":
│  267: resource "google_folder_iam_member" "org_admin_serviceusage_consumer" {

plan

      + tf-cloudbuilder  = {
          + id      = (known after apply)
          + name    = "tf-cloudbuilder"
          + project = "prj-b-cicd-1oi4"
          + url     = (known after apply)

edit push-to-repo-sh

git clone ssh://michael@obrienlabs.dev@source.developers.google.com:2022/p/prj-b-cicd-1oi4/r/gcp-bootstrap

apply 2 to be safe on sh change
michael@cloudshell:~/tef-oldev4/pbmm-on-gcp-onboarding/0-bootstrap (tef-oldev4)$ terraform plan -input=false -out bootstrap.tfplan

apply
michael@cloudshell:~/tef-oldev4/pbmm-on-gcp-onboarding/0-bootstrap (tef-oldev4)$ terraform apply bootstrap.tfplan

odule.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): Executing: ["/bin/sh" "-c" "PATH=/google-cloud-sdk/bin:$PATH\n./scripts/push-to-repo.sh prj-b-cicd-1oi4 tf-cloudbuilder ./Dockerfile\n"]
google_sourcerepo_repository_iam_member.member["proj"]: Creating...
module.tf_cloud_builder.google_service_account.cb_sa[0]: Creating...
module.tf_cloud_builder.google_artifact_registry_repository.tf-image-repo: Creating...
google_sourcerepo_repository_iam_member.member["bootstrap"]: Creating...
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + '[' 3 -lt 3 ']'
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + CSR_PROJECT_ID=prj-b-cicd-1oi4
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + CSR_NAME=tf-cloudbuilder
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + DOCKERFILE_PATH=./Dockerfile
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): ++ mktemp -d
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + tmp_dir=/tmp/tmp.gnROelm0WC
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + git clone ssh://michael@obrienlabs.dev@source.developers.google.com:2022/p/prj-b-cicd-1oi4/r/gcp-bootstrap
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): Cloning into 'gcp-bootstrap'...
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): warning: You appear to have cloned an empty repository.
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + cp ./Dockerfile /tmp/tmp.gnROelm0WC
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + pushd /tmp/tmp.gnROelm0WC
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): /tmp/tmp.gnROelm0WC ~/tef-oldev4/pbmm-on-gcp-onboarding/0-bootstrap
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + git config credential.helper gcloud.sh
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): fatal: not in a git directory
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/artifactregistry.admin"]: Creating...

expected errors on the not required credential helper - will fix later

module.tf_private_pool.module.peered_network[0].module.subnets.google_compute_subnetwork.subnetwork["us-east4/sb-b-cbpools-us-east4"]: Creation complete after 23s [id=projects/prj-b-cicd-1oi4/regions/us-east4/subnetworks/sb-b-cbpools-us-east4]
module.tf_private_pool.google_service_networking_connection.worker_pool_conn[0]: Still creating... [20s elapsed]

1244

forgot to use the repo variable git clone ssh://michael@obrienlabs.dev@source.developers.google.com:2022/p/prj-b-cicd-1oi4/r/gcp-bootstrap

n/serviceAccount:sa-terraform-net@prj-b-seed-322e.iam.gserviceaccount.com]
module.tf_workspace["bootstrap"].google_storage_bucket_iam_member.log_admin: Creation complete after 4s [id=b/bkt-prj-b-cicd-1oi4-gcp-bootstrap-build-logs/roles/storage.admin/serviceAccount:sa-terraform-bootstrap@prj-b-seed-322e.iam.gserviceaccount.com]
╷
│ Error: local-exec provisioner error
│ 
│   with module.bootstrap_csr_repo.null_resource.run_command[0],
│   on .terraform/modules/bootstrap_csr_repo/main.tf line 232, in resource "null_resource" "run_command":
│  232:   provisioner "local-exec" {
│ 
│ Error running command 'PATH=/google-cloud-sdk/bin:$PATH
│ ./scripts/push-to-repo.sh prj-b-cicd-1oi4 tf-cloudbuilder ./Dockerfile
│ ': exit status 128. Output: + '[' 3 -lt 3 ']'
│ + CSR_PROJECT_ID=prj-b-cicd-1oi4
│ + CSR_NAME=tf-cloudbuilder
│ + DOCKERFILE_PATH=./Dockerfile
│ ++ mktemp -d
│ + tmp_dir=/tmp/tmp.gnROelm0WC
│ + git clone ssh://michael@obrienlabs.dev@source.developers.google.com:2022/p/prj-b-cicd-1oi4/r/gcp-bootstrap
│ Cloning into 'gcp-bootstrap'...
│ warning: You appear to have cloned an empty repository.
│ + cp ./Dockerfile /tmp/tmp.gnROelm0WC
│ + pushd /tmp/tmp.gnROelm0WC
│ /tmp/tmp.gnROelm0WC ~/tef-oldev4/pbmm-on-gcp-onboarding/0-bootstrap
│ + git config credential.helper gcloud.sh
│ fatal: not in a git directory
│ 

adjusting script

git clone ssh://michael@obrienlabs.dev@source.developers.google.com:2022/p/prj-b-cicd-1oi4/r/${CSR_NAME}
cp "${DOCKERFILE_PATH}" "${tmp_dir}"
pushd "${tmp_dir}"
#git config credential.helper gcloud.sh

Plan: 5 to add, 0 to change, 1 to destroy.

odule.bootstrap_csr_repo.null_resource.run_command[0]: Creating...
module.bootstrap_csr_repo.null_resource.run_command[0]: Provisioning with 'local-exec'...
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): Executing: ["/bin/sh" "-c" "PATH=/google-cloud-sdk/bin:$PATH\n./scripts/push-to-repo.sh prj-b-cicd-1oi4 tf-cloudbuilder ./Dockerfile\n"]
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + '[' 3 -lt 3 ']'
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + CSR_PROJECT_ID=prj-b-cicd-1oi4
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + CSR_NAME=tf-cloudbuilder
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + DOCKERFILE_PATH=./Dockerfile
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): ++ mktemp -d
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + tmp_dir=/tmp/tmp.LOlygK6mYp
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + git clone ssh://michael@obrienlabs.dev@source.developers.google.com:2022/p/prj-b-cicd-1oi4/r/tf-cloudbuilder
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): Cloning into 'tf-cloudbuilder'...
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): warning: You appear to have cloned an empty repository.
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + cp ./Dockerfile /tmp/tmp.LOlygK6mYp
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + pushd /tmp/tmp.LOlygK6mYp
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): /tmp/tmp.LOlygK6mYp ~/tef-oldev4/pbmm-on-gcp-onboarding/0-bootstrap
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + git config init.defaultBranch main
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): fatal: not in a git directory
╷
│ Error: local-exec provisioner error
│ 
│   with module.bootstrap_csr_repo.null_resource.run_command[0],
│   on .terraform/modules/bootstrap_csr_repo/main.tf line 232, in resource "null_resource" "run_command":
│  232:   provisioner "local-exec" {
│ 
│ Error running command 'PATH=/google-cloud-sdk/bin:$PATH
│ ./scripts/push-to-repo.sh prj-b-cicd-1oi4 tf-cloudbuilder ./Dockerfile
│ ': exit status 128. Output: + '[' 3 -lt 3 ']'
│ + CSR_PROJECT_ID=prj-b-cicd-1oi4
│ + CSR_NAME=tf-cloudbuilder
│ + DOCKERFILE_PATH=./Dockerfile
│ ++ mktemp -d
│ + tmp_dir=/tmp/tmp.LOlygK6mYp
│ + git clone ssh://michael@obrienlabs.dev@source.developers.google.com:2022/p/prj-b-cicd-1oi4/r/tf-cloudbuilder
│ Cloning into 'tf-cloudbuilder'...
│ warning: You appear to have cloned an empty repository.
│ + cp ./Dockerfile /tmp/tmp.LOlygK6mYp
│ + pushd /tmp/tmp.LOlygK6mYp
│ /tmp/tmp.LOlygK6mYp ~/tef-oldev4/pbmm-on-gcp-onboarding/0-bootstrap
│ + git config init.defaultBranch main
│ fatal: not in a git directory
│ 

dont need

git clone ssh://michael@obrienlabs.dev@source.developers.google.com:2022/p/prj-b-cicd-1oi4/r/${CSR_NAME} "${tmp_dir}"

#git config init.defaultBranch main
#git config user.email "terraform-robot@example.com"
#git config user.name "TF Robot"
michael@cloudshell:~/tef-oldev4/pbmm-on-gcp-onboarding/0-bootstrap (tef-oldev4)$ terraform apply bootstrap.tfplan
Acquiring state lock. This may take a few moments...
module.bootstrap_csr_repo.null_resource.run_command[0]: Destroying... [id=6044269765929482659]
module.bootstrap_csr_repo.null_resource.run_command[0]: Destruction complete after 0s
module.bootstrap_csr_repo.null_resource.run_command[0]: Creating...
module.bootstrap_csr_repo.null_resource.run_command[0]: Provisioning with 'local-exec'...
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): Executing: ["/bin/sh" "-c" "PATH=/google-cloud-sdk/bin:$PATH\n./scripts/push-to-repo.sh prj-b-cicd-1oi4 tf-cloudbuilder ./Dockerfile\n"]
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + '[' 3 -lt 3 ']'
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + CSR_PROJECT_ID=prj-b-cicd-1oi4
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + CSR_NAME=tf-cloudbuilder
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + DOCKERFILE_PATH=./Dockerfile
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): ++ mktemp -d
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + tmp_dir=/tmp/tmp.w2KS5OLoxY
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + git clone ssh://michael@obrienlabs.dev@source.developers.google.com:2022/p/prj-b-cicd-1oi4/r/tf-cloudbuilder /tmp/tmp.w2KS5OLoxY
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): Cloning into '/tmp/tmp.w2KS5OLoxY'...
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): warning: You appear to have cloned an empty repository.
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + cp ./Dockerfile /tmp/tmp.w2KS5OLoxY
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + pushd /tmp/tmp.w2KS5OLoxY
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): /tmp/tmp.w2KS5OLoxY ~/tef-oldev4/pbmm-on-gcp-onboarding/0-bootstrap
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + git checkout main
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): error: pathspec 'main' did not match any file(s) known to git
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + git checkout -b main
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): Switched to a new branch 'main'
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + git add Dockerfile
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + git commit -m 'Initialize tf dockerfile repo'
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): [main (root-commit) 6ade77a] Initialize tf dockerfile repo
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec):  1 file changed, 39 insertions(+)
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec):  create mode 100644 Dockerfile
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + git push origin main -f
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): remote: Waiting for private key checker: 1/1 objects left
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): To ssh://obrienlabs.dev@source.developers.google.com:2022/p/prj-b-cicd-1oi4/r/tf-cloudbuilder
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec):  * [new branch]      main -> main
module.bootstrap_csr_repo.null_resource.run_command[0]: Creation complete after 2s [id=2382314262542131856]
time_sleep.cloud_builder: Creating...

time_sleep.cloud_builder: Still creating... [20s elapsed]
time_sleep.cloud_builder: Still creating... [30s elapsed]
time_sleep.cloud_builder: Creation complete after 30s [id=2024-05-23T16:56:14Z]
module.build_terraform_image.null_resource.module_depends_on[0]: Creating...
module.build_terraform_image.null_resource.module_depends_on[0]: Creation complete after 0s [id=6859923724488767261]
module.build_terraform_image.null_resource.run_destroy_command[0]: Creating...
module.build_terraform_image.null_resource.run_command[0]: Creating...
module.build_terraform_image.null_resource.run_destroy_command[0]: Creation complete after 0s [id=5838933205176940769]
module.build_terraform_image.null_resource.run_command[0]: Provisioning with 'local-exec'...
module.build_terraform_image.null_resource.run_command[0] (local-exec): Executing: ["/bin/sh" "-c" "PATH=/google-cloud-sdk/bin:$PATH\ngcloud beta builds triggers run  60f9c2b8-3e42-422b-9b0c-86b7bc04ab2d --branch main --region us-east4 --project prj-b-cicd-1oi4\n"]
module.build_terraform_image.null_resource.run_command[0] (local-exec): metadata:
module.build_terraform_image.null_resource.run_command[0] (local-exec):   '@type': type.googleapis.com/google.devtools.cloudbuild.v1.BuildOperationMetadata
module.build_terraform_image.null_resource.run_command[0] (local-exec):   build:
module.build_terraform_image.null_resource.run_command[0] (local-exec):     artifacts:
module.build_terraform_image.null_resource.run_command[0] (local-exec):       images:
module.build_terraform_image.null_resource.run_command[0] (local-exec):       - us-east4-docker.pkg.dev/prj-b-cicd-1oi4/tf-runners/terraform:v1.3.0
module.build_terraform_image.null_resource.run_command[0] (local-exec):       - us-east4-docker.pkg.dev/prj-b-cicd-1oi4/tf-runners/terraform:v1
module.build_terraform_image.null_resource.run_command[0] (local-exec):       - us-east4-docker.pkg.dev/prj-b-cicd-1oi4/tf-runners/terraform:v1.3
module.build_terraform_image.null_resource.run_command[0] (local-exec):     buildTriggerId: 60f9c2b8-3e42-422b-9b0c-86b7bc04ab2d
module.build_terraform_image.null_resource.run_command[0] (local-exec):     createTime: '2024-05-23T16:56:16.360017Z'
module.build_terraform_image.null_resource.run_command[0] (local-exec):     id: 94ba40ab-b6d4-4a49-b5ee-6cd30e4089a0
module.build_terraform_image.null_resource.run_command[0] (local-exec):     images:
module.build_terraform_image.null_resource.run_command[0] (local-exec):     - us-east4-docker.pkg.dev/prj-b-cicd-1oi4/tf-runners/terraform:v1.3.0
module.build_terraform_image.null_resource.run_command[0] (local-exec):     - us-east4-docker.pkg.dev/prj-b-cicd-1oi4/tf-runners/terraform:v1
module.build_terraform_image.null_resource.run_command[0] (local-exec):     - us-east4-docker.pkg.dev/prj-b-cicd-1oi4/tf-runners/terraform:v1.3
module.build_terraform_image.null_resource.run_command[0] (local-exec):     logUrl: https://console.cloud.google.com/cloud-build/builds;region=us-east4/94ba40ab-b6d4-4a49-b5ee-6cd30e4089a0?project=706705603726
module.build_terraform_image.null_resource.run_command[0] (local-exec):     logsBucket: gs://bkt-prj-b-cicd-1oi4-tf-cloudbuilder-build-logs
module.build_terraform_image.null_resource.run_command[0] (local-exec):     name: projects/706705603726/locations/us-east4/builds/94ba40ab-b6d4-4a49-b5ee-6cd30e4089a0
module.build_terraform_image.null_resource.run_command[0] (local-exec):     options:
module.build_terraform_image.null_resource.run_command[0] (local-exec):       dynamicSubstitutions: true
module.build_terraform_image.null_resource.run_command[0] (local-exec):       logging: LEGACY
module.build_terraform_image.null_resource.run_command[0] (local-exec):       pool:
module.build_terraform_image.null_resource.run_command[0] (local-exec):         name: projects/prj-b-cicd-1oi4/locations/us-east4/workerPools/private-pool-z1w6
module.build_terraform_image.null_resource.run_command[0] (local-exec):       substitutionOption: ALLOW_LOOSE
module.build_terraform_image.null_resource.run_command[0] (local-exec):     projectId: prj-b-cicd-1oi4
module.build_terraform_image.null_resource.run_command[0] (local-exec):     queueTtl: 3600s
module.build_terraform_image.null_resource.run_command[0] (local-exec):     serviceAccount: projects/prj-b-cicd-1oi4/serviceAccounts/tf-cb-builder-sa@prj-b-cicd-1oi4.iam.gserviceaccount.com
module.build_terraform_image.null_resource.run_command[0] (local-exec):     source:
module.build_terraform_image.null_resource.run_command[0] (local-exec):       repoSource:
module.build_terraform_image.null_resource.run_command[0] (local-exec):         commitSha: 6ade77a1e3e97830c6eff1afae5ebae87ab9d32a
module.build_terraform_image.null_resource.run_command[0] (local-exec):         projectId: prj-b-cicd-1oi4
module.build_terraform_image.null_resource.run_command[0] (local-exec):         repoName: tf-cloudbuilder
module.build_terraform_image.null_resource.run_command[0] (local-exec):     sourceProvenance:
module.build_terraform_image.null_resource.run_command[0] (local-exec):       resolvedRepoSource:
module.build_terraform_image.null_resource.run_command[0] (local-exec):         commitSha: 6ade77a1e3e97830c6eff1afae5ebae87ab9d32a
module.build_terraform_image.null_resource.run_command[0] (local-exec):         projectId: prj-b-cicd-1oi4
module.build_terraform_image.null_resource.run_command[0] (local-exec):         repoName: tf-cloudbuilder
module.build_terraform_image.null_resource.run_command[0] (local-exec):     status: QUEUED
module.build_terraform_image.null_resource.run_command[0] (local-exec):     steps:
module.build_terraform_image.null_resource.run_command[0] (local-exec):     - args:
module.build_terraform_image.null_resource.run_command[0] (local-exec):       - build
module.build_terraform_image.null_resource.run_command[0] (local-exec):       - --tag=us-east4-docker.pkg.dev/prj-b-cicd-1oi4/tf-runners/terraform:v1.3.0
module.build_terraform_image.null_resource.run_command[0] (local-exec):       - --tag=us-east4-docker.pkg.dev/prj-b-cicd-1oi4/tf-runners/terraform:v1
module.build_terraform_image.null_resource.run_command[0] (local-exec):       - --tag=us-east4-docker.pkg.dev/prj-b-cicd-1oi4/tf-runners/terraform:v1.3
module.build_terraform_image.null_resource.run_command[0] (local-exec):       - --build-arg=TERRAFORM_VERSION=1.3.0
module.build_terraform_image.null_resource.run_command[0] (local-exec):       - .
module.build_terraform_image.null_resource.run_command[0] (local-exec):       name: gcr.io/cloud-builders/docker
module.build_terraform_image.null_resource.run_command[0] (local-exec):     - args:
module.build_terraform_image.null_resource.run_command[0] (local-exec):       - version
module.build_terraform_image.null_resource.run_command[0] (local-exec):       name: us-east4-docker.pkg.dev/prj-b-cicd-1oi4/tf-runners/terraform:v1.3.0
module.build_terraform_image.null_resource.run_command[0] (local-exec):     substitutions:
module.build_terraform_image.null_resource.run_command[0] (local-exec):       BRANCH_NAME: main
module.build_terraform_image.null_resource.run_command[0] (local-exec):       COMMIT_SHA: 6ade77a1e3e97830c6eff1afae5ebae87ab9d32a
module.build_terraform_image.null_resource.run_command[0] (local-exec):       REF_NAME: main
module.build_terraform_image.null_resource.run_command[0] (local-exec):       REPO_NAME: tf-cloudbuilder
module.build_terraform_image.null_resource.run_command[0] (local-exec):       REVISION_ID: 6ade77a1e3e97830c6eff1afae5ebae87ab9d32a
module.build_terraform_image.null_resource.run_command[0] (local-exec):       SHORT_SHA: 6ade77a
module.build_terraform_image.null_resource.run_command[0] (local-exec):       TRIGGER_BUILD_CONFIG_PATH: ''
module.build_terraform_image.null_resource.run_command[0] (local-exec):       TRIGGER_NAME: tf-cloud-builder-build
module.build_terraform_image.null_resource.run_command[0] (local-exec):       _TERRAFORM_FULL_VERSION: 1.3.0
module.build_terraform_image.null_resource.run_command[0] (local-exec):       _TERRAFORM_MAJOR_VERSION: '1'
module.build_terraform_image.null_resource.run_command[0] (local-exec):       _TERRAFORM_MINOR_VERSION: '1.3'
module.build_terraform_image.null_resource.run_command[0] (local-exec):     tags:
module.build_terraform_image.null_resource.run_command[0] (local-exec):     - trigger-60f9c2b8-3e42-422b-9b0c-86b7bc04ab2d
module.build_terraform_image.null_resource.run_command[0] (local-exec):     timeout: 1200s
module.build_terraform_image.null_resource.run_command[0] (local-exec): name: operations/build/prj-b-cicd-1oi4/OTRiYTQwYWItYjZkNC00YTQ5LWI1ZWUtNmNkMzBlNDA4OWEw
module.build_terraform_image.null_resource.run_command[0]: Creation complete after 2s [id=9044421779404560526]

Apply complete! Resources: 5 added, 0 changed, 1 destroyed.

Outputs:

bootstrap_step_terraform_service_account_email = "sa-terraform-bootstrap@prj-b-seed-322e.iam.gserviceaccount.com"
cloud_build_peered_network_id = "projects/prj-b-cicd-1oi4/global/networks/vpc-b-cbpools"
cloud_build_private_worker_pool_id = "projects/prj-b-cicd-1oi4/locations/us-east4/workerPools/private-pool-z1w6"
cloud_build_worker_peered_ip_range = "192.168.0.0/24"
cloud_build_worker_range_id = "projects/prj-b-cicd-1oi4/global/addresses/ga-b-cbpools-worker-pool-range"
cloud_builder_artifact_repo = "projects/prj-b-cicd-1oi4/locations/us-east4/repositories/tf-runners"
cloudbuild_project_id = "prj-b-cicd-1oi4"
common_config = {
  "billing_account" = "019283-6F1AB5-7AD576"
  "bootstrap_folder_name" = "folders/650369275016"
  "default_region" = "us-east4"
  "folder_prefix" = "fldr"
  "org_id" = "583675367868"
  "parent_folder" = "479872525237"
  "parent_id" = "folders/479872525237"
  "project_prefix" = "prj"
}
csr_repos = {
  "gcp-bootstrap" = {
    "id" = "projects/prj-b-cicd-1oi4/repos/gcp-bootstrap"
    "name" = "gcp-bootstrap"
    "project" = "prj-b-cicd-1oi4"
    "url" = "https://source.developers.google.com/p/prj-b-cicd-1oi4/r/gcp-bootstrap"
  }
  "gcp-environments" = {
    "id" = "projects/prj-b-cicd-1oi4/repos/gcp-environments"
    "name" = "gcp-environments"
    "project" = "prj-b-cicd-1oi4"
    "url" = "https://source.developers.google.com/p/prj-b-cicd-1oi4/r/gcp-environments"
  }
  "gcp-networks" = {
    "id" = "projects/prj-b-cicd-1oi4/repos/gcp-networks"
    "name" = "gcp-networks"
    "project" = "prj-b-cicd-1oi4"
    "url" = "https://source.developers.google.com/p/prj-b-cicd-1oi4/r/gcp-networks"
  }
  "gcp-org" = {
    "id" = "projects/prj-b-cicd-1oi4/repos/gcp-org"
    "name" = "gcp-org"
    "project" = "prj-b-cicd-1oi4"
    "url" = "https://source.developers.google.com/p/prj-b-cicd-1oi4/r/gcp-org"
  }
  "gcp-policies" = {
    "id" = "projects/prj-b-cicd-1oi4/repos/gcp-policies"
    "name" = "gcp-policies"
    "project" = "prj-b-cicd-1oi4"
    "url" = "https://source.developers.google.com/p/prj-b-cicd-1oi4/r/gcp-policies"
  }
  "gcp-projects" = {
    "id" = "projects/prj-b-cicd-1oi4/repos/gcp-projects"
    "name" = "gcp-projects"
    "project" = "prj-b-cicd-1oi4"
    "url" = "https://source.developers.google.com/p/prj-b-cicd-1oi4/r/gcp-projects"
  }
  "tf-cloudbuilder" = {
    "id" = "projects/prj-b-cicd-1oi4/repos/tf-cloudbuilder"
    "name" = "tf-cloudbuilder"
    "project" = "prj-b-cicd-1oi4"
    "url" = "https://source.developers.google.com/p/prj-b-cicd-1oi4/r/tf-cloudbuilder"
  }
}
environment_step_terraform_service_account_email = "sa-terraform-env@prj-b-seed-322e.iam.gserviceaccount.com"
gcs_bucket_cloudbuild_artifacts = {
  "bootstrap" = "bkt-prj-b-cicd-1oi4-gcp-bootstrap-build-artifacts"
  "env" = "bkt-prj-b-cicd-1oi4-gcp-environments-build-artifacts"
  "net" = "bkt-prj-b-cicd-1oi4-gcp-networks-build-artifacts"
  "org" = "bkt-prj-b-cicd-1oi4-gcp-org-build-artifacts"
  "proj" = "bkt-prj-b-cicd-1oi4-gcp-projects-build-artifacts"
}
gcs_bucket_cloudbuild_logs = {
  "bootstrap" = "bkt-prj-b-cicd-1oi4-gcp-bootstrap-build-logs"
  "env" = "bkt-prj-b-cicd-1oi4-gcp-environments-build-logs"
  "net" = "bkt-prj-b-cicd-1oi4-gcp-networks-build-logs"
  "org" = "bkt-prj-b-cicd-1oi4-gcp-org-build-logs"
  "proj" = "bkt-prj-b-cicd-1oi4-gcp-projects-build-logs"
}
gcs_bucket_tfstate = "bkt-prj-b-seed-tfstate-d36e"
networks_step_terraform_service_account_email = "sa-terraform-net@prj-b-seed-322e.iam.gserviceaccount.com"
optional_groups = tomap({
  "gcp_global_secrets_admin" = "gcp_global_secrets_admin5@obrienlabs.dev"
  "gcp_kms_admin" = "gcp_kms_admin5@obrienlabs.dev"
  "gcp_network_viewer" = "gcp_network_viewer5@obrienlabs.dev"
  "gcp_scc_admin" = "gcp_scc_admin5@obrienlabs.dev"
  "gcp_security_reviewer" = "gcp_security_reviewer5@obrienlabs.dev"
})
organization_step_terraform_service_account_email = "sa-terraform-org@prj-b-seed-322e.iam.gserviceaccount.com"
projects_gcs_bucket_tfstate = "bkt-prj-b-seed-322e-gcp-projects-tfstate"
projects_step_terraform_service_account_email = "sa-terraform-proj@prj-b-seed-322e.iam.gserviceaccount.com"
required_groups = tomap({
  "audit_data_users" = "gcp-audit-data5@obrienlabs.dev"
  "billing_data_users" = "gcp-billing-data5@obrienlabs.dev"
  "group_billing_admins" = "gcp-billing-admins5@obrienlabs.dev"
  "group_org_admins" = "gcp-organization-admins5@obrienlabs.dev"
  "monitoring_workspace_users" = "gcp-monitoring-workspace5@obrienlabs.dev"
})
seed_project_id = "prj-b-seed-322e"

working

set -ex

if [ "$#" -lt 3 ]; then
    >&2 echo "Not all expected arguments set."
    exit 1
fi

CSR_PROJECT_ID=$1
CSR_NAME=$2
DOCKERFILE_PATH=$3

# create temp dir, cleanup at exit
tmp_dir=$(mktemp -d)
# # shellcheck disable=SC2064
# trap "rm -rf $tmp_dir" EXIT
#gcloud source repos clone "${CSR_NAME}" "${tmp_dir}" --project "${CSR_PROJECT_ID}"
# no project name until runtime - let it fail once
git clone ssh://michael@obrienlabs.dev@source.developers.google.com:2022/p/prj-b-cicd-1oi4/r/${CSR_NAME} "${tmp_dir}"
cp "${DOCKERFILE_PATH}" "${tmp_dir}"
pushd "${tmp_dir}"
#git config credential.helper gcloud.sh
#git config init.defaultBranch main
#git config user.email "terraform-robot@example.com"
#git config user.name "TF Robot"
git checkout main || git checkout -b main
git add Dockerfile
git commit -m "Initialize tf dockerfile repo"
git push origin main -f

remember

gcloud source repos clone gcp-policies --project=${cloudbuild_project_id}

git clone ssh://michael@obrienlabs.dev@source.developers.google.com:2022/p/${cloudbuild_project_id}/r/gcp-policies

github-actions[bot] commented 3 months ago

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 7 days