GoogleCloudPlatform / pbmm-on-gcp-onboarding

GCP Canadian Public Sector Landing Zone overlay on top of the TEF via CFT modules - a secure cloud foundation

https://cloud.google.com/architecture/security-foundations

Apache License 2.0

45 stars 56 forks source link

Windows amd64/ia64 local gcloud client support: terraform apply via ming64/git-bash tested - expected win32 issue running bootstrap module check_env.sh works only inside docker container (ubuntu or google/cloud-sdk) - or use cloud shell #429

Closed fmichaelobrien closed 3 months ago

fmichaelobrien commented 6 months ago

Fix: run google/cloud-sdk docker container, install terraform, clone repo inside container - move plan/statefile out via -v separate shared dir to host

C:\wse_github\GoogleCloudPlatform\p6gen1\pbmm-on-gcp-onboarding>docker run -it --name gcloud -v .:/pbmm google/cloud-sdk:latest /bin/bash

   21  mkdir _pbmm_inside
   22  cd _pbmm_inside/
   23  git status
   24  git clone https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding.git
   25  cd pbmm-on-gcp-onboarding/
   26  cd 0-bootstrap/
   27  cp ../../../pbmm/0-bootstrap/terraform .
   28  cp ../../../pbmm/0-bootstrap/terraform.tfvars .
   29  rm -rf terraform.example.tfvars
   30  ls .la
   31  ls -la
   32  cp terraform ../
   33  rm -rf terraform
   34  ls
   35  ../terraform init
   36  ../terraform plan --input=false -out bootstrap.tfplan

Issue

First time a 2nd party user ran the LZ from an AVD windows vm step 4 of https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/tree/main/0-bootstrap#deploying-with-cloud-build

specifically sh on windows.

 fork/exec .terraform/modules/bootstrap_csr_repo/scripts/check_env.sh: %1 is not a valid Win32 application.

Quick fix - run ming64 and watch symlinks on #428

Reproduction: I will run the LZ from my Lenovo P1Gen6 and document in https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/wiki/DevOps#local-windows

from client

on terraform apply 
│ Error: External Program Execution Failed
│
│   with module.bootstrap_csr_repo.data.external.env_override[0],
│   on .terraform\modules\bootstrap_csr_repo\main.tf line 74, in data "external" "env_override":
│   74:   program = ["${path.module}/scripts/check_env.sh"]
│
│ The data source received an unexpected error while attempting to execute the program.
│
│ The program was executed, however it returned no additional error messaging.
│
│ Program: .terraform/modules/bootstrap_csr_repo/scripts/check_env.sh
│ State: fork/exec .terraform/modules/bootstrap_csr_repo/scripts/check_env.sh: %1 is not a valid Win32 application.

see https://registry.terraform.io/modules/terraform-google-modules/bootstrap/google/latest

for example in my last run on cloud shell (unix) we were ok in #360 The following is during apply where the sh scripts would eventually fail as well.

module.bootstrap_csr_repo.null_resource.run_command[0]: Creating...
module.tf_private_pool.module.peered_network[0].module.vpc.google_compute_network.network: Creating...
google_sourcerepo_repository_iam_member.member["org"]: Creating...
module.tf_cloud_builder.google_service_account.cb_sa[0]: Creating...
module.tf_cloud_builder.google_artifact_registry_repository.tf-image-repo: Creating...
module.bootstrap_csr_repo.null_resource.run_command[0]: Provisioning with 'local-exec'...
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): Executing: ["/bin/sh" "-c" "PATH=/google-cloud-sdk/bin:$PATH\n./scripts/push-to-repo.sh prj-b-cicd-82vv tf-cloudbuilder ./Dockerfile\n"]
google_sourcerepo_repository_iam_member.member["proj"]: Creating...
module.tf_cloud_builder.google_service_account.workflow_sa[0]: Creating...
google_sourcerepo_repository_iam_member.member["env"]: Creating...
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + '[' 3 -lt 3 ']'
module.tf_cloud_builder.module.bucket.google_storage_bucket.bucket: Creating...
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + CSR_PROJECT_ID=prj-b-cicd-82vv
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + CSR_NAME=tf-cloudbuilder
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + DOCKERFILE_PATH=./Dockerfile
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): ++ mktemp -d
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + tmp_dir=/tmp/tmp.xMNfsxhn6Q
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): + gcloud source repos clone tf-cloudbuilder /tmp/tmp.xMNfsxhn6Q --project prj-b-cicd-82vv
google_sourcerepo_repository_iam_member.member["net"]: Creating...
module.tf_cloud_builder.module.bucket.google_storage_bucket.bucket: Creation complete after 0s [id=bkt-prj-b-cicd-82vv-tf-cloudbuilder-build-logs]
google_sourcerepo_repository_iam_member.member["bootstrap"]: Creating...
module.tf_cloud_builder.google_service_account.cb_sa[0]: Creation complete after 0s [id=projects/prj-b-cicd-82vv/serviceAccounts/tf-cb-builder-sa@prj-b-cicd-82vv.iam.gserviceaccount.com]
module.cicd_project_iam_member["bootstrap"].google_project_iam_member.project_parent_iam["roles/source.admin"]: Creating...
module.bootstrap_csr_repo.null_resource.run_command[0] (local-exec): ERROR: (gcloud.source.repos.clone) UNAUTHENTICATED: Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.

Reproducing in

https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/wiki/DevOps#local-windows

triage

check https://github.com/terraform-google-modules/terraform-google-kubernetes-engine/issues/630

specifically

skip_provisioners = true
look for usage via
enabled = var.skip_provisioners

and https://github.com/terraform-google-modules/terraform-google-cloud-operations/issues/13

obriensystems commented 6 months ago

reproducing on oldev

C:\wse_github\GoogleCloudPlatform\p6gen1\pbmm-on-gcp-onboarding\0-bootstrap>terraform --version
Terraform v1.3.10
on windows_amd64

Your version of Terraform is out of date! The latest version
is 1.8.2. You can update by downloading from https://www.terraform.io/downloads.html

C:\wse_github\GoogleCloudPlatform\p6gen1\pbmm-on-gcp-onboarding\0-bootstrap>terraform init
Initializing modules...
Downloading registry.terraform.io/terraform-google-modules/gcloud/google 3.4.0 for bootstrap_csr_repo...
- bootstrap_csr_repo in .terraform\modules\bootstrap_csr_repo
- bootstrap_projects_remove_editor in modules\parent-iam-remove-role
Downloading registry.terraform.io/terraform-google-modules/gcloud/google 3.4.0 for build_terraform_image...
- build_terraform_image in .terraform\modules\build_terraform_image
- cicd_project_iam_member in modules\parent-iam-member
Downloading registry.terraform.io/terraform-google-modules/cloud-storage/google 5.0.0 for gcp_projects_state_bucket...
- gcp_projects_state_bucket in .terraform\modules\gcp_projects_state_bucket\modules\simple_bucket
Downloading registry.terraform.io/terraform-google-modules/group/google 0.6.1 for optional_group...
- optional_group in .terraform\modules\optional_group
- org_iam_member in modules\parent-iam-member
- parent_iam_member in modules\parent-iam-member
Downloading registry.terraform.io/terraform-google-modules/group/google 0.6.1 for required_group...
- required_group in .terraform\modules\required_group
Downloading registry.terraform.io/terraform-google-modules/bootstrap/google 7.2.0 for seed_bootstrap...
- seed_bootstrap in .terraform\modules\seed_bootstrap
Downloading registry.terraform.io/terraform-google-modules/org-policy/google 5.3.0 for seed_bootstrap.enable_cross_project_service_account_usage...
- seed_bootstrap.enable_cross_project_service_account_usage in .terraform\modules\seed_bootstrap.enable_cross_project_service_account_usage
Downloading registry.terraform.io/terraform-google-modules/kms/google 2.3.0 for seed_bootstrap.kms...
- seed_bootstrap.kms in .terraform\modules\seed_bootstrap.kms
Downloading registry.terraform.io/terraform-google-modules/project-factory/google 14.5.0 for seed_bootstrap.seed_project...
- seed_bootstrap.seed_project in .terraform\modules\seed_bootstrap.seed_project
- seed_bootstrap.seed_project.budget in .terraform\modules\seed_bootstrap.seed_project\modules\budget
- seed_bootstrap.seed_project.essential_contacts in .terraform\modules\seed_bootstrap.seed_project\modules\essential_contacts
- seed_bootstrap.seed_project.gsuite_group in .terraform\modules\seed_bootstrap.seed_project\modules\gsuite_group
- seed_bootstrap.seed_project.project-factory in .terraform\modules\seed_bootstrap.seed_project\modules\core_project_factory
- seed_bootstrap.seed_project.project-factory.project_services in .terraform\modules\seed_bootstrap.seed_project\modules\project_services
- seed_bootstrap.seed_project.quotas in .terraform\modules\seed_bootstrap.seed_project\modules\quota_manager
- seed_bootstrap.seed_project.shared_vpc_access in .terraform\modules\seed_bootstrap.seed_project\modules\shared_vpc_access
- seed_project_iam_member in modules\parent-iam-member
Downloading registry.terraform.io/terraform-google-modules/bootstrap/google 7.2.0 for tf_cloud_builder...
- tf_cloud_builder in .terraform\modules\tf_cloud_builder\modules\tf_cloudbuild_builder
Downloading registry.terraform.io/terraform-google-modules/cloud-storage/google 5.0.0 for tf_cloud_builder.bucket...
- tf_cloud_builder.bucket in .terraform\modules\tf_cloud_builder.bucket\modules\simple_bucket
- tf_private_pool in modules\cb-private-pool
Downloading registry.terraform.io/terraform-google-modules/network/google 9.1.0 for tf_private_pool.firewall_rules...
- tf_private_pool.firewall_rules in .terraform\modules\tf_private_pool.firewall_rules\modules\firewall-rules
Downloading registry.terraform.io/terraform-google-modules/network/google 9.1.0 for tf_private_pool.peered_network...
- tf_private_pool.peered_network in .terraform\modules\tf_private_pool.peered_network
- tf_private_pool.peered_network.firewall_rules in .terraform\modules\tf_private_pool.peered_network\modules\firewall-rules
- tf_private_pool.peered_network.routes in .terraform\modules\tf_private_pool.peered_network\modules\routes
- tf_private_pool.peered_network.subnets in .terraform\modules\tf_private_pool.peered_network\modules\subnets
- tf_private_pool.peered_network.vpc in .terraform\modules\tf_private_pool.peered_network\modules\vpc
Downloading registry.terraform.io/terraform-google-modules/vpn/google 4.0.0 for tf_private_pool.vpn_ha_cb_to_onprem...
- tf_private_pool.vpn_ha_cb_to_onprem in .terraform\modules\tf_private_pool.vpn_ha_cb_to_onprem\modules\vpn_ha
Downloading registry.terraform.io/terraform-google-modules/bootstrap/google 7.2.0 for tf_source...
- tf_source in .terraform\modules\tf_source\modules\tf_cloudbuild_source
Downloading registry.terraform.io/terraform-google-modules/cloud-storage/google 5.0.0 for tf_source.cloudbuild_bucket...
- tf_source.cloudbuild_bucket in .terraform\modules\tf_source.cloudbuild_bucket\modules\simple_bucket
Downloading registry.terraform.io/terraform-google-modules/project-factory/google 14.5.0 for tf_source.cloudbuild_project...
- tf_source.cloudbuild_project in .terraform\modules\tf_source.cloudbuild_project
- tf_source.cloudbuild_project.budget in .terraform\modules\tf_source.cloudbuild_project\modules\budget
- tf_source.cloudbuild_project.essential_contacts in .terraform\modules\tf_source.cloudbuild_project\modules\essential_contacts
- tf_source.cloudbuild_project.gsuite_group in .terraform\modules\tf_source.cloudbuild_project\modules\gsuite_group
- tf_source.cloudbuild_project.project-factory in .terraform\modules\tf_source.cloudbuild_project\modules\core_project_factory
- tf_source.cloudbuild_project.project-factory.project_services in .terraform\modules\tf_source.cloudbuild_project\modules\project_services
- tf_source.cloudbuild_project.quotas in .terraform\modules\tf_source.cloudbuild_project\modules\quota_manager
- tf_source.cloudbuild_project.shared_vpc_access in .terraform\modules\tf_source.cloudbuild_project\modules\shared_vpc_access
Downloading registry.terraform.io/terraform-google-modules/bootstrap/google 7.2.0 for tf_workspace...
- tf_workspace in .terraform\modules\tf_workspace\modules\tf_cloudbuild_workspace
Downloading registry.terraform.io/terraform-google-modules/cloud-storage/google 5.0.0 for tf_workspace.artifacts_bucket...
- tf_workspace.artifacts_bucket in .terraform\modules\tf_workspace.artifacts_bucket\modules\simple_bucket
Downloading registry.terraform.io/terraform-google-modules/cloud-storage/google 5.0.0 for tf_workspace.log_bucket...
- tf_workspace.log_bucket in .terraform\modules\tf_workspace.log_bucket\modules\simple_bucket
Downloading registry.terraform.io/terraform-google-modules/cloud-storage/google 5.0.0 for tf_workspace.state_bucket...
- tf_workspace.state_bucket in .terraform\modules\tf_workspace.state_bucket\modules\simple_bucket

Initializing the backend...

Initializing provider plugins...
- Finding hashicorp/random versions matching ">= 2.1.0, >= 2.2.0, >= 3.1.0, ~> 3.4"...
- Finding hashicorp/google-beta versions matching ">= 3.43.0, >= 3.50.0, >= 3.64.0, >= 3.67.0, >= 3.77.0, >= 4.11.0, >= 4.17.0, >= 4.28.0, != 4.31.0, >= 4.64.0, >= 5.7.0, < 6.0.0"...
- Finding hashicorp/external versions matching ">= 2.2.2"...
- Finding hashicorp/null versions matching ">= 2.1.0"...
- Finding hashicorp/google versions matching ">= 3.33.0, >= 3.43.0, >= 3.50.0, >= 3.53.0, >= 3.64.0, >= 3.67.0, >= 3.77.0, >= 3.83.0, >= 4.17.0, >= 4.25.0, >= 4.28.0, != 4.31.0, >= 4.46.0, >= 4.64.0, >= 5.7.0, < 6.0.0"...
- Finding hashicorp/time versions matching ">= 0.5.0"...
- Installing hashicorp/google v5.28.0...
- Installed hashicorp/google v5.28.0 (signed by HashiCorp)
- Installing hashicorp/time v0.11.1...
- Installed hashicorp/time v0.11.1 (signed by HashiCorp)
- Installing hashicorp/random v3.6.1...
- Installed hashicorp/random v3.6.1 (signed by HashiCorp)
- Installing hashicorp/google-beta v5.28.0...
- Installed hashicorp/google-beta v5.28.0 (signed by HashiCorp)
- Installing hashicorp/external v2.3.3...
- Installed hashicorp/external v2.3.3 (signed by HashiCorp)
- Installing hashicorp/null v3.2.2...
- Installed hashicorp/null v3.2.2 (signed by HashiCorp)

Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

terraform plan

C:\wse_github\GoogleCloudPlatform\p6gen1\pbmm-on-gcp-onboarding\0-bootstrap>terraform plan -input=false -out bootstrap.tfplan
module.build_terraform_image.data.external.env_override[0]: Reading...
module.bootstrap_csr_repo.data.external.env_override[0]: Reading...
╷
│ Error: Invalid provider configuration
│
│ Provider "registry.terraform.io/hashicorp/google" requires explicit configuration. Add a provider block to the root
│ module and configure the provider's required arguments as described in the provider documentation.
│
╵
╷
│ Error: Attempted to load application default credentials since neither `credentials` nor `access_token` was set in the provider block.  No credentials loaded. To use your gcloud credentials, run 'gcloud auth application-default login'
│
│   with provider["registry.terraform.io/hashicorp/google"],
│   on <empty> line 0:
│   (source code not available)
│
│ google: could not find default credentials. See https://cloud.google.com/docs/authentication/external/set-up-adc for
│ more information
╵
╷
│ Error: Attempted to load application default credentials since neither `credentials` nor `access_token` was set in the provider block.  No credentials loaded. To use your gcloud credentials, run 'gcloud auth application-default login'
│
│   with provider["registry.terraform.io/hashicorp/google-beta"],
│   on provider.tf line 17, in provider "google-beta":
│   17: provider "google-beta" {
│
│ google: could not find default credentials. See https://cloud.google.com/docs/authentication/external/set-up-adc for
│ more information
╵
╷
│ Error: External Program Execution Failed
│
│   with module.bootstrap_csr_repo.data.external.env_override[0],
│   on .terraform\modules\bootstrap_csr_repo\main.tf line 74, in data "external" "env_override":
│   74:   program = ["${path.module}/scripts/check_env.sh"]
│
│ The data source received an unexpected error while attempting to execute the program.
│
│ The program was executed, however it returned no additional error messaging.
│
│ Program: .terraform/modules/bootstrap_csr_repo/scripts/check_env.sh
│ State: fork/exec .terraform/modules/bootstrap_csr_repo/scripts/check_env.sh: %1 is not a valid Win32 application.
╵
╷
│ Error: External Program Execution Failed
│
│   with module.build_terraform_image.data.external.env_override[0],
│   on .terraform\modules\build_terraform_image\main.tf line 74, in data "external" "env_override":
│   74:   program = ["${path.module}/scripts/check_env.sh"]
│
│ The data source received an unexpected error while attempting to execute the program.
│
│ The program was executed, however it returned no additional error messaging.
│
│ Program: .terraform/modules/build_terraform_image/scripts/check_env.sh
│ State: fork/exec .terraform/modules/build_terraform_image/scripts/check_env.sh: %1 is not a valid Win32 application.
╵

C:\wse_github\GoogleCloudPlatform\p6gen1\pbmm-on-gcp-onboarding\0-bootstrap>

obriensystems commented 6 months ago

run terraform from git-bash

expected until adjust the path

micha@p1gen6 MINGW64 /c/wse_github/GoogleCloudPlatform/p6gen1/pbmm-on-gcp-onboarding/0-bootstrap (main)
$ gcloud config set project tef-oldev-p1gen6
Python was not found; run without arguments to install from the Microsoft Store, or disable this shortcut from Settings > Manage App Execution Aliases.

micha@p1gen6 MINGW64 /c/wse_github/GoogleCloudPlatform/p6gen1/pbmm-on-gcp-onboarding/0-bootstrap (main)
$ ./terraform --version
Terraform v1.3.10
on windows_amd64
+ provider registry.terraform.io/hashicorp/external v2.3.3
+ provider registry.terraform.io/hashicorp/google v5.28.0
+ provider registry.terraform.io/hashicorp/google-beta v5.28.0
+ provider registry.terraform.io/hashicorp/null v3.2.2
+ provider registry.terraform.io/hashicorp/random v3.6.1
+ provider registry.terraform.io/hashicorp/time v0.11.1

Your version of Terraform is out of date! The latest version
is 1.8.2. You can update by downloading from https://www.terraform.io/downloads.html

install

add to path
micha@p1gen6 MINGW64 ~
$ gcloud
Python was not found; run without arguments to install from the Microsoft Store, or disable this shortcut from Settings > Manage App Execution Aliases.

already have python 3
micha@p1gen6 MINGW64 ~
$ python --version
Python 3.12.2

micha@p1gen6 MINGW64 ~ $ cat ~/.bash_rc alias gcloud="gcloud.cmd"

add sys env variable nope

trying working

micha@p1gen6 MINGW64 ~
$ gcloud --version
Google Cloud SDK 453.0.0
bq 2.0.98
core 2023.10.27
gcloud-crc32c 1.0.0
gsutil 5.27
Updates are available for some Google Cloud CLI components.  To install them,
please run:
  $ gcloud components update

obriensystems commented 6 months ago

run terraform plan in git-bash/ming64

micha@p1gen6 MINGW64 /c/wse_github/GoogleCloudPlatform/p6gen1/pbmm-on-gcp-onboarding/0-bootstrap (main)
$ gcloud config list
[accessibility]
screen_reader = False
[core]
account = michael@obrienlabs.dev
disable_usage_reporting = False
project = tef-oldev-p1gen6

Your active configuration is: [default]

micha@p1gen6 MINGW64 /c/wse_github/GoogleCloudPlatform/p6gen1/pbmm-on-gcp-onboarding/0-bootstrap (main)
$ ./terraform.exe init
Initializing modules...

Initializing the backend...

Initializing provider plugins...
- Reusing previous version of hashicorp/google from the dependency lock file
- Reusing previous version of hashicorp/time from the dependency lock file
- Reusing previous version of hashicorp/random from the dependency lock file
- Reusing previous version of hashicorp/google-beta from the dependency lock file
- Reusing previous version of hashicorp/null from the dependency lock file
- Reusing previous version of hashicorp/external from the dependency lock file
- Using previously-installed hashicorp/google v5.28.0
- Using previously-installed hashicorp/time v0.11.1
- Using previously-installed hashicorp/random v3.6.1
- Using previously-installed hashicorp/google-beta v5.28.0
- Using previously-installed hashicorp/null v3.2.2
- Using previously-installed hashicorp/external v2.3.3

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

micha@p1gen6 MINGW64 /c/wse_github/GoogleCloudPlatform/p6gen1/pbmm-on-gcp-onboarding/0-bootstrap (main)
$

micha@p1gen6 MINGW64 /c/wse_github/GoogleCloudPlatform/p6gen1/pbmm-on-gcp-onboarding/0-bootstrap (main)
$ gcloud auth application-default login
Your browser has been opened to visit:

    https://accounts.google.com/o/oauth2/auth?response_type=code&client_id=764086051850-6qr4p6gpi6hn506pt8ejuq83di341hur.apps.googleusercontent.com&redirect_uri=http%3A%2F%2Flocalhost%3A8085%2F&scope=openid+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fuserinfo.email+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fcloud-platform+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fsqlservice.login&state=IuOmXO8rHD65SFSgx9WMrpMbSavgXR&access_type=offline&code_challenge=J3e88NrTTdbWjHRtIISJ6rD5xzS4ZFrFcNF17IB1D08&code_challenge_method=S256

Credentials saved to file: [C:\Users\micha\AppData\Roaming\gcloud\application_default_credentials.json]

These credentials will be used by any library that requests Application Default Credentials (ADC).

Quota project "tef-oldev-p1gen6" was added to ADC which can be used by Google client libraries for billing and quota. Note that some services may still bill the project owning the resource.

terraform plan

same

micha@p1gen6 MINGW64 /c/wse_github/GoogleCloudPlatform/p6gen1/pbmm-on-gcp-onboarding/0-bootstrap (main)
$ ./terraform plan -input=false -out bootstrap.tfplan
module.build_terraform_image.data.external.env_override[0]: Reading...
module.bootstrap_csr_repo.data.external.env_override[0]: Reading...
data.google_organization.org[0]: Reading...
data.google_organization.org[0]: Read complete after 0s [id=organizations/583675367868]
╷
│ Error: External Program Execution Failed
│
│   with module.bootstrap_csr_repo.data.external.env_override[0],
│   on .terraform\modules\bootstrap_csr_repo\main.tf line 74, in data "external" "env_override":
│   74:   program = ["${path.module}/scripts/check_env.sh"]
│
│ The data source received an unexpected error while attempting to execute
│ the program.
│
│ The program was executed, however it returned no additional error
│ messaging.
│
│ Program: .terraform/modules/bootstrap_csr_repo/scripts/check_env.sh
│ State: fork/exec
│ .terraform/modules/bootstrap_csr_repo/scripts/check_env.sh: %1 is not a
│ valid Win32 application.
╵
╷
│ Error: External Program Execution Failed
│
│   with module.build_terraform_image.data.external.env_override[0],
│   on .terraform\modules\build_terraform_image\main.tf line 74, in data "external" "env_override":
│   74:   program = ["${path.module}/scripts/check_env.sh"]
│
│ The data source received an unexpected error while attempting to execute
│ the program.
│
│ The program was executed, however it returned no additional error
│ messaging.
│
│ Program: .terraform/modules/build_terraform_image/scripts/check_env.sh
│ State: fork/exec
│ .terraform/modules/build_terraform_image/scripts/check_env.sh: %1 is not a
│ valid Win32 application.
╵

obriensystems commented 6 months ago

terraform docker container via rebuilt for amd64 image

https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/blob/gh399-ado/build/docker/build.sh

 docker run --name tef obrienlabs/terraform-example-foundation-ado:0.0.2-ia64 --version
micha@p1gen6 MINGW64 /c/wse_github/GoogleCloudPlatform/p6gen1/pbmm-on-gcp-onboarding/0-bootstrap (main)
$ docker run --name tef obrienlabs/terraform-example-foundation-ado:0.0.2-ia64 plan -input=false -out bootstrap.tfplan
╷
│ Error: No configuration files

Need local directory set

add to Dockerfile
WORKDIR /terraform
COPY ..

micha@p1gen6 MINGW64 /c/wse_github/GoogleCloudPlatform/p6gen1/pbmm-on-gcp-onboarding/0-bootstrap (main)
$ docker run -v $(pwd):/terraform --name tef obrienlabs/terraform-example-foundation-ado:0.0.3-ia64 init
Terraform initialized in an empty directory!

The directory has no Terraform configuration files. You may begin working
with Terraform immediately by creating Terraform configuration files.

micha@p1gen6 MINGW64 /c/wse_github/GoogleCloudPlatform/p6gen1/pbmm-on-gcp-onboarding/0-bootstrap (main)
$ docker run -v $(pwd):/terraform --name tef obrienlabs/terraform-example-foundation-ado:0.0.3-ia64 plan -input=false -out bootstrap.tfplan
docker: Error response from daemon: Conflict. The container name "/tef" is already in use by container "a517658fde1289f44cc3667a96d4206736e8ebe21175305f906c1dea1885f7fb". You have to remove (or rename) that container to be able to reuse that name.
See 'docker run --help'.

micha@p1gen6 MINGW64 /c/wse_github/GoogleCloudPlatform/p6gen1/pbmm-on-gcp-onboarding/0-bootstrap (main)
$ docker rm tef
tef

micha@p1gen6 MINGW64 /c/wse_github/GoogleCloudPlatform/p6gen1/pbmm-on-gcp-onboarding/0-bootstrap (main)
$ docker run -v $(pwd):/terraform --name tef obrienlabs/terraform-example-foundation-ado:0.0.3-ia64 plan -input=false -out bootstrap.tfplan
╷
│ Error: No configuration files
│
│ Plan requires configuration to be present. Planning without a configuration
│ would mark everything for destruction, which is normally not what is
│ desired. If you would like to destroy everything, run plan with the
│ -destroy option. Otherwise, create a Terraform configuration file (.tf
│ file) and try again.
╵

I'll need a vm or persistent container - as rerunning the terraform executable is not preserving state

obriensystems commented 6 months ago

Switch to base ubuntu image - or gcloud-slim, add terraform - run as headless container - mount 0-bootstrap dir

testing before gcloud/terraform retrofit of DockerFile

C:\wse_github\GoogleCloudPlatform\p6gen1\pbmm-on-gcp-onboarding\0-bootstrap>docker run -it --name ubuntu -v .:/bootstrap ubuntu
:18.04 /bin/bash
root@f65e48cd0e9c:/# ls boot
root@f65e48cd0e9c:/# ls bootstrap
Dockerfile                 backend.tf.local    modules                     terraform.exe
README-GitHub.md           cb.tf               onprem.md                   terraform.tfvars
README-GitLab.md           files               outputs.tf                  terraform_cloud.tf.example
README-Jenkins.md          github.tf.example   outputs.tf.local            variables.tf
README-Terraform-Cloud.md  gitlab.tf.example   provider.tf                 versions.tf
README.md                  groups.tf           sa.tf
backend.tf.cloud.example   jenkins.tf.example  scripts
backend.tf.example         main.tf             terraform-local.tf.example

C:\wse_github\GoogleCloudPlatform\p6gen1\pbmm-on-gcp-onboarding\0-bootstrap>docker exec -it ubuntu /bin/bash
root@f65e48cd0e9c:/# ls
bin   bootstrap  etc   in-docker.sh  lib64  mnt  proc  run   srv  tmp  var
boot  dev        home  lib           media  opt  root  sbin  sys  usr
root@f65e48cd0e9c:/# cd bootstrap
root@f65e48cd0e9c:/bootstrap# ls
Dockerfile                 backend.tf.local    modules                     terraform.exe
README-GitHub.md           cb.tf               onprem.md                   terraform.tfvars
README-GitLab.md           files               outputs.tf                  terraform_cloud.tf.example
README-Jenkins.md          github.tf.example   outputs.tf.local            variables.tf
README-Terraform-Cloud.md  gitlab.tf.example   provider.tf                 versions.tf
README.md                  groups.tf           sa.tf
backend.tf.cloud.example   jenkins.tf.example  scripts
backend.tf.example         main.tf             terraform-local.tf.example
root@f65e48cd0e9c:/bootstrap# touch in-docker.sh
root@f65e48cd0e9c:/bootstrap# mkdir _in-docker
root@f65e48cd0e9c:/bootstrap# ls
Dockerfile                 backend.tf.example  jenkins.tf.example  scripts
README-GitHub.md           backend.tf.local    main.tf             terraform-local.tf.example
README-GitLab.md           cb.tf               modules             terraform.exe
README-Jenkins.md          files               onprem.md           terraform.tfvars
README-Terraform-Cloud.md  github.tf.example   outputs.tf          terraform_cloud.tf.example
README.md                  gitlab.tf.example   outputs.tf.local    variables.tf
_in-docker                 groups.tf           provider.tf         versions.tf
backend.tf.cloud.example   in-docker.sh        sa.tf
root@f65e48cd0e9c:/bootstrap# exit
exit

C:\wse_github\GoogleCloudPlatform\p6gen1\pbmm-on-gcp-onboarding\0-bootstrap>ls
Dockerfile                 backend.tf.example  jenkins.tf.example  scripts
README-GitHub.md           backend.tf.local    main.tf             terraform-local.tf.example
README-GitLab.md           cb.tf               modules             terraform.exe
README-Jenkins.md          files               onprem.md           terraform.tfvars
README-Terraform-Cloud.md  github.tf.example   outputs.tf          terraform_cloud.tf.example
README.md                  gitlab.tf.example   outputs.tf.local    variables.tf
_in-docker                 groups.tf           provider.tf         versions.tf
backend.tf.cloud.example   in-docker.sh        sa.tf

obriensystems commented 6 months ago

https://hub.docker.com/r/google/cloud-sdk/

C:\wse_github\GoogleCloudPlatform\p6gen1\pbmm-on-gcp-onboarding\0-bootstrap>docker run -it --name gcloud -v .:/bootstrap google
/cloud-sdk:latest /bin/bash

C:\wse_github\GoogleCloudPlatform\p6gen1\pbmm-on-gcp-onboarding\0-bootstrap>gcloud config set project tef-oldev-p1gen6
Updated property [core/project].

C:\wse_github\GoogleCloudPlatform\p6gen1\pbmm-on-gcp-onboarding\0-bootstrap>docker exec -it gcloud /bin/bash
root@1eaecbe8f848:/# gcloud --version
Google Cloud SDK 475.0.0
alpha 2024.05.03
app-engine-go 1.9.76
app-engine-java 2.0.26
app-engine-python 1.9.113
app-engine-python-extras 1.9.106
beta 2024.05.03
bigtable
bq 2.1.4
bundled-python3-unix 3.11.8
cbt 1.19.0
cloud-datastore-emulator 2.3.1
cloud-firestore-emulator 1.19.6
cloud-spanner-emulator 1.5.16
core 2024.05.03
gcloud-crc32c 1.0.0
gke-gcloud-auth-plugin 0.5.8
gsutil 5.27
kpt 1.0.0-beta.49
kubectl 1.26.15
local-extract 1.5.9
pubsub-emulator 0.8.14
root@1eaecbe8f848:/# gcloud config get project
(unset)
root@1eaecbe8f848:/# gcloud config set project tef-oldev-p1gen6
Updated property [core/project].
root@1eaecbe8f848:/# gcloud services list
ERROR: (gcloud.services.list) You do not currently have an active account selected.
Please run:

  $ gcloud auth login

to obtain new credentials.

If you have already logged in with a different account, run:

  $ gcloud config set account ACCOUNT

to select an already authenticated account to use.
root@1eaecbe8f848:/# gcloud auth login
Go to the following link in your browser, and complete the sign-in prompts:

    https://accounts.google.com/o/oauth2/auth?response_type=code&client_id=32555940559.apps.googleusercontent.com&redirect_uri=https%3A%2F%2Fsdk.cloud.google.com%2Fauthcode.html&scope=openid+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fuserinfo.email+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fcloud-platform+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fappengine.admin+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fsqlservice.login+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fcompute+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Faccounts.reauth&state=ddcYcheHCwmKRzO8jImASecGFm1K5j&prompt=consent&token_usage=remote&access_type=offline&code_challenge=0aqf1sH9N1LFt8Kt1MGLiMG4kXvpvn9ea2c_BPNIZMg&code_challenge_method=S256

Once finished, enter the verification code provided in your browser: 4/0AdLIrYfIMeJ63E2SX5KhK8StO6y7-gC9YeK-OhcDYDqOz2tjG6m54fdydttboQEAtJzmgA

You are now logged in as [michael@obrienlabs.dev].
Your current project is [tef-oldev-p1gen6].  You can change this setting by running:
  $ gcloud config set project PROJECT_ID
root@1eaecbe8f848:/#

manually add terraform before modifying the DockerFile

https://releases.hashicorp.com/terraform/1.3.10/

root@f7be1a75a354:/bootstrap# ./terraform init
Initializing modules...

Initializing the backend...

Initializing provider plugins...
- Reusing previous version of hashicorp/google from the dependency lock file
- Reusing previous version of hashicorp/random from the dependency lock file
- Reusing previous version of hashicorp/time from the dependency lock file
- Reusing previous version of hashicorp/google-beta from the dependency lock file
- Reusing previous version of hashicorp/external from the dependency lock file
- Reusing previous version of hashicorp/null from the dependency lock file
- Installing hashicorp/random v3.6.1...
- Installed hashicorp/random v3.6.1 (signed by HashiCorp)
- Installing hashicorp/time v0.11.1...
- Installed hashicorp/time v0.11.1 (signed by HashiCorp)
- Installing hashicorp/google-beta v5.28.0...
- Installed hashicorp/google-beta v5.28.0 (signed by HashiCorp)
- Installing hashicorp/external v2.3.3...
- Installed hashicorp/external v2.3.3 (signed by HashiCorp)
- Installing hashicorp/null v3.2.2...
- Installed hashicorp/null v3.2.2 (signed by HashiCorp)
- Installing hashicorp/google v5.28.0...
- Installed hashicorp/google v5.28.0 (signed by HashiCorp)

Terraform has made some changes to the provider dependency selections recorded
in the .terraform.lock.hcl file. Review those changes and commit them to your
version control system if they represent changes you intended to make.

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
root@f7be1a75a354:/bootstrap#

root@f7be1a75a354:/bootstrap# gcloud services list
ERROR: (gcloud.services.list) You do not currently have an active account selected.
Please run:

  $ gcloud auth login

to obtain new credentials.

If you have already logged in with a different account, run:

  $ gcloud config set account ACCOUNT

to select an already authenticated account to use.
root@f7be1a75a354:/bootstrap# gcloud auth login
Go to the following link in your browser, and complete the sign-in prompts:

    https://accounts.google.com/o/oauth2/auth?response_type=code&client_id=32555940559.apps.googleusercontent.com&redirect_uri=https%3A%2F%2Fsdk.cloud.google.com%2Fauthcode.html&scope=openid+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fuserinfo.email+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fcloud-platform+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fappengine.admin+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fsqlservice.login+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fcompute+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Faccounts.reauth&state=GmtMMp0IJfHr9wMpt5KmC7V6WK4CVN&prompt=consent&token_usage=remote&access_type=offline&code_challenge=GrcpRe4AJwAluXHNh7TDoNSwidtT9UrAIy3bmrkqQKg&code_challenge_method=S256

Once finished, enter the verification code provided in your browser: 4/0AdLIrYcHWdIzvhYUmO2jr6yK_FFYUhygmZqSi5Yt5cmR5nHpkphcA7k-Qw6tBRlruZeSaA

You are now logged in as [michael@obrienlabs.dev].
Your current project is [tef-olapp-p1gen6].  You can change this setting by running:
  $ gcloud config set project PROJECT_ID
root@f7be1a75a354:/bootstrap# gcloud services list
ERROR: (gcloud.services.list) User [michael@obrienlabs.dev] does not have permission to access projects instance [tef-olapp-p1gen6] (or it may not exist): Permission denied to list services for consumer container [projects/916108113130]
Help Token: AWUw39XAUnVKznL0voTvoKiOX3dNaHV50tZK-i6ndL9iqNU7p01EL6y4CnqSQC7QzRMWyssdMHFZBLlo1Do4JYwJ1y1u9w3rNd7sKBq9VukXXCve
- '@type': type.googleapis.com/google.rpc.PreconditionFailure
  violations:
  - subject: ?error_code=110002&service=cloudresourcemanager.googleapis.com&permission=serviceusage.services.list&resource=projects/tef-olapp-p1gen6
    type: googleapis.com
- '@type': type.googleapis.com/google.rpc.ErrorInfo
  domain: serviceusage.googleapis.com
  metadata:
    permission: serviceusage.services.list
    resource: projects/tef-olapp-p1gen6
    service: cloudresourcemanager.googleapis.com
  reason: AUTH_PERMISSION_DENIED
root@f7be1a75a354:/bootstrap# ^C
root@f7be1a75a354:/bootstrap# gcloud config set project tef-olapp-p1gen6
WARNING: You do not appear to have access to project [tef-olapp-p1gen6] or it does not exist.
Are you sure you wish to set property [core/project] to tef-olapp-p1gen6?

Do you want to continue (Y/n)?  ^C

Command killed by keyboard interrupt

root@f7be1a75a354:/bootstrap# gcloud config set project tef-oldev-p1gen6
Updated property [core/project].
root@f7be1a75a354:/bootstrap# gcloud services list
NAME                                 TITLE
artifactregistry.googleapis.com      Artifact Registry API
cloudbilling.googleapis.com          Cloud Billing API
cloudbuild.googleapis.com            Cloud Build API
cloudkms.googleapis.com              Cloud Key Management Service (KMS) API
cloudresourcemanager.googleapis.com  Cloud Resource Manager API
containerregistry.googleapis.com     Container Registry API
iam.googleapis.com                   Identity and Access Management (IAM) API
iamcredentials.googleapis.com        IAM Service Account Credentials API
logging.googleapis.com               Cloud Logging API
pubsub.googleapis.com                Cloud Pub/Sub API
servicenetworking.googleapis.com     Service Networking API
storage-api.googleapis.com           Google Cloud Storage JSON API
root@f7be1a75a354:/bootstrap#

terraform plan in the container

root@f7be1a75a354:/bootstrap# ./terraform plan --input=false -out bootstrap.tfplan
module.bootstrap_csr_repo.data.external.env_override[0]: Reading...
module.build_terraform_image.data.external.env_override[0]: Reading...
╷
│ Error: Invalid provider configuration
│
│ Provider "registry.terraform.io/hashicorp/google" requires explicit configuration. Add a provider block to the root module and configure the provider's
│ required arguments as described in the provider documentation.
│
╵
╷
│ Error: Attempted to load application default credentials since neither `credentials` nor `access_token` was set in the provider block.  No credentials loaded. To use your gcloud credentials, run 'gcloud auth application-default login'
│
│   with provider["registry.terraform.io/hashicorp/google"],
│   on <empty> line 0:
│   (source code not available)
│
│ google: could not find default credentials. See https://cloud.google.com/docs/authentication/external/set-up-adc for more information
╵
╷
│ Error: Attempted to load application default credentials since neither `credentials` nor `access_token` was set in the provider block.  No credentials loaded. To use your gcloud credentials, run 'gcloud auth application-default login'
│
│   with provider["registry.terraform.io/hashicorp/google-beta"],
│   on provider.tf line 17, in provider "google-beta":
│   17: provider "google-beta" {
│
│ google: could not find default credentials. See https://cloud.google.com/docs/authentication/external/set-up-adc for more information
╵
╷
│ Error: External Program Execution Failed
│
│   with module.bootstrap_csr_repo.data.external.env_override[0],
│   on .terraform/modules/bootstrap_csr_repo/main.tf line 74, in data "external" "env_override":
│   74:   program = ["${path.module}/scripts/check_env.sh"]
│
│ The data source received an unexpected error while attempting to execute the program.
│
│ The program was executed, however it returned no additional error messaging.
│
│ Program: .terraform/modules/bootstrap_csr_repo/scripts/check_env.sh
│ State: fork/exec .terraform/modules/bootstrap_csr_repo/scripts/check_env.sh: no such file or directory
╵
╷
│ Error: External Program Execution Failed
│
│   with module.build_terraform_image.data.external.env_override[0],
│   on .terraform/modules/build_terraform_image/main.tf line 74, in data "external" "env_override":
│   74:   program = ["${path.module}/scripts/check_env.sh"]
│
│ The data source received an unexpected error while attempting to execute the program.
│
│ The program was executed, however it returned no additional error messaging.
│
│ Program: .terraform/modules/build_terraform_image/scripts/check_env.sh
│ State: fork/exec .terraform/modules/build_terraform_image/scripts/check_env.sh: no such file or directory
╵
root@f7be1a75a354:/bootstrap# ./terraform init

gcloud auth application-default login

root@f7be1a75a354:/bootstrap# gcloud auth application-default login
Go to the following link in your browser, and complete the sign-in prompts:

    https://accounts.google.com/o/oauth2/auth?response_type=code&client_id=764086051850-6qr4p6gpi6hn506pt8ejuq83di341hur.apps.googleusercontent.com&redirect_uri=https%3A%2F%2Fsdk.cloud.google.com%2Fapplicationdefaultauthcode.html&scope=openid+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fuserinfo.email+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fcloud-platform+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fsqlservice.login&state=WHMwdaaKtJq5N6RYvvFPbUqdAlSsOe&prompt=consent&token_usage=remote&access_type=offline&code_challenge=JgXmoyXSfh2LZOW77vO6Z1LSQoyHFY32__ucFAUbOaM&code_challenge_method=S256

Once finished, enter the verification code provided in your browser: 4/0AdLIrYdBqmEIT_ZiwruzWR1Dcqdsr-9iBMmWz0e4SAcXvGhyDtAAPOZoSVgZVAvMGt8Q6g

Credentials saved to file: [/root/.config/gcloud/application_default_credentials.json]

These credentials will be used by any library that requests Application Default Credentials (ADC).

Quota project "tef-oldev-p1gen6" was added to ADC which can be used by Google client libraries for billing and quota. Note that some services may still bill the project owning the resource.

root@f7be1a75a354:/bootstrap# ./terraform plan --input=false -out bootstrap.tfplan
module.build_terraform_image.data.external.env_override[0]: Reading...
module.bootstrap_csr_repo.data.external.env_override[0]: Reading...
data.google_organization.org[0]: Reading...
data.google_organization.org[0]: Read complete after 0s [id=organizations/583675367868]
╷
│ Error: External Program Execution Failed
│
│   with module.bootstrap_csr_repo.data.external.env_override[0],
│   on .terraform/modules/bootstrap_csr_repo/main.tf line 74, in data "external" "env_override":
│   74:   program = ["${path.module}/scripts/check_env.sh"]
│
│ The data source received an unexpected error while attempting to execute the program.
│
│ The program was executed, however it returned no additional error messaging.
│
│ Program: .terraform/modules/bootstrap_csr_repo/scripts/check_env.sh
│ State: fork/exec .terraform/modules/bootstrap_csr_repo/scripts/check_env.sh: no such file or directory
╵
╷
│ Error: External Program Execution Failed
│
│   with module.build_terraform_image.data.external.env_override[0],
│   on .terraform/modules/build_terraform_image/main.tf line 74, in data "external" "env_override":
│   74:   program = ["${path.module}/scripts/check_env.sh"]
│
│ The data source received an unexpected error while attempting to execute the program.
│
│ The program was executed, however it returned no additional error messaging.
│
│ Program: .terraform/modules/build_terraform_image/scripts/check_env.sh
│ State: fork/exec .terraform/modules/build_terraform_image/scripts/check_env.sh: no such file or directory
╵
root@f7be1a75a354:/bootstrap#

ok the container still complaining about win32 even though we are in a linux container

root@f7be1a75a354:/bootstrap# df
Filesystem      1K-blocks      Used  Available Use% Mounted on
overlay        1055762868  55998912  946060484   6% /
tmpfs               65536         0      65536   0% /dev
tmpfs            16347364         0   16347364   0% /sys/fs/cgroup
shm                 65536         0      65536   0% /dev/shm
C:\            3998454780 477534832 3520919948  12% /bootstrap
/dev/sde       1055762868  55998912  946060484   6% /root/.kube
tmpfs            16347364         0   16347364   0% /proc/acpi
tmpfs            16347364         0   16347364   0% /sys/firmware

Issue is symlinks up above current dir needs to be flattened first - or the shared dir moved up one to retest

obriensystems commented 6 months ago

Issue is symlinks up above current dir needs to be flattened first - or the shared dir moved up one to retest

reverify main branch ok with osx/linux

retested on org 2 on a test deploy already applied = ok as expected

module.tf_workspace["proj"].google_cloudbuild_trigger.triggers["plan"]: Refreshing state... [id=projects/prj-b-cicd-wm4z/locations/us-central1/triggers/6d4c8167-7e8a-4abb-8b54-d973a2cfcf1c]

No changes. Your infrastructure matches the configuration.

Terraform has compared your real infrastructure against your configuration and found no differences, so no changes are needed.
michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ terraform plan -input=false -out bootstrap.tfplan

later, I'll try a generic ubuntu container - add gcloud sdk and terraform

obriensystems commented 6 months ago

rerun docker container in root directory above 0-bootstrap

history

root@f7be1a75a354:/bootstrap# history
    1  ls
    2  cd bootstrap
    3  ls
    4  cd tf
    5  ./terraform --version
    6  cp terraform ../
    7  cd ..
    8  ls -la
    9  ./terraform init
   10  gcloud services list
   11  gcloud config set project tef-olapp-p1gen6
   12  gcloud services list
   13  gcloud auth login
   14  gcloud services list
   15  gcloud config set project tef-olapp-p1gen6
   16  gcloud config set project tef-oldev-p1gen6
   17  gcloud services list
   18  ./terraform init
   19  ./terraform plan --input=false -out bootstrap.tfplan
   20  gcloud auth application-default login
   21  ./terraform init
   22  ./terraform plan --input=false -out bootstrap.tfplan
   23  df
   24  history

shutdown / restart container

C:\wse_github\GoogleCloudPlatform\p6gen1\pbmm-on-gcp-onboarding\0-bootstrap>docker ps
CONTAINER ID   IMAGE     COMMAND   CREATED   STATUS    PORTS     NAMES

C:\wse_github\GoogleCloudPlatform\p6gen1\pbmm-on-gcp-onboarding\0-bootstrap>docker rm gcloud
gcloud

C:\wse_github\GoogleCloudPlatform\p6gen1\pbmm-on-gcp-onboarding\0-bootstrap>cd ..

C:\wse_github\GoogleCloudPlatform\p6gen1\pbmm-on-gcp-onboarding>docker run -it --name gcloud -v .:/pbmm google/cloud-sdk:latest /bin/bash
root@676c5ffae045:/# ls
bin  boot  dev  etc  home  lib  lib64  media  mnt  opt  pbmm  proc  root  run  sbin  srv  sys  tmp  usr  var
root@676c5ffae045:/# ls pbmm
0-bootstrap     3-networks-dual-svpc      5-app-infra      ERRATA.md  README.md  fix_tfvars_symlinks.py  policy-library  z_2024_v020_pre_tef_v4
1-org           3-networks-hub-and-spoke  CHANGELOG.md     LICENSE    build      go.work                 scripts
2-environments  4-projects                CONTRIBUTING.MD  Makefile   docs       helpers                 test
root@676c5ffae045:/#

root@676c5ffae045:/pbmm/0-bootstrap# ./terraform --version
Terraform v1.3.10
on linux_amd64
+ provider registry.terraform.io/hashicorp/external v2.3.3
+ provider registry.terraform.io/hashicorp/google v5.28.0
+ provider registry.terraform.io/hashicorp/google-beta v5.28.0
+ provider registry.terraform.io/hashicorp/null v3.2.2
+ provider registry.terraform.io/hashicorp/random v3.6.1
+ provider registry.terraform.io/hashicorp/time v0.11.1

Your version of Terraform is out of date! The latest version
is 1.8.3. You can update by downloading from https://www.terraform.io/downloads.html
root@676c5ffae045:/pbmm/0-bootstrap# gcloud auth login
Go to the following link in your browser, and complete the sign-in prompts:

    https://accounts.google.com/o/oauth2/auth?response_type=code&client_id=32555940559.apps.googleusercontent.com&redirect_uri=https%3A%2F%2Fsdk.cloud.google.com%2Fauthcode.html&scope=openid+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fuserinfo.email+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fcloud-platform+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fappengine.admin+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fsqlservice.login+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fcompute+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Faccounts.reauth&state=KKyyZDG9MI84qgca0K3FpRrdQkmxnN&prompt=consent&token_usage=remote&access_type=offline&code_challenge=-v4p6aeju-Tf0UBL7YQtnZe94ixbOO3UnscK90Ucpfo&code_challenge_method=S256

Once finished, enter the verification code provided in your browser: 4/0AdLIrYetfjUyO83LB9-wn6bXBelazr-dwFxrRKnl7Vnxs1NFM_VBvHZiCbk-wUl5GPSxRQ

You are now logged in as [michael@obrienlabs.dev].
Your current project is [None].  You can change this setting by running:
  $ gcloud config set project PROJECT_ID
root@676c5ffae045:/pbmm/0-bootstrap# gcloud config set project tef-oldev-p1gen6
Updated property [core/project].
root@676c5ffae045:/pbmm/0-bootstrap# ./terraform init
Initializing modules...

Initializing the backend...

Initializing provider plugins...
- Reusing previous version of hashicorp/time from the dependency lock file
- Reusing previous version of hashicorp/random from the dependency lock file
- Reusing previous version of hashicorp/google-beta from the dependency lock file
- Reusing previous version of hashicorp/null from the dependency lock file
- Reusing previous version of hashicorp/external from the dependency lock file
- Reusing previous version of hashicorp/google from the dependency lock file
- Using previously-installed hashicorp/google-beta v5.28.0
- Using previously-installed hashicorp/null v3.2.2
- Using previously-installed hashicorp/external v2.3.3
- Using previously-installed hashicorp/google v5.28.0
- Using previously-installed hashicorp/time v0.11.1
- Using previously-installed hashicorp/random v3.6.1

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
root@676c5ffae045:/pbmm/0-bootstrap#

root@676c5ffae045:/pbmm/0-bootstrap# ./terraform plan --input=false -out bootstrap.tfplan
module.build_terraform_image.data.external.env_override[0]: Reading...
module.bootstrap_csr_repo.data.external.env_override[0]: Reading...
╷
│ Error: Invalid provider configuration
│
│ Provider "registry.terraform.io/hashicorp/google" requires explicit configuration. Add a provider block to the root module and configure the provider's
│ required arguments as described in the provider documentation.
│
╵
╷
│ Error: Attempted to load application default credentials since neither `credentials` nor `access_token` was set in the provider block.  No credentials loaded. To use your gcloud credentials, run 'gcloud auth application-default login'
│
│   with provider["registry.terraform.io/hashicorp/google"],
│   on <empty> line 0:
│   (source code not available)
│
│ google: could not find default credentials. See https://cloud.google.com/docs/authentication/external/set-up-adc for more information
╵
╷
│ Error: Attempted to load application default credentials since neither `credentials` nor `access_token` was set in the provider block.  No credentials loaded. To use your gcloud credentials, run 'gcloud auth application-default login'
│
│   with provider["registry.terraform.io/hashicorp/google-beta"],
│   on provider.tf line 17, in provider "google-beta":
│   17: provider "google-beta" {
│
│ google: could not find default credentials. See https://cloud.google.com/docs/authentication/external/set-up-adc for more information
╵
╷
│ Error: External Program Execution Failed
│
│   with module.bootstrap_csr_repo.data.external.env_override[0],
│   on .terraform/modules/bootstrap_csr_repo/main.tf line 74, in data "external" "env_override":
│   74:   program = ["${path.module}/scripts/check_env.sh"]
│
│ The data source received an unexpected error while attempting to execute the program.
│
│ The program was executed, however it returned no additional error messaging.
│
│ Program: .terraform/modules/bootstrap_csr_repo/scripts/check_env.sh
│ State: fork/exec .terraform/modules/bootstrap_csr_repo/scripts/check_env.sh: no such file or directory
╵
╷
│ Error: External Program Execution Failed
│
│   with module.build_terraform_image.data.external.env_override[0],
│   on .terraform/modules/build_terraform_image/main.tf line 74, in data "external" "env_override":
│   74:   program = ["${path.module}/scripts/check_env.sh"]
│
│ The data source received an unexpected error while attempting to execute the program.
│
│ The program was executed, however it returned no additional error messaging.
│
│ Program: .terraform/modules/build_terraform_image/scripts/check_env.sh
│ State: fork/exec .terraform/modules/build_terraform_image/scripts/check_env.sh: no such file or directory
╵
root@676c5ffae045:/pbmm/0-bootstrap#   gcloud auth application-default login
Go to the following link in your browser, and complete the sign-in prompts:

    https://accounts.google.com/o/oauth2/auth?response_type=code&client_id=764086051850-6qr4p6gpi6hn506pt8ejuq83di341hur.apps.googleusercontent.com&redirect_uri=https%3A%2F%2Fsdk.cloud.google.com%2Fapplicationdefaultauthcode.html&scope=openid+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fuserinfo.email+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fcloud-platform+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fsqlservice.login&state=R1hK4ILDlf34KYiOhxUptOFxqEcDds&prompt=consent&token_usage=remote&access_type=offline&code_challenge=dRyofBHN1MimGd7b2caWgE1ofjDgjDLcOtGPzp-hYrg&code_challenge_method=S256

Once finished, enter the verification code provided in your browser: 4/0AdLIrYfDYqrgDTuXhzLor7T6qv4Noxqj4sgvLfbQiDb0p_jjcb7VHQ16o2gHH4GJ66p35Q

Credentials saved to file: [/root/.config/gcloud/application_default_credentials.json]

These credentials will be used by any library that requests Application Default Credentials (ADC).

Quota project "tef-oldev-p1gen6" was added to ADC which can be used by Google client libraries for billing and quota. Note that some services may still bill the project owning the resource.
root@676c5ffae045:/pbmm/0-bootstrap# ./terraform init
Initializing modules...

Initializing the backend...

Initializing provider plugins...
- Reusing previous version of hashicorp/external from the dependency lock file
- Reusing previous version of hashicorp/google from the dependency lock file
- Reusing previous version of hashicorp/time from the dependency lock file
- Reusing previous version of hashicorp/random from the dependency lock file
- Reusing previous version of hashicorp/google-beta from the dependency lock file
- Reusing previous version of hashicorp/null from the dependency lock file
- Using previously-installed hashicorp/external v2.3.3
- Using previously-installed hashicorp/google v5.28.0
- Using previously-installed hashicorp/time v0.11.1
- Using previously-installed hashicorp/random v3.6.1
- Using previously-installed hashicorp/google-beta v5.28.0
- Using previously-installed hashicorp/null v3.2.2

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
root@676c5ffae045:/pbmm/0-bootstrap#

root@676c5ffae045:/pbmm/0-bootstrap# ./terraform plan --input=false -out bootstrap.tfplan
module.bootstrap_csr_repo.data.external.env_override[0]: Reading...
module.build_terraform_image.data.external.env_override[0]: Reading...
data.google_organization.org[0]: Reading...
data.google_organization.org[0]: Read complete after 1s [id=organizations/583675367868]
╷
│ Error: External Program Execution Failed
│
│   with module.bootstrap_csr_repo.data.external.env_override[0],
│   on .terraform/modules/bootstrap_csr_repo/main.tf line 74, in data "external" "env_override":
│   74:   program = ["${path.module}/scripts/check_env.sh"]
│
│ The data source received an unexpected error while attempting to execute the program.
│
│ The program was executed, however it returned no additional error messaging.
│
│ Program: .terraform/modules/bootstrap_csr_repo/scripts/check_env.sh
│ State: fork/exec .terraform/modules/bootstrap_csr_repo/scripts/check_env.sh: no such file or directory
╵
╷
│ Error: External Program Execution Failed
│
│   with module.build_terraform_image.data.external.env_override[0],
│   on .terraform/modules/build_terraform_image/main.tf line 74, in data "external" "env_override":
│   74:   program = ["${path.module}/scripts/check_env.sh"]
│
│ The data source received an unexpected error while attempting to execute the program.
│
│ The program was executed, however it returned no additional error messaging.
│
│ Program: .terraform/modules/build_terraform_image/scripts/check_env.sh
│ State: fork/exec .terraform/modules/build_terraform_image/scripts/check_env.sh: no such file or directory
╵
root@676c5ffae045:/pbmm/0-bootstrap#

same think at the root dir - missing provider dir

checking whether there is a conflict between terraform binary and .terraform dir

root@676c5ffae045:/pbmm/0-bootstrap# ls -la
total 120940
drwxrwxrwx 1 root root     4096 May  9 16:14 .
drwxrwxrwx 1 root root     4096 May  9 14:53 ..
-rwxrwxrwx 1 root root      991 May  7 15:57 .gitignore
drwxrwxrwx 1 root root     4096 May  9 11:46 .terraform
-rw-r--r-- 1 root root     7221 May  9 14:55 .terraform.lock.hcl
-rwxrwxrwx 1 root root     2133 May  9 14:44 Dockerfile
-rwxrwxrwx 1 root root    43865 May  7 15:57 README-GitHub.md
-rwxrwxrwx 1 root root    44355 May  7 15:57 README-GitLab.md
-rwxrwxrwx 1 root root    44278 May  7 15:57 README-Jenkins.md
-rwxrwxrwx 1 root root    55787 May  7 15:57 README-Terraform-Cloud.md
-rwxrwxrwx 1 root root    19889 May  7 15:57 README.md
drwxr-xr-x 1 root root     4096 May  9 14:05 _in-docker
drwxr-xr-x 1 root root     4096 May  9 14:21 _test
-rwxrwxrwx 1 root root      672 May  7 15:57 backend.tf.cloud.example
-rwxrwxrwx 1 root root      699 May  7 15:57 backend.tf.example
-rwxrwxrwx 1 root root      699 May  7 15:57 backend.tf.local
-rwxrwxrwx 1 root root     9549 May  7 15:57 cb.tf
drwxrwxrwx 1 root root     4096 May  7 15:57 files
-rwxrwxrwx 1 root root     3436 May  7 15:57 github.tf.example
-rwxrwxrwx 1 root root     4825 May  7 15:57 gitlab.tf.example
-rwxrwxrwx 1 root root     1877 May  7 15:57 groups.tf
-rw-r--r-- 1 root root        0 May  9 14:05 in-docker.sh
-rwxrwxrwx 1 root root     2955 May  7 15:57 jenkins.tf.example
-rwxrwxrwx 1 root root     4098 May  7 15:57 main.tf
drwxrwxrwx 1 root root     4096 May  7 15:57 modules
-rwxrwxrwx 1 root root     3149 May  7 15:57 onprem.md
-rwxrwxrwx 1 root root    10045 May  7 15:57 outputs.tf
-rwxrwxrwx 1 root root     3223 May  7 15:57 outputs.tf.local
-rwxrwxrwx 1 root root     1160 May  7 15:57 provider.tf
-rwxrwxrwx 1 root root     7388 May  7 15:57 sa.tf
drwxrwxrwx 1 root root     4096 May  7 15:57 scripts
-rwxr-xr-x 1 root root 61313024 May  9 14:55 terraform

should not be

root@676c5ffae045:/pbmm/0-bootstrap# ls .terraform
modules  providers
root@676c5ffae045:/pbmm/0-bootstrap# ls .terraform/modules/build_terraform_image/
CHANGELOG.md  CONTRIBUTING.md  Makefile   build  examples     main.tf  outputs.tf  test          versions.tf
CODEOWNERS    LICENSE          README.md  cache  kitchen.yml  modules  scripts     variables.tf

obriensystems commented 6 months ago

run the repo inside the container - emulating cloud shell

root@676c5ffae045:/pbmm# cd ..
root@676c5ffae045:/# mkdir _pbmm_inside
root@676c5ffae045:/# cd _pbmm_inside/
root@676c5ffae045:/_pbmm_inside# git status
fatal: not a git repository (or any of the parent directories): .git
root@676c5ffae045:/_pbmm_inside# git clone https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding.git
Cloning into 'pbmm-on-gcp-onboarding'...
remote: Enumerating objects: 6588, done.
remote: Counting objects: 100% (2332/2332), done.
remote: Compressing objects: 100% (822/822), done.
remote: Total 6588 (delta 1614), reused 2015 (delta 1478), pack-reused 4256
Receiving objects: 100% (6588/6588), 31.82 MiB | 7.84 MiB/s, done.
Resolving deltas: 100% (4036/4036), done.
root@676c5ffae045:/_pbmm_inside# cd pbmm-on-gcp-onboarding/
root@676c5ffae045:/_pbmm_inside/pbmm-on-gcp-onboarding# cd 0-bootstrap/
root@676c5ffae045:/_pbmm_inside/pbmm-on-gcp-onboarding/0-bootstrap# cp ../../../
.dockerenv    bin/          dev/          home/         lib64/        mnt/          pbmm/         root/         sbin/         sys/          usr/
_pbmm_inside/ boot/         etc/          lib/          media/        opt/          proc/         run/          srv/          tmp/          var/
root@676c5ffae045:/_pbmm_inside/pbmm-on-gcp-onboarding/0-bootstrap# cp ../../../pbmm/0-bootstrap/terraform .
root@676c5ffae045:/_pbmm_inside/pbmm-on-gcp-onboarding/0-bootstrap# cp ../../../pbmm/0-bootstrap/terraform.tfvars .
root@676c5ffae045:/_pbmm_inside/pbmm-on-gcp-onboarding/0-bootstrap# rm -rf terraform.example.tfvars

obriensystems commented 6 months ago

run terraform from root dir - clone inside docker container

root@676c5ffae045:/_pbmm_inside/pbmm-on-gcp-onboarding/0-bootstrap# cp terraform ../
root@676c5ffae045:/_pbmm_inside/pbmm-on-gcp-onboarding/0-bootstrap# rm -rf terraform
root@676c5ffae045:/_pbmm_inside/pbmm-on-gcp-onboarding/0-bootstrap# ls
Dockerfile                 README.md                 files               main.tf           provider.tf                 terraform_cloud.tf.example
README-GitHub.md           backend.tf.cloud.example  github.tf.example   modules           sa.tf                       variables.tf
README-GitLab.md           backend.tf.example        gitlab.tf.example   onprem.md         scripts                     versions.tf
README-Jenkins.md          backend.tf.local          groups.tf           outputs.tf        terraform-local.tf.example
README-Terraform-Cloud.md  cb.tf                     jenkins.tf.example  outputs.tf.local  terraform.tfvars
root@676c5ffae045:/_pbmm_inside/pbmm-on-gcp-onboarding/0-bootstrap# ../terraform init
Initializing modules...
Downloading registry.terraform.io/terraform-google-modules/gcloud/google 3.4.0 for bootstrap_csr_repo...
- bootstrap_csr_repo in .terraform/modules/bootstrap_csr_repo
- bootstrap_projects_remove_editor in modules/parent-iam-remove-role
Downloading registry.terraform.io/terraform-google-modules/gcloud/google 3.4.0 for build_terraform_image...
- build_terraform_image in .terraform/modules/build_terraform_image
- cicd_project_iam_member in modules/parent-iam-member
Downloading registry.terraform.io/terraform-google-modules/cloud-storage/google 5.0.0 for gcp_projects_state_bucket...
- gcp_projects_state_bucket in .terraform/modules/gcp_projects_state_bucket/modules/simple_bucket
Downloading registry.terraform.io/terraform-google-modules/group/google 0.6.1 for optional_group...
- optional_group in .terraform/modules/optional_group
- org_iam_member in modules/parent-iam-member
- parent_iam_member in modules/parent-iam-member
Downloading registry.terraform.io/terraform-google-modules/group/google 0.6.1 for required_group...
- required_group in .terraform/modules/required_group
Downloading registry.terraform.io/terraform-google-modules/bootstrap/google 7.2.0 for seed_bootstrap...
- seed_bootstrap in .terraform/modules/seed_bootstrap
Downloading registry.terraform.io/terraform-google-modules/org-policy/google 5.3.0 for seed_bootstrap.enable_cross_project_service_account_usage...
- seed_bootstrap.enable_cross_project_service_account_usage in .terraform/modules/seed_bootstrap.enable_cross_project_service_account_usage
Downloading registry.terraform.io/terraform-google-modules/kms/google 2.3.0 for seed_bootstrap.kms...
- seed_bootstrap.kms in .terraform/modules/seed_bootstrap.kms
Downloading registry.terraform.io/terraform-google-modules/project-factory/google 14.5.0 for seed_bootstrap.seed_project...
- seed_bootstrap.seed_project in .terraform/modules/seed_bootstrap.seed_project
- seed_bootstrap.seed_project.budget in .terraform/modules/seed_bootstrap.seed_project/modules/budget
- seed_bootstrap.seed_project.essential_contacts in .terraform/modules/seed_bootstrap.seed_project/modules/essential_contacts
- seed_bootstrap.seed_project.gsuite_group in .terraform/modules/seed_bootstrap.seed_project/modules/gsuite_group
- seed_bootstrap.seed_project.project-factory in .terraform/modules/seed_bootstrap.seed_project/modules/core_project_factory
- seed_bootstrap.seed_project.project-factory.project_services in .terraform/modules/seed_bootstrap.seed_project/modules/project_services
- seed_bootstrap.seed_project.quotas in .terraform/modules/seed_bootstrap.seed_project/modules/quota_manager
- seed_bootstrap.seed_project.shared_vpc_access in .terraform/modules/seed_bootstrap.seed_project/modules/shared_vpc_access
- seed_project_iam_member in modules/parent-iam-member
Downloading registry.terraform.io/terraform-google-modules/bootstrap/google 7.2.0 for tf_cloud_builder...
- tf_cloud_builder in .terraform/modules/tf_cloud_builder/modules/tf_cloudbuild_builder
Downloading registry.terraform.io/terraform-google-modules/cloud-storage/google 5.0.0 for tf_cloud_builder.bucket...
- tf_cloud_builder.bucket in .terraform/modules/tf_cloud_builder.bucket/modules/simple_bucket
- tf_private_pool in modules/cb-private-pool
Downloading registry.terraform.io/terraform-google-modules/network/google 9.1.0 for tf_private_pool.firewall_rules...
- tf_private_pool.firewall_rules in .terraform/modules/tf_private_pool.firewall_rules/modules/firewall-rules
Downloading registry.terraform.io/terraform-google-modules/network/google 9.1.0 for tf_private_pool.peered_network...
- tf_private_pool.peered_network in .terraform/modules/tf_private_pool.peered_network
- tf_private_pool.peered_network.firewall_rules in .terraform/modules/tf_private_pool.peered_network/modules/firewall-rules
- tf_private_pool.peered_network.routes in .terraform/modules/tf_private_pool.peered_network/modules/routes
- tf_private_pool.peered_network.subnets in .terraform/modules/tf_private_pool.peered_network/modules/subnets
- tf_private_pool.peered_network.vpc in .terraform/modules/tf_private_pool.peered_network/modules/vpc
Downloading registry.terraform.io/terraform-google-modules/vpn/google 4.0.0 for tf_private_pool.vpn_ha_cb_to_onprem...
- tf_private_pool.vpn_ha_cb_to_onprem in .terraform/modules/tf_private_pool.vpn_ha_cb_to_onprem/modules/vpn_ha
Downloading registry.terraform.io/terraform-google-modules/bootstrap/google 7.2.0 for tf_source...
- tf_source in .terraform/modules/tf_source/modules/tf_cloudbuild_source
Downloading registry.terraform.io/terraform-google-modules/cloud-storage/google 5.0.0 for tf_source.cloudbuild_bucket...
- tf_source.cloudbuild_bucket in .terraform/modules/tf_source.cloudbuild_bucket/modules/simple_bucket
Downloading registry.terraform.io/terraform-google-modules/project-factory/google 14.5.0 for tf_source.cloudbuild_project...
- tf_source.cloudbuild_project in .terraform/modules/tf_source.cloudbuild_project
- tf_source.cloudbuild_project.budget in .terraform/modules/tf_source.cloudbuild_project/modules/budget
- tf_source.cloudbuild_project.essential_contacts in .terraform/modules/tf_source.cloudbuild_project/modules/essential_contacts
- tf_source.cloudbuild_project.gsuite_group in .terraform/modules/tf_source.cloudbuild_project/modules/gsuite_group
- tf_source.cloudbuild_project.project-factory in .terraform/modules/tf_source.cloudbuild_project/modules/core_project_factory
- tf_source.cloudbuild_project.project-factory.project_services in .terraform/modules/tf_source.cloudbuild_project/modules/project_services
- tf_source.cloudbuild_project.quotas in .terraform/modules/tf_source.cloudbuild_project/modules/quota_manager
- tf_source.cloudbuild_project.shared_vpc_access in .terraform/modules/tf_source.cloudbuild_project/modules/shared_vpc_access
Downloading registry.terraform.io/terraform-google-modules/bootstrap/google 7.2.0 for tf_workspace...
- tf_workspace in .terraform/modules/tf_workspace/modules/tf_cloudbuild_workspace
Downloading registry.terraform.io/terraform-google-modules/cloud-storage/google 5.0.0 for tf_workspace.artifacts_bucket...
- tf_workspace.artifacts_bucket in .terraform/modules/tf_workspace.artifacts_bucket/modules/simple_bucket
Downloading registry.terraform.io/terraform-google-modules/cloud-storage/google 5.0.0 for tf_workspace.log_bucket...
- tf_workspace.log_bucket in .terraform/modules/tf_workspace.log_bucket/modules/simple_bucket
Downloading registry.terraform.io/terraform-google-modules/cloud-storage/google 5.0.0 for tf_workspace.state_bucket...
- tf_workspace.state_bucket in .terraform/modules/tf_workspace.state_bucket/modules/simple_bucket

Initializing the backend...

Initializing provider plugins...
- Finding hashicorp/external versions matching ">= 2.2.2"...
- Finding hashicorp/null versions matching ">= 2.1.0"...
- Finding hashicorp/google versions matching ">= 3.33.0, >= 3.43.0, >= 3.50.0, >= 3.53.0, >= 3.64.0, >= 3.67.0, >= 3.77.0, >= 3.83.0, >= 4.17.0, >= 4.25.0, >= 4.28.0, != 4.31.0, >= 4.46.0, >= 4.64.0, >= 5.7.0, < 6.0.0"...
- Finding hashicorp/random versions matching ">= 2.1.0, >= 2.2.0, >= 3.1.0, ~> 3.4"...
- Finding hashicorp/time versions matching ">= 0.5.0"...
- Finding hashicorp/google-beta versions matching ">= 3.43.0, >= 3.50.0, >= 3.64.0, >= 3.67.0, >= 3.77.0, >= 4.11.0, >= 4.17.0, >= 4.28.0, != 4.31.0, >= 4.64.0, >= 5.7.0, < 6.0.0"...
- Installing hashicorp/google v5.28.0...
- Installed hashicorp/google v5.28.0 (signed by HashiCorp)
- Installing hashicorp/random v3.6.1...
- Installed hashicorp/random v3.6.1 (signed by HashiCorp)
- Installing hashicorp/time v0.11.1...
- Installed hashicorp/time v0.11.1 (signed by HashiCorp)
- Installing hashicorp/google-beta v5.28.0...
- Installed hashicorp/google-beta v5.28.0 (signed by HashiCorp)
- Installing hashicorp/external v2.3.3...
- Installed hashicorp/external v2.3.3 (signed by HashiCorp)
- Installing hashicorp/null v3.2.2...
- Installed hashicorp/null v3.2.2 (signed by HashiCorp)

Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
root@676c5ffae045:/_pbmm_inside/pbmm-on-gcp-onboarding/0-bootstrap#

root@676c5ffae045:/_pbmm_inside/pbmm-on-gcp-onboarding/0-bootstrap# ../terraform plan --input=false -out bootstrap.tfplan

works inside container - with terraform outside the current directory

check on shared drive

  # module.tf_source.module.cloudbuild_project.module.project-factory.module.project_services.google_project_service.project_services["workflows.googleapis.com"] will be created
  + resource "google_project_service" "project_services" {
      + disable_dependent_services = true
      + disable_on_destroy         = false
      + id                         = (known after apply)
      + project                    = (known after apply)
      + service                    = "workflows.googleapis.com"
    }

Plan: 271 to add, 0 to change, 0 to destroy.

Changes to Outputs:
  + bootstrap_step_terraform_service_account_email    = (known after apply)
  + cloud_build_peered_network_id                     = (known after apply)
  + cloud_build_private_worker_pool_id                = (known after apply)
  + cloud_build_worker_peered_ip_range                = "192.168.0.0/24"
  + cloud_build_worker_range_id                       = (known after apply)
  + cloud_builder_artifact_repo                       = (known after apply)
  + cloudbuild_project_id                             = (known after apply)
  + common_config                                     = {
      + billing_account       = "019283-6F1AB5-7AD576"
      + bootstrap_folder_name = (known after apply)
      + default_region        = "northamerica-northeast1"
      + folder_prefix         = "fldr"
      + org_id                = "583675367868"
      + parent_folder         = "878436685331"
      + parent_id             = "folders/878436685331"
      + project_prefix        = "prj"
    }
  + csr_repos                                         = {
      + gcp-bootstrap    = {
          + id      = (known after apply)
          + name    = "gcp-bootstrap"
          + project = (known after apply)
          + url     = (known after apply)
        }
      + gcp-environments = {
          + id      = (known after apply)
          + name    = "gcp-environments"
          + project = (known after apply)
          + url     = (known after apply)
        }
      + gcp-networks     = {
          + id      = (known after apply)
          + name    = "gcp-networks"
          + project = (known after apply)
          + url     = (known after apply)
        }
      + gcp-org          = {
          + id      = (known after apply)
          + name    = "gcp-org"
          + project = (known after apply)
          + url     = (known after apply)
        }
      + gcp-policies     = {
          + id      = (known after apply)
          + name    = "gcp-policies"
          + project = (known after apply)
          + url     = (known after apply)
        }
      + gcp-projects     = {
          + id      = (known after apply)
          + name    = "gcp-projects"
          + project = (known after apply)
          + url     = (known after apply)
        }
      + tf-cloudbuilder  = {
          + id      = (known after apply)
          + name    = "tf-cloudbuilder"
          + project = (known after apply)
          + url     = (known after apply)
        }
    }
  + environment_step_terraform_service_account_email  = (known after apply)
  + gcs_bucket_cloudbuild_artifacts                   = {
      + bootstrap = (known after apply)
      + env       = (known after apply)
      + net       = (known after apply)
      + org       = (known after apply)
      + proj      = (known after apply)
    }
  + gcs_bucket_cloudbuild_logs                        = {
      + bootstrap = (known after apply)
      + env       = (known after apply)
      + net       = (known after apply)
      + org       = (known after apply)
      + proj      = (known after apply)
    }
  + gcs_bucket_tfstate                                = (known after apply)
  + networks_step_terraform_service_account_email     = (known after apply)
  + optional_groups                                   = {
      + "gcp_global_secrets_admin" = "gcp_global_secrets_admin2@obrienlabs.dev"
      + "gcp_kms_admin"            = "gcp_kms_admin2@obrienlabs.dev"
      + "gcp_network_viewer"       = "gcp_network_viewer2@obrienlabs.dev"
      + "gcp_scc_admin"            = "gcp_scc_admin2@obrienlabs.dev"
      + "gcp_security_reviewer"    = "gcp_security_reviewer2@obrienlabs.dev"
    }
  + organization_step_terraform_service_account_email = (known after apply)
  + projects_gcs_bucket_tfstate                       = (known after apply)
  + projects_step_terraform_service_account_email     = (known after apply)
  + required_groups                                   = {
      + "audit_data_users"           = "gcp-audit-data2@obrienlabs.dev"
      + "billing_data_users"         = "gcp-billing-data2@obrienlabs.dev"
      + "group_billing_admins"       = "gcp-billing-admins2@obrienlabs.dev"
      + "group_org_admins"           = "gcp-organization-admins2@obrienlabs.dev"
      + "monitoring_workspace_users" = "gcp-monitoring-workspace2@obrienlabs.dev"
    }
  + seed_project_id                                   = (known after apply)

───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

Saved the plan to: bootstrap.tfplan

To perform exactly these actions, run the following command to apply:
    terraform apply "bootstrap.tfplan"
root@676c5ffae045:/_pbmm_inside/pbmm-on-gcp-onboarding/0-bootstrap#

run on shared dir

root@676c5ffae045:/_pbmm_inside/pbmm-on-gcp-onboarding/0-bootstrap# cd ..
root@676c5ffae045:/_pbmm_inside/pbmm-on-gcp-onboarding# cd ..
root@676c5ffae045:/_pbmm_inside# cd ..
root@676c5ffae045:/# ls
_pbmm_inside  bin  boot  dev  etc  home  lib  lib64  media  mnt  opt  pbmm  proc  root  run  sbin  srv  sys  tmp  usr  var
root@676c5ffae045:/# cd pbmm/
root@676c5ffae045:/pbmm# ls
0-bootstrap     3-networks-dual-svpc      5-app-infra      ERRATA.md  README.md  fix_tfvars_symlinks.py  policy-library  z_2024_v020_pre_tef_v4
1-org           3-networks-hub-and-spoke  CHANGELOG.md     LICENSE    build      go.work                 scripts
2-environments  4-projects                CONTRIBUTING.MD  Makefile   docs       helpers                 test
root@676c5ffae045:/pbmm# cp 0-bootstrap/terraform .
root@676c5ffae045:/pbmm# rm -rf 0-bootstrap/terraform

nope
│ State: fork/exec .terraform/modules/build_terraform_image/scripts/check_env.sh: no such file or directory

keep the repo inside the container - and copy out the plan and state files after

obriensystems commented 6 months ago

fix

C:\wse_github\GoogleCloudPlatform\p6gen1\pbmm-on-gcp-onboarding>docker run -it --name gcloud -v .:/pbmm google/cloud-sdk:latest /bin/bash

   21  mkdir _pbmm_inside
   22  cd _pbmm_inside/
   23  git status
   24  git clone https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding.git
   25  cd pbmm-on-gcp-onboarding/
   26  cd 0-bootstrap/
   27  cp ../../../pbmm/0-bootstrap/terraform .
   28  cp ../../../pbmm/0-bootstrap/terraform.tfvars .
   29  rm -rf terraform.example.tfvars
   30  ls .la
   31  ls -la
   32  cp terraform ../
   33  rm -rf terraform
   34  ls
   35  ../terraform init
   36  ../terraform plan --input=false -out bootstrap.tfplan

obriensystems commented 6 months ago

An example of an attempted gcloud csr clone will kick in the authentication dialog

michael@cloudshell:~/tef-oldev3/_test_repo (clouddeploy-ol)$ gcloud source repos clone gcp-environments --project=prj-b-cicd-khce
ERROR: (gcloud.source.repos.clone) You do not currently have an active account selected.
Please run:

  $ gcloud auth login

to obtain new credentials.

If you have already logged in with a different account, run:

  $ gcloud config set account ACCOUNT

to select an already authenticated account to use.

michael@cloudshell:~/tef-oldev3/_test_repo (clouddeploy-ol)$ gcloud auth login

You are already authenticated with gcloud when running
inside the Cloud Shell and so do not need to run this
command. Do you wish to proceed anyway?

Do you want to continue (Y/n)?  y

Go to the following link in your browser, and complete the sign-in prompts:

    https://accounts.google.com/o/oauth2/auth?response_type=code&client_id=32555940559.apps.googleusercontent.com&redirect_uri=https%3A%2F%2Fsdk.cloud.google.com%2Fauthcode.html&scope=openid+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fuserinfo.email+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fcloud-platform+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fappengine.admin+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fsqlservice.login+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fcompute+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Faccounts.reauth&state=aG2dXYAYft3Th6tRhLrlPta9R2SFnH&prompt=consent&token_usage=remote&access_type=offline&code_challenge=o5hpUDq-NnEEBvYal1yeuN_sDodjpFN03ngTn8wr3Es&code_challenge_method=S256

Once finished, enter the verification code provided in your browser: 4/0AdLIrYdzyTUtvEi5ypXe28QTkjUE_7b1amWrJV4u_TH8QI9cl9DXDVjFnZ-UawcrZuz9KA

You are now logged in as [michael@obrienlabs.dev].
Your current project is [tef-oldev].  You can change this setting by running:
  $ gcloud config set project PROJECT_ID

michael@cloudshell:~/tef-oldev3/_test_repo (tef-oldev)$ gcloud source repos clone gcp-environments --project=prj-b-cicd-khce
Cloning into '/home/michael/tef-oldev3/_test_repo/gcp-environments'...
warning: You appear to have cloned an empty repository.
Project [prj-b-cicd-khce] repository [gcp-environments] was cloned to [/home/michael/tef-oldev3/_test_repo/gcp-environments].
michael@cloudshell:~/tef-oldev3/_test_repo (tef-oldev)$ ls 
gcp-environments  gcp-policies

Client after re-authentication sa is admin and csr admin in iam, and also set in the repo permissions block copy

" ERROR: (gcloud.source.repos.clone) Command '['git', 'clone', 'https://source.developers.google.com/p/prj-b-cicd-orcl/r/gcp-networks', '/home/user-account/lz-tef-dev-2/temp/gcp-networks', '--config', 'credential.https://source.developers.google.com/.helper=', '--config', 'credential.https://source.developers.google.com/.helper=!gcloud auth git-helper --account=user@gcp..ca --ignore-unknown $@']' returned non-zero exit status 128.

move to #431

fmichaelobrien commented 6 months ago

github-actions[bot] commented 3 months ago

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 7 days