search

terraform-google-modules / terraform-example-foundation

Shows how the CFT modules can be composed to build a secure cloud foundation
https://cloud.google.com/architecture/security-foundations
Apache License 2.0
1.22k stars 714 forks source link

0-bootstrap needs cloudbilling api enabled on seed project #1139

Closed obriensystems closed 5 months ago

obriensystems commented 7 months ago

TL;DR

michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ gcloud services enable cloudbilling.googleapis.com
Operation "operations/acat.p2-153288813308-9c2dddaa-7b1d-4ac0-bd9c-3fe344d1e782" finished successfully.

Expected behavior

No response

Observed behavior

michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ gcloud services enable cloudresourcemanager.googleapis.com
Operation "operations/acat.p2-153288813308-adc4acf2-18f5-4617-bd64-7d5df77820f6" finished successfully.
michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ gcloud services enable cloudidentity.googleapis.com
Operation "operations/acat.p2-153288813308-796324ee-c8f6-45f6-9c6b-79c27589f037" finished successfully.
michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ gcloud services list | grep NAME
NAME: cloudidentity.googleapis.com
NAME: cloudresourcemanager.googleapis.com
michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ gcloud services enable cloudapis.googleapis.com
Operation "operations/acat.p2-153288813308-0b7d17c4-8781-4af3-9e61-ccececbb4119" finished successfully.
michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ gcloud services enable servicemanagement.googleapis.com
michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ gcloud services enable serviceusage.googleapis.com
michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ gcloud services enable storage-api.googleapis.com
michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ gcloud services enable storage.googleapis.com
michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ gcloud services list | grep NAME
NAME: analyticshub.googleapis.com
NAME: bigquery.googleapis.com
NAME: bigqueryconnection.googleapis.com
NAME: bigquerydatapolicy.googleapis.com
NAME: bigquerymigration.googleapis.com
NAME: bigqueryreservation.googleapis.com
NAME: bigquerystorage.googleapis.com
NAME: cloudapis.googleapis.com
NAME: cloudidentity.googleapis.com
NAME: cloudresourcemanager.googleapis.com
NAME: cloudtrace.googleapis.com
NAME: dataform.googleapis.com
NAME: dataplex.googleapis.com
NAME: datastore.googleapis.com
NAME: logging.googleapis.com
NAME: monitoring.googleapis.com
NAME: servicemanagement.googleapis.com
NAME: serviceusage.googleapis.com
NAME: sql-component.googleapis.com
NAME: storage-api.googleapis.com
NAME: storage-component.googleapis.com
NAME: storage.googleapis.com

check roles
Billing Account Administrator
Compute Shared VPC Admin
Folder Admin
Folder Creator
Organization Administrator
Organization Policy Administrator
Project Billing Manager
Project Creator
Project Deleter
Project IAM Admin
Service Account Token Creator
Service Usage Consumer

check
https://github.com/terraform-google-modules/terraform-example-foundation/issues/965

michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ terraform apply bootstrap.tfplan

module.seed_bootstrap.random_id.suffix: Creating...
module.seed_bootstrap.module.seed_project.module.project-factory.random_id.random_project_id_suffix: Creating...
module.tf_private_pool.random_string.suffix: Creating...
random_string.suffix: Creating...
module.seed_bootstrap.module.seed_project.module.project-factory.random_id.random_project_id_suffix: Creation complete after 0s [id=Mco]
module.seed_bootstrap.random_id.suffix: Creation complete after 0s [id=zKQ]
random_string.suffix: Creation complete after 0s [id=wm4z]
module.tf_private_pool.random_string.suffix: Creation complete after 0s [id=4ika]
google_folder.bootstrap: Creating...
module.seed_bootstrap.google_organization_iam_binding.billing_creator: Creating...
module.seed_bootstrap.google_folder_iam_member.org_admin_serviceusage_consumer[0]: Creating...
module.seed_bootstrap.google_folder_iam_member.org_admin_service_account_user[0]: Creating...
module.seed_bootstrap.google_organization_iam_member.org_admins_group["roles/billing.user"]: Creating...
module.seed_bootstrap.google_organization_iam_member.org_billing_admin: Creating...
module.seed_bootstrap.google_folder_iam_member.tmp_project_creator[0]: Creating...
module.seed_bootstrap.google_organization_iam_member.org_admins_group["roles/resourcemanager.organizationAdmin"]: Creating...
module.seed_bootstrap.google_organization_iam_binding.billing_creator: Creation complete after 5s [id=630259462753/roles/billing.creator]
module.seed_bootstrap.google_folder_iam_member.org_admin_service_account_user[0]: Creation complete after 9s [id=folders/1078109772786/roles/iam.serviceAccountUser/group:gcp-organization-admins@obrienlabs.app]
google_folder.bootstrap: Still creating... [10s elapsed]
module.seed_bootstrap.google_folder_iam_member.org_admin_serviceusage_consumer[0]: Still creating... [10s elapsed]
module.seed_bootstrap.google_organization_iam_member.org_billing_admin: Still creating... [10s elapsed]
module.seed_bootstrap.google_organization_iam_member.org_admins_group["roles/billing.user"]: Still creating... [10s elapsed]
module.seed_bootstrap.google_folder_iam_member.tmp_project_creator[0]: Still creating... [10s elapsed]
module.seed_bootstrap.google_organization_iam_member.org_admins_group["roles/resourcemanager.organizationAdmin"]: Still creating... [10s elapsed]
google_folder.bootstrap: Creation complete after 12s [id=folders/865611452734]
module.seed_bootstrap.google_folder_iam_member.org_admin_serviceusage_consumer[0]: Creation complete after 14s [id=folders/1078109772786/roles/serviceusage.serviceUsageConsumer/group:gcp-organization-admins@obrienlabs.app]
module.seed_bootstrap.google_folder_iam_member.tmp_project_creator[0]: Creation complete after 14s [id=folders/1078109772786/roles/resourcemanager.projectCreator/group:gcp-organization-admins@obrienlabs.app]
module.seed_bootstrap.module.seed_project.module.project-factory.google_project.main: Creating...
module.seed_bootstrap.google_organization_iam_member.org_billing_admin: Creation complete after 18s [id=630259462753/roles/billing.admin/group:gcp-billing-admins@obrienlabs.app]
module.seed_bootstrap.google_organization_iam_member.org_admins_group["roles/billing.user"]: Creation complete after 19s [id=630259462753/roles/billing.user/group:gcp-organization-admins@obrienlabs.app]
module.seed_bootstrap.google_organization_iam_member.org_admins_group["roles/resourcemanager.organizationAdmin"]: Creation complete after 19s [id=630259462753/roles/resourcemanager.organizationAdmin/group:gcp-organization-admins@obrienlabs.app]
╷
│ Error: failed pre-requisites: failed to check permissions on billing account "billingAccounts/012EDD-5AD5ED-ECFF0B": googleapi: Error 403: Cloud Billing API has not been used in project tef-olapp before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/cloudbilling.googleapis.com/overview?project=tef-olapp then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.
│ Details:
│ [
│   {
│     "@type": "type.googleapis.com/google.rpc.Help",
│     "links": [
│       {
│         "description": "Google developers console API activation",
│         "url": "https://console.developers.google.com/apis/api/cloudbilling.googleapis.com/overview?project=tef-olapp"
│       }
│     ]
│   },
│   {
│     "@type": "type.googleapis.com/google.rpc.ErrorInfo",
│     "domain": "googleapis.com",
│     "metadata": {
│       "consumer": "projects/tef-olapp",
│       "service": "cloudbilling.googleapis.com"
│     },
│     "reason": "SERVICE_DISABLED"
│   }
│ ]
│ , accessNotConfigured
│ 
│   with module.seed_bootstrap.module.seed_project.module.project-factory.google_project.main,
│   on .terraform/modules/seed_bootstrap.seed_project/modules/core_project_factory/main.tf line 73, in resource "google_project" "main":
│   73: resource "google_project" "main" {
│ 
╵
michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ 
billing is enabled but not the api

enabling billing api
michael@cloudshell:~/tef-olapp/github/terraform-example-foundation/0-bootstrap (tef-olapp)$ gcloud services enable cloudbilling.googleapis.com
Operation "operations/acat.p2-153288813308-9c2dddaa-7b1d-4ac0-bd9c-3fe344d1e782" finished successfully.

Terraform Configuration

cloud shell

Terraform Version

1.7.4

Additional information

No response

The list is in https://github.com/terraform-google-modules/terraform-example-foundation/blob/master/0-bootstrap/main.tf#L78

activate_apis = [
    "serviceusage.googleapis.com",
    "servicenetworking.googleapis.com",
    "cloudkms.googleapis.com",
    "compute.googleapis.com",
    "logging.googleapis.com",
    "bigquery.googleapis.com",
    "cloudresourcemanager.googleapis.com",
    "cloudbilling.googleapis.com",
    "cloudbuild.googleapis.com",
    "iam.googleapis.com",
    "admin.googleapis.com",
    "appengine.googleapis.com",
    "storage-api.googleapis.com",
    "monitoring.googleapis.com",
    "pubsub.googleapis.com",
    "securitycenter.googleapis.com",
    "accesscontextmanager.googleapis.com",
    "billingbudgets.googleapis.com",
    "essentialcontacts.googleapis.com",
    "assuredworkloads.googleapis.com",
    "cloudasset.googleapis.com"
  ]
fmichaelobrien commented 7 months ago

0-bootstrap is up with the changes from #1133 to #1143

Screenshot 2024-03-07 at 10 12 58
fmichaelobrien commented 5 months ago

stale bot timer restart - https://github.com/terraform-google-modules/terraform-example-foundation/blob/master/.github/workflows/stale.yml#L21

fmichaelobrien commented 5 months ago

fixed by

PRs merged to the terraform-example-foundation upstream repo