search

canada-ca / accelerators_accelerateurs-gcp

[GCP] Tools and templates to accelerate GC service delivery. Outils et modèles pour accélérer la prestation de services du GC.
MIT License
10 stars 8 forks source link

Automate billing/org id generation #14

Open fmichaelobrien opened 2 years ago

fmichaelobrien commented 2 years ago 
export DEPT_NAME=oldev1
export PROJECT_ID=accelerator-gcp-ol
export BILLING_ACCOUNT=$(gcloud alpha billing projects describe $PROJECT_ID '--format=value(billingAccountName)' | sed 's/.*\///')
export ORG_ID=$(gcloud projects get-ancestors ${PROJECT_ID} --format='get(id)' | tail -1)

echo $ORG_ID ./bootstrap.sh -d $DEPT_NAME -o $ORG_ID -b $BILLING_ACCOUNT

obriensystems commented 2 years ago

example

michael@cloudshell:~/accelerators_accelerateurs-gcp/deployment-templates/Terraform/guardrails/0-bootstrap (accelerator-dev-nn)$ ./bootstrap.sh -d dev -p accelerator-dev-nn
seed project id: dev-seed-project
boostrap project id: accelerator-dev-nn
org id: 19738nnnnnnn
billing id: 01A635-CF0E78-nnnn

https://registry.terraform.io/providers/hashicorp/google/latest/docs/guides/getting_started

obriensystems commented 2 years ago

will also need typo fix from Claudia on line 96 where role should be roles - to create more than the storageAdmin role - thanks Claudia

https://github.com/canada-ca/accelerators_accelerateurs-gcp/blob/main/deployment-templates/Terraform/guardrails/0-bootstrap/bootstrap.sh#L96

fmichaelobrien commented 2 years ago

Reference [Claudia Navarro Fragoso]'s fix for line 96 in bootstrap.sh

The typo above in https://github.com/canada-ca/accelerators_accelerateurs-gcp/blob/main/deployment-templates/Terraform/guardrails/0-bootstrap/bootstrap.sh#L96 Where role should be roles when updating the roles on the service account caused only storageAdmin to be added. I forgot but I added the remaining roles manually to get past permissions issues - thanks Claudia

Pull request 14 will need the change from Claudia on line 96 https://github.com/canada-ca/accelerators_accelerateurs-gcp/pull/21

original
deleted:serviceAccount:tfadmin-corg2@corg2-seed-project.iam.gserviceaccount.com?uid=105756688582571897002

Storage Admin Manually fixed
tfadmin-oldev1@oldev1-seed-project.iam.gserviceaccount.com

And 1 role at a time - rediscovered Claudia's fix (need to scroll more)

gcloud organizations add-iam-policy-binding ${org_id} --member=serviceAccount:${act} --role=roles/billing.user gcloud organizations add-iam-policy-binding ${org_id} --member=serviceAccount:${act} --role=roles/compute.networkAdmin gcloud organizations add-iam-policy-binding ${org_id} --member=serviceAccount:${act} --role=roles/compute.xpnAdmin gcloud organizations add-iam-policy-binding ${org_id} --member=serviceAccount:${act} --role=roles/iam.organizationRoleAdmin

gcloud organizations add-iam-policy-binding ${org_id} --member=serviceAccount:${act} --role=roles/orgpolicy.policyAdmin

gcloud organizations add-iam-policy-binding ${org_id} --member=serviceAccount:${act} --role=roles/resourcemanager.folderAdmin gcloud organizations add-iam-policy-binding ${org_id} --member=serviceAccount:${act} --role=roles/resourcemanager.organizationAdmin gcloud organizations add-iam-policy-binding ${org_id} --member=serviceAccount:${act} --role=roles/resourcemanager.projectCreator gcloud organizations add-iam-policy-binding ${org_id} --member=serviceAccount:${act} --role=roles/resourcemanager.projectDeleter

gcloud organizations add-iam-policy-binding ${org_id} --member=serviceAccount:${act} --role=roles/resourcemanager.projectIamAdmin

gcloud organizations add-iam-policy-binding ${org_id} --member=serviceAccount:${act} --role=roles/resourcemanager.projectMover gcloud organizations add-iam-policy-binding ${org_id} --member=serviceAccount:${act} --role=roles/orgpolicy.PolicyAdmin gcloud organizations add-iam-policy-binding ${org_id} --member=serviceAccount:${act} --role=roles/logging.configWriter gcloud organizations add-iam-policy-binding ${org_id} --member=serviceAccount:${act} --role=roles/resourcemanager.projectIamAdmin gcloud organizations add-iam-policy-binding ${org_id} --member=serviceAccount:${act} --role=roles/serviceusage.serviceUsageAdmin gcloud organizations add-iam-policy-binding ${org_id} --member=serviceAccount:${act} --role=roles/bigquery.dataEditor gcloud organizations add-iam-policy-binding ${org_id} --member=serviceAccount:${act} --role=roles/storage.admin

There is also two duplicates that would have caused issues one the code was converted to 1:1 api:role calls

Shout out to Claudia Navarro Fragoso Project: accelerator github project Claudia questioned why only storageAdmin was added as a role to the TF service account - it turns out in the past I had manually added roles to the SA in my 2nd run. Claudia rapidly fixed 3 issues - The organizations add-iam-policy-binding turns out to be single --role capable no grouping works as a single call except the last role. Claudia found two changes that would have caused issues after the 1:1 api:role calls - a typo in one of the role/roles flag, and 2 duplicates As a result Issue 21 patch 14 needs to be updated because of her work. Thank you very much Claudia

testing

ERROR: Policy modification failed. For a binding with condition, run "gcloud alpha iam policies lint-condition" to identify issues in condition. ERROR: (gcloud.organizations.add-iam-policy-binding) INVALID_ARGUMENT: Role roles/orgpolicy.PolicyAdmin is not supported for this resource.

-gcloud organizations add-iam-policy-binding ${org_id} --member=serviceAccount:${act} --role=roles/orgpolicy.PolicyAdmin +gcloud organizations add-iam-policy-binding ${org_id} --member=serviceAccount:${act} --role=roles/orgpolicy.policyAdmin

patch gcloud organizations add-iam-policy-binding ${org_id} --member=serviceAccount:${act} --role=roles/billing.user gcloud organizations add-iam-policy-binding ${org_id} --member=serviceAccount:${act} --role=roles/compute.networkAdmin gcloud organizations add-iam-policy-binding ${org_id} --member=serviceAccount:${act} --role=roles/compute.xpnAdmin gcloud organizations add-iam-policy-binding ${org_id} --member=serviceAccount:${act} --role=roles/iam.organizationRoleAdmin

gcloud organizations add-iam-policy-binding ${org_id} --member=serviceAccount:${act} --role=roles/orgpolicy.policyAdmin

gcloud organizations add-iam-policy-binding ${org_id} --member=serviceAccount:${act} --role=roles/resourcemanager.folderAdmin gcloud organizations add-iam-policy-binding ${org_id} --member=serviceAccount:${act} --role=roles/resourcemanager.organizationAdmin gcloud organizations add-iam-policy-binding ${org_id} --member=serviceAccount:${act} --role=roles/resourcemanager.projectCreator gcloud organizations add-iam-policy-binding ${org_id} --member=serviceAccount:${act} --role=roles/resourcemanager.projectDeleter

gcloud organizations add-iam-policy-binding ${org_id} --member=serviceAccount:${act} --role=roles/resourcemanager.projectIamAdmin

gcloud organizations add-iam-policy-binding ${org_id} --member=serviceAccount:${act} --role=roles/resourcemanager.projectMover gcloud organizations add-iam-policy-binding ${org_id} --member=serviceAccount:${act} --role=roles/orgpolicy.policyAdmin gcloud organizations add-iam-policy-binding ${org_id} --member=serviceAccount:${act} --role=roles/logging.configWriter gcloud organizations add-iam-policy-binding ${org_id} --member=serviceAccount:${act} --role=roles/resourcemanager.projectIamAdmin gcloud organizations add-iam-policy-binding ${org_id} --member=serviceAccount:${act} --role=roles/serviceusage.serviceUsageAdmin gcloud organizations add-iam-policy-binding ${org_id} --member=serviceAccount:${act} --role=roles/bigquery.dataEditor gcloud organizations add-iam-policy-binding ${org_id} --member=serviceAccount:${act} --role=roles/storage.admin

Screen Shot 2022-03-07 at 7 22 00 PM
fmichaelobrien commented 2 years ago

Claudia also caught

sed -i "s/guardrails-asset-bkt/${dpt}-guardrails-assets/g" ${HOME}/accelerators_accelerateurs-gcp/deployment-templates/Terraform/guardrails/1-guardrails/variables.tfvar

sed -i "s/guardrails-asset-bkt/${seed_project_id}-guardrails/g" ${HOME}/accelerators_accelerateurs-gcp/deployment-templates/Terraform/guardrails/1-guardrails/variables.tfvar

fmichaelobrien commented 2 years ago

missing role pubsub and service account admin https://github.com/terraform-google-modules/terraform-google-log-export

retrofitting for TF SA impersonation - using the LZ as a reference https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/blob/main/environments/bootstrap/bootstrap.sh#L178 via https://cloud.google.com/blog/topics/developers-practitioners/using-google-cloud-service-account-impersonation-your-terraform-code will update the jira/doc as we go https://github.com/canada-ca/accelerators_accelerateurs-gcp/issues/22

fmichaelobrien commented 2 years ago

see #24