Setting up separate terraform google cloud project

[aroelo]

Partially following https://cloud.google.com/community/tutorials/managing-gcp-projects-with-terraform

Create project: terraform-admin, project id: terraform-admin-279004

Linked billing account

Create service account:

gcloud iam service-accounts create terraform \
>   --display-name "Terraform admin account"

Create & Download keys json Stored locally: /Users/aniekroelofs/Downloads/terraform-admin-279004-2e72b7feddc4.json

Grant the service account permission to view the Admin Project and manage Cloud Storage:

export TF_ADMIN=terraform-admin-279004

gcloud projects add-iam-policy-binding ${TF_ADMIN} \
  --member serviceAccount:terraform@${TF_ADMIN}.iam.gserviceaccount.com \
  --role roles/viewer

gcloud projects add-iam-policy-binding ${TF_ADMIN} \
  --member serviceAccount:terraform@${TF_ADMIN}.iam.gserviceaccount.com \
  --role roles/storage.admin

TODO Any actions that Terraform performs require that the API be enabled to do so. Possible list:

gcloud services enable cloudresourcemanager.googleapis.com
gcloud services enable cloudbilling.googleapis.com
gcloud services enable iam.googleapis.com
gcloud services enable compute.googleapis.com
gcloud services enable serviceusage.googleapis.com
gcloud services enable composer.googleapis.com
gcloud services enable bigquery.googleapis.com
gcloud services enable bigquerystorage.googleapis.com

TODO Grant the service account permission to create projects and assign billing accounts:

export TF_VAR_org_id=670938844927

gcloud organizations add-iam-policy-binding ${TF_VAR_org_id} \
  --member serviceAccount:terraform@${TF_ADMIN}.iam.gserviceaccount.com \
  --role roles/resourcemanager.projectCreator

gcloud organizations add-iam-policy-binding ${TF_VAR_org_id} \
  --member serviceAccount:terraform@${TF_ADMIN}.iam.gserviceaccount.com \
  --role roles/billing.user

ERROR: (gcloud.organizations.add-iam-policy-binding) User [aniek.roelofs@observatory.academy] does not have permission to access organization [670938844927:getIamPolicy] (or it may not exist): The caller does not have permission

TODO Create bucket & enable object versioning:

gsutil mb -p ${TF_ADMIN} gs://${TF_ADMIN}

gsutil versioning set on gs://${TF_ADMIN}

[rhosking]

@aroelo I believe this has been completed? or is there still something remaining to be done on this task?

[aroelo]

@rhosking it depends a bit on how we want to set it up. If we would like a terraform-admin account that is capable of creating a new project and setting up resources in this project you'll have to give the terraform-admin account permissions to create projects and assign billing accounts to new projects.

See

export TF_ADMIN=terraform-admin-279004
export TF_VAR_org_id=670938844927

gcloud organizations add-iam-policy-binding ${TF_VAR_org_id} \
  --member serviceAccount:terraform@${TF_ADMIN}.iam.gserviceaccount.com \
  --role roles/resourcemanager.projectCreator

gcloud organizations add-iam-policy-binding ${TF_VAR_org_id} \
  --member serviceAccount:terraform@${TF_ADMIN}.iam.gserviceaccount.com \
  --role roles/billing.user

[jdddog]

@aroelo figured out how to enable these APIs for a Google Cloud project with Terraform.

These were the APIs that I had to manually enable for https://github.com/The-Academic-Observatory/academic-observatory/pull/206:

storagetransfer.googleapis.com
iam.googleapis.com
servicenetworking.googleapis.com
secretmanager.googleapis.com
sqladmin.googleapis.com

There might be others that were required as well, e.g. cloudbilling.googleapis.com might be required for secretmanager.googleapis.com to be enabled.

@aroelo could you add that functionality in a pull request along with the Terraform Cloud work?

[rhosking]

@aroelo I don't see a linked pulled request, is this work still relevant, or has it been included in the recent changes you are making?

[aroelo]

This is included in the pull request here https://github.com/The-Academic-Observatory/observatory-platform/pull/231 thanks for mentioning it.