search

canada-ca / accelerators_accelerateurs-gcp

[GCP] Tools and templates to accelerate GC service delivery. Outils et modèles pour accélérer la prestation de services du GC.
MIT License
10 stars 8 forks source link

Fixing permissions issue on module log_export_to_biqquery - running terraform apply #10

Open fmichaelobrien opened 2 years ago

fmichaelobrien commented 2 years ago

ENV

Run bootstrap then terraform apply

michael@cloudshell:~/accelerators_accelerateurs-gcp/deployment-templates/Terraform/guardrails/0-bootstrap (landingzone-stg-ol)$ ./bootstrap.sh -d $DEPT_NAME -o $ORG_ID -b $BILLING_ACCOUNT

Plan first
michael@cloudshell:~/accelerators_accelerateurs-gcp/deployment-templates/Terraform/guardrails/1-guardrails (oldev1-seed-project)$ terraform plan -var-file variables.tfvar

Plan: 37 to add, 0 to change, 0 to destroy.
1547…1552 errors later

michael@cloudshell:~/accelerators_accelerateurs-gcp/deployment-templates/Terraform/guardrails/1-guardrails (oldev1-seed-project)$ terraform apply -var-file variables.tfvar

module.log_export_to_storage.google_logging_organization_sink.sink[0]: Creating...
module.pubsub_destination.google_pubsub_topic.topic: Creation complete after 2s [id=projects/guardrails-5fa4/topics/tp-org-logs-brxp]
module.pubsub_destination.google_pubsub_subscription.pubsub_subscription[0]: Creating...
module.pubsub_destination.google_service_account.pubsub_subscriber[0]: Creating...
module.log_export_to_pubsub.google_logging_organization_sink.sink[0]: Creating...
module.pubsub_destination.google_service_account.pubsub_subscriber[0]: Creation complete after 1s [id=projects/guardrails-5fa4/serviceAccounts/tp-org-logs-brxp-subscriber@guardrails-5fa4.iam.gserviceaccount.com]
module.pubsub_destination.google_pubsub_topic_iam_member.pubsub_viewer_role[0]: Creating...
module.pubsub_destination.google_pubsub_subscription.pubsub_subscription[0]: Creation complete after 2s [id=projects/guardrails-5fa4/subscriptions/tp-org-logs-brxp-subscription]
module.pubsub_destination.google_pubsub_subscription_iam_member.pubsub_subscriber_role[0]: Creating...
module.pubsub_destination.google_pubsub_topic_iam_member.pubsub_viewer_role[0]: Creation complete after 4s [id=projects/guardrails-5fa4/topics/tp-org-logs-brxp/roles/pubsub.viewer/serviceAccount:tp-org-logs-brxp-subscriber@guardrails-5fa4.iam.gserviceaccount.com]
module.pubsub_destination.google_pubsub_subscription_iam_member.pubsub_subscriber_role[0]: Creation complete after 4s [id=projects/guardrails-5fa4/subscriptions/tp-org-logs-brxp-subscription/roles/pubsub.subscriber/serviceAccount:tp-org-logs-brxp-subscriber@guardrails-5fa4.iam.gserviceaccount.com]
╷
│ Error: googleapi: Error 403: The caller does not have permission, forbidden
│
│   with module.log_export_to_biqquery.google_logging_organization_sink.sink[0],
│   on .terraform/modules/log_export_to_biqquery/main.tf line 93, in resource "google_logging_organization_sink" "sink":
│   93: resource "google_logging_organization_sink" "sink" {

│

Check tg modules
module "bigquery_destination" {
 source                     = "terraform-google-modules/log-export/google//modules/bigquery"
 project_id                 = module.administration.project_id
 dataset_name               = "audit_logs"
 log_sink_writer_identity   = module.log_export_to_biqquery.writer_identity
 expiration_days            = var.audit_logs_table_expiration_days
 delete_contents_on_destroy = var.audit_logs_table_delete_contents_on_destroy
 location                   = var.default_region
}
fmichaelobrien commented 2 years ago

https://github.com/terraform-google-modules/terraform-google-service-accounts/blob/master/main.tf https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/logging_project_sink https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/logging_organization_sink

source? https://github.com/terraform-google-modules/terraform-google-log-export/blob/master/modules/pubsub/main.tf

see related https://github.com/terraform-google-modules/terraform-example-foundation/issues/546 added "security reviewer" role to the SA - no effect checking fix on https://github.com/hashicorp/terraform-provider-google/pull/10493 at the end of 4.10 https://github.com/hashicorp/terraform-provider-google/blob/master/CHANGELOG.md

testing from the opposite end - bring in "Folder Admin or "Organization Administrator" temporarily and remove roles from there to return least privilege added bigquery admin

added "Logs Configuration Writer" , Project IAM Admin, Folder Admin forgot to enable the Cloud Resource Manager API

fmichaelobrien commented 2 years ago

commented out log_sinks.tf for now until I triage the modules separately down from 27 to 13 modules - these work in isolation - but rerun a 2nd time with the additional SA roles removed

Plan: 13 to add, 0 to change, 3 to destroy.

Do you want to perform these actions? Terraform will perform the actions described above. Only 'yes' will be accepted to approve.

Enter a value: yes

random_string.suffix: Creating... random_string.suffix: Creation complete after 0s [id=c0da] google_bigquery_dataset.billing_dataset: Destroying... [id=projects/guardrails-5fa4/datasets/billing_data] module.bigquery_destination.google_bigquery_dataset.dataset: Destroying... [id=projects/guardrails-5fa4/datasets/audit_logs] google_organization_iam_audit_config.org_config[0]: Creating... google_organization_iam_member.billing_viewer: Creating... google_organization_iam_member.ssc-billing: Creating... module.org-policy.google_organization_policy.org_policy_list_allow_values[0]: Creating... google_organization_iam_member.asset_inventory_viewer: Creating... module.administration.module.project-factory.google_service_account.default_service_account[0]: Creating... google_project_iam_member.billing_bq_user: Creating... module.administration.module.project-factory.google_project_default_service_accounts.default_service_accounts[0]: Creating... module.bigquery_destination.google_bigquery_dataset.dataset: Destruction complete after 0s google_project_iam_member.audit_log_bq_data_viewer: Creating... google_bigquery_dataset.billing_dataset: Destruction complete after 1s google_project_iam_member.billing_bq_viewer: Creating... module.org-policy.google_organization_policy.org_policy_list_allow_values[0]: Creation complete after 0s [id=583675367868/constraints/gcp.resourceLocations] google_storage_bucket.guardrails-bucket: Creating... module.administration.module.project-factory.google_service_account.default_service_account[0]: Creation complete after 0s [id=projects/guardrails-5fa4/serviceAccounts/project-service-account@guardrails-5fa4.iam.gserviceaccount.com] google_project_iam_member.audit_log_bq_user: Creating... module.administration.module.project-factory.google_project_default_service_accounts.default_service_accounts[0]: Creation complete after 1s [id=projects/guardrails-5fa4] module.bigquery_destination.google_project_service.enable_destination_api: Destroying... [id=guardrails-5fa4/bigquery.googleapis.com] module.bigquery_destination.google_project_service.enable_destination_api: Destruction complete after 0s google_storage_bucket.guardrails-bucket: Creation complete after 1s [id=oldev1-guardrails-assets] google_organization_iam_member.billing_viewer: Creation complete after 4s [id=583675367868/roles/billing.viewer/group:billingdata@obrienlabs.dev] google_project_iam_member.audit_log_bq_user: Creation complete after 7s [id=guardrails-5fa4/roles/bigquery.user/group:auditdata@obrienlabs.dev] google_project_iam_member.audit_log_bq_data_viewer: Creation complete after 7s [id=guardrails-5fa4/roles/bigquery.dataViewer/group:auditdata@obrienlabs.dev] google_project_iam_member.billing_bq_user: Creation complete after 7s [id=guardrails-5fa4/roles/bigquery.user/group:billingdata@obrienlabs.dev] google_project_iam_member.billing_bq_viewer: Creation complete after 8s [id=guardrails-5fa4/roles/bigquery.dataViewer/group:billingdata@obrienlabs.dev] google_organization_iam_audit_config.org_config[0]: Still creating... [10s elapsed] google_organization_iam_member.ssc-billing: Still creating... [10s elapsed] google_organization_iam_member.asset_inventory_viewer: Still creating... [10s elapsed] google_organization_iam_audit_config.org_config[0]: Creation complete after 17s [id=583675367868/audit_config/allServices] google_organization_iam_member.asset_inventory_viewer: Creation complete after 17s [id=583675367868/roles/cloudasset.viewer/group:sscbroker@obrienlabs.dev] google_organization_iam_member.ssc-billing: Creation complete after 17s [id=583675367868/roles/billing.viewer/group:sscbroker@obrienlabs.dev]

Apply complete! Resources: 13 added, 0 changed, 3 destroyed.

fmichaelobrien commented 2 years ago

Added bigquery admin role to SA - still permissions issue

module.log_export_to_biqquery.google_logging_organization_sink.sink[0]: Creating... module.log_export_to_pubsub.google_logging_organization_sink.sink[0]: Creating... module.log_export_to_storage.google_logging_organization_sink.sink[0]: Creating... ╷ │ Error: googleapi: Error 403: The caller does not have permission, forbidden │ │ with module.log_export_to_biqquery.google_logging_organization_sink.sink[0], │ on .terraform/modules/log_export_to_biqquery/main.tf line 93, in resource "google_logging_organization_sink" "sink": │ 93: resource "google_logging_organization_sink" "sink" { │ ╵ ╷ │ Error: googleapi: Error 403: The caller does not have permission, forbidden │ │ with module.log_export_to_pubsub.google_logging_organization_sink.sink[0], │ on .terraform/modules/log_export_to_pubsub/main.tf line 93, in resource "google_logging_organization_sink" "sink": │ 93: resource "google_logging_organization_sink" "sink" { │ ╵ ╷ │ Error: googleapi: Error 403: The caller does not have permission, forbidden │ │ with module.log_export_to_storage.google_logging_organization_sink.sink[0], │ on .terraform/modules/log_export_to_storage/main.tf line 93, in resource "google_logging_organization_sink" "sink": │ 93: resource "google_logging_organization_sink" "sink" { │ ╵

referencing changes in https://github.com/terraform-google-modules/terraform-example-foundation/blob/master/docs/TROUBLESHOOTING.md#Caller-does-not-have-permission-in-the-organization

obriensystems commented 2 years ago

as per https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/logging_project_sink

enable resource manager API (on guardrails project - already enabled on acc and seed projects) https://console.cloud.google.com/apis/api/cloudresourcemanager.googleapis.com/overview?project=guardrails-571e

already set gcloud organizations add-iam-policy-binding ${org_id} --member=serviceAccount:${act} --role=roles/logging.configWriter

ass service account token creator to SA

fmichaelobrien commented 2 years ago

Thanks Chris Carty - retrofitting for TF SA impersonation - using the LZ as a reference https://github.com/GoogleCloudPlatform/pbmm-on-gcp-onboarding/blob/main/environments/bootstrap/bootstrap.sh#L178 via https://cloud.google.com/blog/topics/developers-practitioners/using-google-cloud-service-account-impersonation-your-terraform-code will update the jira/doc as we go

fmichaelobrien commented 2 years ago

22

fmichaelobrien commented 2 years ago

see #24