GoogleCloudPlatform / cloud-foundation-fabric

End-to-end modular samples and landing zones toolkit for Terraform on GCP.
Apache License 2.0
1.53k stars 881 forks source link

FAST: Bootstrap deployment on v33.0.0 produces error when impersonating sa #2519

Closed tszebeni closed 2 months ago

tszebeni commented 2 months ago

Describe the bug When a fresh install of FAST v33.0.0 is performed, the following error appears. Terraform.tfvars file looks like in the article, steps are followed. First apply of bootstrap in the name of the user was successful, stage-links produce the symlink command, which is executed. When its time to execute terraform init -migrate-state, the following error happens.

Error: Error inspecting states in the "local" backend:
│     querying Cloud Storage failed: googleapi: Error 403: d1-prod-bootstrap-0@d1-prod-iac-core-0.iam.gserviceaccount.com does not have serviceusage.services.use access to the Google Cloud project. Permission 'serviceusage.services.use' denied on resource (or it may not exist)., forbidden

Please investigate and recommend, thanks

Environment

Terraform v1.9.5
on linux_amd64
+ provider registry.terraform.io/hashicorp/google v5.42.0
+ provider registry.terraform.io/hashicorp/google-beta v5.42.0
+ provider registry.terraform.io/hashicorp/local v2.5.1
6e938132

To Reproduce Deploy Fabric FAST into a vanilla new organization

Expected behavior Deployment should be possible, all the necessary permission should be granted to SA by automation

Result see above

Additional context Add any other context about the problem here

ludoo commented 2 months ago

Is the user part of the gcp-organization-admins group?

tszebeni commented 2 months ago

Hi Ludovico, he is part of the group but he is also owner, and there is no problem there. The problem is with the created sa, error says it: "does not have serviceusage.services.use access to the Google Cloud project". I also noticed that some steps are also missing from deployment as the seed project does not have the required APIs being enabled: IAM, Resource Manager and Billing APIs, FAST deployment depends on these but not mentioned in the readme. By default only these are enabled: https://cloud.google.com/service-usage/docs/enabled-service#default

tszebeni commented 2 months ago

There is also an alternating resource, when executing terraform apply, one time it creates, the next time it deletes the following resource, maybe there is a race condition there with authorative/nonauthorative iam setup:

# module.organization.google_organization_iam_member.bindings["roles/accesscontextmanager.policyReader-serviceAccount:d1-prod-vpcsc-0r@d1-prod-iac-core-0.iam.gserviceaccount.com"] will be created
  + resource "google_organization_iam_member" "bindings" {
      + etag   = (known after apply)
      + id     = (known after apply)
      + member = "serviceAccount:d1-prod-vpcsc-0r@d1-prod-iac-core-0.iam.gserviceaccount.com"
      + org_id = "xyz"
      + role   = "roles/accesscontextmanager.policyReader"
    }

next time:

 # module.organization.google_organization_iam_binding.authoritative["roles/accesscontextmanager.policyReader"] will be updated in-place
  ~ resource "google_organization_iam_binding" "authoritative" {
        id      = "xyz/roles/accesscontextmanager.policyReader"
      ~ members = [
          - "serviceAccount:d1-prod-vpcsc-0r@d1-prod-iac-core-0.iam.gserviceaccount.com",
            # (1 unchanged element hidden)
        ]
        # (3 unchanged attributes hidden)
    }
tszebeni commented 2 months ago

I could fix my original message by switching the current project from seed to d1-prod-iac-core-0. It may need to be mentioned in readme, otherwise this can be closed.

Thanks

ludoo commented 2 months ago

Yep, I was going to suggest that, you beast me to it :)