search

terraform-google-modules / terraform-google-project-factory

Creates an opinionated Google Cloud project by using Shared VPC, IAM, and Google Cloud APIs
https://registry.terraform.io/modules/terraform-google-modules/project-factory/google
Apache License 2.0
826 stars 535 forks source link

HTTP 403 when running preconditions.py #404

Closed mgrzechocinski closed 3 years ago

mgrzechocinski commented 4 years ago

Hi.

I followed up the documentation to set all the required permissions and enable all the APIs on my service project which is used to create new project by Terraform, using project-factory module. This project hosts a service account which is actually used to create new projects and resources within them.

When I run terraform apply, I see that this module runs some custom code and the beginning, using Terraform's null_resource. One of them is the preconditions.py script. This one actually fails in the logs but the Terraform process continues normally. I'm just wondering why this could happen and how to fix it? Error: 

HttpError 403 when requesting https://serviceusage.googleapis.com/v1/projects/mg-terraform-bootstrap/services/admin.googleapis.com?alt=json returned "The caller does not have permission"

Detailed log:

module.project-factory.module.project-factory.null_resource.preconditions (local-exec): Executing: ["/bin/sh" "-c" "python3 /###/###/###/.terraform/modules/project-factory/terraform-google-project-factory-8.0.0/modules/core_project_factory/scripts/preconditions/preconditions.py --billing_account=\"##########\" --credentials_path=\"/###/######-16567b0b2cf3.json\" --folder_id=\"#######\" --impersonate_service_account=\"\" --org_id=\"#######\" --shared_vpc=\"\" "]
module.project-factory.module.project-factory.null_resource.preconditions (local-exec): WARNING:googleapiclient.http:Invalid JSON content from response: b'{\n  "error": {\n    "code": 403,\n    "message": "The caller does not have permission",\n    "status": "PERMISSION_DENIED"\n  }\n}\n'
module.project-factory.module.project-factory.null_resource.preconditions (local-exec): Traceback (most recent call last):
module.project-factory.module.project-factory.null_resource.preconditions (local-exec):   File "/###/###/###/.terraform/modules/project-factory/terraform-google-project-factory-8.0.0/modules/core_project_factory/scripts/preconditions/preconditions.py", line 493, in <module>
module.project-factory.module.project-factory.null_resource.preconditions (local-exec):     retcode = main(sys.argv)
module.project-factory.module.project-factory.null_resource.preconditions (local-exec):   File "/###/###/###/.terraform/modules/project-factory/terraform-google-project-factory-8.0.0/modules/core_project_factory/scripts/preconditions/preconditions.py", line 475, in main
module.project-factory.module.project-factory.null_resource.preconditions (local-exec):     results.append(validator.validate(credentials))
module.project-factory.module.project-factory.null_resource.preconditions (local-exec):   File "/###/###/###/.terraform/modules/project-factory/terraform-google-project-factory-8.0.0/modules/core_project_factory/scripts/preconditions/preconditions.py", line 263, in validate
module.project-factory.module.project-factory.null_resource.preconditions (local-exec):     response = request.execute()
module.project-factory.module.project-factory.null_resource.preconditions (local-exec):   File "/usr/local/lib/python3.7/site-packages/googleapiclient/_helpers.py", line 134, in positional_wrapper
module.project-factory.module.project-factory.null_resource.preconditions (local-exec):     return wrapped(*args, **kwargs)
module.project-factory.module.project-factory.null_resource.preconditions (local-exec):   File "/usr/local/lib/python3.7/site-packages/googleapiclient/http.py", line 898, in execute
module.project-factory.module.project-factory.null_resource.preconditions (local-exec):     raise HttpError(resp, content, uri=self.uri)
module.project-factory.module.project-factory.null_resource.preconditions (local-exec): googleapiclient.errors.HttpError: <HttpError 403 when requesting https://serviceusage.googleapis.com/v1/projects/mg-terraform-bootstrap/services/admin.googleapis.com?alt=json returned "The caller does not have permission">
mgrzechocinski commented 4 years ago

Adding Service Usage Viewer to my service account on the service project (bootstrap) fixes the issue.

module.project-factory.module.project-factory.null_resource.preconditions (local-exec): Executing: ["/bin/sh" "-c" "python3 /###/###//.terraform/modules/project-factory/terraform-google-project-factory-8.0.0/modules/core_project_factory/scripts/preconditions/preconditions.py --billing_account=\"#######\" --credentials_path=\"/#####/#####/####-.json\" --folder_id=\"#####\" --impersonate_service_account=\"\" --org_id=\"#######\" --shared_vpc=\"\" "]
module.project-factory.module.project-factory.null_resource.preconditions: Still creating... [10s elapsed]
module.project-factory.module.project-factory.null_resource.preconditions (local-exec): [
module.project-factory.module.project-factory.null_resource.preconditions (local-exec):     {
module.project-factory.module.project-factory.null_resource.preconditions (local-exec):         "type": "Required APIs on service account project",
module.project-factory.module.project-factory.null_resource.preconditions (local-exec):         "name": "projects/mg-terraform-bootstrap",
module.project-factory.module.project-factory.null_resource.preconditions (local-exec):         "satisfied": [
module.project-factory.module.project-factory.null_resource.preconditions (local-exec):             "iam.googleapis.com",
module.project-factory.module.project-factory.null_resource.preconditions (local-exec):             "cloudresourcemanager.googleapis.com",
module.project-factory.module.project-factory.null_resource.preconditions (local-exec):             "cloudbilling.googleapis.com",
module.project-factory.module.project-factory.null_resource.preconditions (local-exec):             "admin.googleapis.com"
module.project-factory.module.project-factory.null_resource.preconditions (local-exec):         ],
module.project-factory.module.project-factory.null_resource.preconditions (local-exec):         "unsatisfied": [
module.project-factory.module.project-factory.null_resource.preconditions (local-exec):             "appengine.googleapis.com"
module.project-factory.module.project-factory.null_resource.preconditions (local-exec):         ]
module.project-factory.module.project-factory.null_resource.preconditions (local-exec):     },
module.project-factory.module.project-factory.null_resource.preconditions (local-exec):     {
module.project-factory.module.project-factory.null_resource.preconditions (local-exec):         "type": "Service account permissions on billing account",
module.project-factory.module.project-factory.null_resource.preconditions (local-exec):         "name": "billingAccounts/018F26-4A09F0-7F4B1D",
module.project-factory.module.project-factory.null_resource.preconditions (local-exec):         "satisfied": [
module.project-factory.module.project-factory.null_resource.preconditions (local-exec):             "billing.resourceAssociations.create"
module.project-factory.module.project-factory.null_resource.preconditions (local-exec):         ],
module.project-factory.module.project-factory.null_resource.preconditions (local-exec):         "unsatisfied": []
module.project-factory.module.project-factory.null_resource.preconditions (local-exec):     },
module.project-factory.module.project-factory.null_resource.preconditions (local-exec):     {
module.project-factory.module.project-factory.null_resource.preconditions (local-exec):         "type": "Service account permissions on parent folder",
module.project-factory.module.project-factory.null_resource.preconditions (local-exec):         "name": "folders/876609625413",
module.project-factory.module.project-factory.null_resource.preconditions (local-exec):         "satisfied": [
module.project-factory.module.project-factory.null_resource.preconditions (local-exec):             "resourcemanager.projects.create"
module.project-factory.module.project-factory.null_resource.preconditions (local-exec):         ],
module.project-factory.module.project-factory.null_resource.preconditions (local-exec):         "unsatisfied": []
module.project-factory.module.project-factory.null_resource.preconditions (local-exec):     },
module.project-factory.module.project-factory.null_resource.preconditions (local-exec):     {
module.project-factory.module.project-factory.null_resource.preconditions (local-exec):         "type": "Service account permissions on organization",
module.project-factory.module.project-factory.null_resource.preconditions (local-exec):         "name": "organizations/1003110894063",
module.project-factory.module.project-factory.null_resource.preconditions (local-exec):         "satisfied": [],
module.project-factory.module.project-factory.null_resource.preconditions (local-exec):         "unsatisfied": []
module.project-factory.module.project-factory.null_resource.preconditions (local-exec):     }
module.project-factory.module.project-factory.null_resource.preconditions (local-exec): ]

I guess it should be added to the Permissions section of the README file?

morgante commented 4 years ago

I'm not sure we should actually change this, since the precondition script is mainly meant as a helper/troubleshooter. It's not necessarily required to grant the permissions to check.

bharathkkb commented 3 years ago

closing this as preconditions script has been removed from module via #407