search

GoogleCloudPlatform / terraform-validator

Terraform Validator is not an officially supported Google product; it is a library for conversion of Terraform plan data to CAI Assets. If you have been using terraform-validator directly in the past, we recommend migrating to `gcloud beta terraform vet`.
https://cloud.google.com/docs/terraform/policy-validation
Apache License 2.0
438 stars 93 forks source link

Fix ancestry_path for top-level resources #4

Open nstogner opened 5 years ago

nstogner commented 5 years ago

Currently asset.ancestry_path assumes the resource lives within a project. This does not work for folders, etc.

nstogner commented 5 years ago

Examples ancestry_path for top-level resources:

organization: organization/{org_id}
folder: organzation/{org_id}/folder/{folder}
project: organzation/{org_id}/folder/{folder}/project/{project}
nstogner commented 5 years ago

Strategy: Move the calculation ancestry_path upstream into the conversions library. See https://github.com/GoogleCloudPlatform/magic-modules/pull/1620.

nstogner commented 5 years ago

PS: Once 1620 (above) gets merged, we should be able to get rid of the internally defined Asset struct and use the one from the conversions library.

melinath commented 3 years ago

Closing - seems to be a duplicate of #5.

morgante commented 3 years ago

@melinath Last I checked, I think we still need a fix for this. In particular, if you are validating a google_folder resource it should return an ancestry_path of organzation/{org_id}/folder/{parent_folder}.

melinath commented 3 years ago

Whereas now I assume it includes the project as well? I was thinking of this as being a duplicate of #5 (and by extension of #206) but we could probably resolve it separately in the shorter term.

morgante commented 3 years ago

Right, I think now we use a nonsense project in the ancestry path (folders can't have projects as their parent).

This might be a dupe of #5, but since you closed both I think there was more detail here.

suibinz commented 3 years ago

Are the following errors related to this issue? I have an org structure: org/top-folder/child-folder/projects (1-org cloudbuild with https://github.com/terraform-google-modules/terraform-example-foundation/tree/master/1-org)

... Step #1 - "tf plan validate all": ERROR: logging before flag.Parse: I0528 05:00:40.260261 535 convert.go:183] unknown resource: random_id Step #1 - "tf plan validate all": ERROR: logging before flag.Parse: I0528 05:00:40.260272 535 convert.go:183] unknown resource: random_string Step #1 - "tf plan validate all": Error: converting tfplan to CAI assets: adding resource changes to converter: adding resource create or update augmenting asset: getting resource ancestry: project ######-c-billing-logs-0ccf googleapi: Error 403: The caller does not have permission Step #1 - "tf plan validate all": Usage: Step #1 - "tf plan validate all": terraform-validator validate [flags] Step #1 - "tf plan validate all": Step #1 - "tf plan validate all": Flags: Step #1 - "tf plan validate all": --ancestry string Override the ancestry location of the project when validating resources Step #1 - "tf plan validate all": -h, --help help for validate Step #1 - "tf plan validate all": --offline Do not make network requests Step #1 - "tf plan validate all": --output-json Print violations as JSON Step #1 - "tf plan validate all": --policy-path string Path to directory containing validation policies Step #1 - "tf plan validate all": --project string Provider project override (override the default project configuration assigned to the google terraform provider when validating resources) Step #1 - "tf plan validate all": Step #1 - "tf plan validate all": Global Flags: Step #1 - "tf plan validate all": --verbose Log output to stderr Step #1 - "tf plan validate all":

melinath commented 3 years ago

@suibinz that looks like a separate issue. The key line is:

Error: converting tfplan to CAI assets: adding resource changes to converter: adding resource create or update augmenting asset: getting resource ancestry: project ######-c-billing-logs-0ccf googleapi: Error 403: The caller does not have permission

It sounds like you may not have permissions to call the resource ancestry API. That could mean the API isn't enabled or something about your authentication method isn't working. You could fix the auth issue, or you could work around this by using adding the offline and ancestry flags:

terraform-validator validate ./example/terraform.tfplan --offline --project my-project --ancestry organization/my-org/folder/my-folder --policy-path ./path/to/my/policy/library

Please open a new ticket if you have any problems resolving the issue.

suibinz commented 3 years ago

I would guess the same on the permission issue. But in this case, the sa iam has enough permissions("roles/resourcemanager.projectCreator", "roles/resourcemanager.folderAdmin", and "roles/resourcemanager.organizationViewer". I was running the terraform-example-foundation/1-org.

By cleaning up the resource and re-apply, it can pass the step without the Error. If I have further findings, I will report back.