terraform-google-modules / terraform-example-foundation

Shows how the CFT modules can be composed to build a secure cloud foundation
https://cloud.google.com/architecture/security-foundations
Apache License 2.0
1.21k stars 708 forks source link

fix: tf plan truncated on PR comments and plan_validate_all error logs in github actions #1129

Closed nbugden closed 3 months ago

nbugden commented 6 months ago

Context: The terraform plan generated in GitHub Actions is often truncated when the comment is posted to a PR. This is because the comment is posting the stdout for all environments not just the plan generated by terraform. This PR logs the terraform plan as a separate text file for each environment and then uses those files to create 1 comment per environment on the PR from GitHub Actions.

This also removes the need for #1113 as stdout from the plan_validate_all stage is no longer written to a file.

Issue(s) Resolved:

1109

1130

google-cla[bot] commented 6 months ago

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

nbugden commented 4 months ago

@rjerrems @eeaton @gtsorbo can one of you folks take a look. PR needs /gcbrun and an approving review.

nbugden commented 4 months ago

@rjerrems @eeaton @gtsorbo can one of you folks take a look. PR needs /gcbrun and an approving review.

Nudge, can I get a review. If there is something else required before review, please let me know.

apeabody commented 4 months ago

/gcbrun

(unsure if it "works" in a nested comment)

nbugden commented 4 months ago

@eeaton @apeabody thanks for triggering the checks! terraform-example-foundation-int-trigger-HubAndSpoke (cloud-foundation-cicd) has failed, but I'm unable to see the logs. Can either of you help me with the error so I can address any changes needed in the PR?

apeabody commented 4 months ago

@eeaton @apeabody thanks for triggering the checks! terraform-example-foundation-int-trigger-HubAndSpoke (cloud-foundation-cicd) has failed, but I'm unable to see the logs. Can either of you help me with the error so I can address any changes needed in the PR?

Might be a flake:

Step #30 - "destroy-networks": === NAME  TestNetworks/production
Step #30 - "destroy-networks":     destroy.go:11: 
Step #30 - "destroy-networks":          Error Trace:    /builder/home/go/pkg/mod/github.com/gruntwork-io/terratest@v0.46.11/modules/terraform/destroy.go:11
Step #30 - "destroy-networks":                                      /builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.12.1/pkg/tft/terraform.go:456
Step #30 - "destroy-networks":                                      /builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.12.1/pkg/tft/terraform.go:543
Step #30 - "destroy-networks":                                      /builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.12.1/pkg/tft/terraform.go:556
Step #30 - "destroy-networks":                                      /builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.12.1/pkg/utils/stages.go:31
Step #30 - "destroy-networks":                                      /builder/home/go/pkg/mod/github.com/!google!cloud!platform/cloud-foundation-toolkit/infra/blueprint-test@v0.12.1/pkg/tft/terraform.go:559
Step #30 - "destroy-networks":          Error:          Received unexpected error:
Step #30 - "destroy-networks":                          'terraform [destroy -auto-approve -input=false -var access_context_manager_policy_id=727108865530 -var remote_state_bucket=bkt-d5y-b-seed-tfstate-1323 -var ingress_policies=[{"from" = {"sources" = {"access_levels" = ["*"]}, "identity_type" = "ANY_IDENTITY"}, "to" = {"resources" = ["*"], "operations" = {"storage.googleapis.com" = {"methods" = ["google.storage.objects.get", "google.storage.objects.list"]}}}}] -var egress_policies=[{"from" = {"identity_type" = "ANY_IDENTITY"}, "to" = {"resources" = ["*"], "operations" = {"storage.googleapis.com" = {"methods" = ["google.storage.objects.get", "google.storage.objects.list"]}}}}] -var perimeter_additional_members=[] -no-color -lock=false]' unsuccessful after 1 retries
Step #30 - "destroy-networks":          Test:           TestNetworks/production
apeabody commented 4 months ago

/gcbrun

nbugden commented 4 months ago

Looks like this time both cloud build checks failed. Possibly another flake? I also had to pull in the latest changes. Can you help me with the error so I can address any changes needed in the PR?

apeabody commented 4 months ago

/gcbrun

nbugden commented 4 months ago

Checks have passed. Can a get a review and merge if this is ready to go?

nbugden commented 4 months ago

Pulled in the latest from master, can I get a review as well?

nbugden commented 4 months ago

/gcbrun

daniel-cit commented 4 months ago

@nbugden I will try a deploy on github for the review

nbugden commented 4 months ago

FYI @daniel-cit I just fixed the named branches in the pipeline triggers. Still had non-production from a bad rebase.

nbugden commented 4 months ago

@daniel-cit any questions about the PR? Did you get a chance to test it?

daniel-cit commented 4 months ago

@nbugden tested image

daniel-cit commented 4 months ago

/gcbrun

daniel-cit commented 4 months ago

/gcbrun

daniel-cit commented 4 months ago

/gcbrun

daniel-cit commented 4 months ago

/gcbrun

daniel-cit commented 4 months ago

/gcbrun

daniel-cit commented 4 months ago

/gcbrun

nbugden commented 4 months ago

@daniel-cit one of the integration test pipelines failed

daniel-cit commented 4 months ago

/gcbrun

daniel-cit commented 4 months ago

/gcbrun

daniel-cit commented 4 months ago

/gcbrun

daniel-cit commented 3 months ago

/gcbrun

daniel-cit commented 3 months ago

/gcbrun

daniel-cit commented 3 months ago

@apeabody the build is green :)