search

hashicorp / terraform-provider-google

Terraform Provider for Google Cloud Platform
https://registry.terraform.io/providers/hashicorp/google/latest/docs
Mozilla Public License 2.0
2.32k stars 1.73k forks source link

Add an environment variable to enable CI-only tests (that are skipped locally) #18849

Open rileykarson opened 2 months ago

rileykarson commented 2 months ago

What kind of contribution is this issue about?

Other (specify in details)

Details

Tests like https://github.com/hashicorp/terraform-provider-google/blob/2152e73761c19c29d30ad5c8257640eb2d576a53/google/services/pubsub/resource_pubsub_topic_generated_test.go#L188 can only pass in CI, and will always fail locally, preventing developers from using resource wildcards (i.e. TestAccComputeInstance_) to run tests, forcing them to run them one-by-one.

Similar to tests dependent on environment variables like GOOGLE_ORG having a value, we should check for a variable like GOOGLE_CI_ENVIRONMENT=true, skipping the test if not enabled. That'll mean that contributors can run resource suites locally successfully, and if they need to run these advanced tests they can break glass by providing the specific env var.

References

No response

wyardley commented 1 month ago

https://github.com/GoogleCloudPlatform/magic-modules/blob/e1694376f735ab3c6c587baff0551b51a28a0904/mmv1/third_party/terraform/services/container/resource_container_cluster_test.go.erb#L10018-L10023

☝️ here's one good extreme example (https://github.com/GoogleCloudPlatform/magic-modules/pull/11475)

rileykarson commented 1 month ago

As discussed in @wyardley's comment- we could consider adding a gate on some particularly high cost (possibly just within an otherwise lower cost product) as well, whether it's one or multiple vars.