hashicorp / terraform-provider-google

Terraform Provider for Google Cloud Platform
https://registry.terraform.io/providers/hashicorp/google/latest/docs
Mozilla Public License 2.0
2.35k stars 1.75k forks source link

Run relevant VCR tests in GA #13170

Open melinath opened 1 year ago

melinath commented 1 year ago

Community Note

Currently it is possible for us to introduce crashing bugs like https://github.com/hashicorp/terraform-provider-google/issues/13097 through innocuous-seeming changes like https://github.com/GoogleCloudPlatform/magic-modules/pull/6824.

This caused GA-only test failures due to some missing changes, which were resolved in https://github.com/GoogleCloudPlatform/magic-modules/pull/6860.

This kind of issue is not caught by our current tests, but it's very bad for our users and causes extra load on maintainers to fix. We should automate testing for it by running GA VCR tests on PRs for relevant tests.

melinath commented 1 year ago

may be blocked by project split

rileykarson commented 1 year ago

Related: https://github.com/hashicorp/terraform-provider-google/issues/9148

melinath commented 10 months ago

https://github.com/hashicorp/terraform-provider-google/issues/16661 would have been caught by this.

melinath commented 8 months ago

https://github.com/hashicorp/terraform-provider-google/issues/17388 would have been caught by this (12 100% test failures)

melinath commented 8 months ago

https://github.com/GoogleCloudPlatform/magic-modules/pull/9965 fixed an issue that could have been caught by this - patch behavior was not properly updated during a promotion.

melinath commented 6 months ago

https://github.com/hashicorp/terraform-provider-google/issues/18091 and https://github.com/hashicorp/terraform-provider-google/issues/18092 would have been caught by this. (7 100% failures)

melinath commented 5 months ago

https://github.com/hashicorp/terraform-provider-google/issues/18239 would have been caught by this.

melinath commented 3 months ago

The following would have been caught by this: