Failing test(s): TestAccIapAppEngineServiceIam*, TestAccAppEngineStandardAppVersion_update

[SarahFrench]

Impacted tests

TestAccAppEngineStandardAppVersion_update TestAccIapAppEngineServiceIamBindingGenerated TestAccIapAppEngineServiceIamBindingGenerated_withAndWithoutCondition TestAccIapAppEngineServiceIamBindingGenerated_withCondition TestAccIapAppEngineServiceIamMemberGenerated TestAccIapAppEngineServiceIamMemberGenerated_withAndWithoutCondition TestAccIapAppEngineServiceIamMemberGenerated_withCondition TestAccIapAppEngineServiceIamPolicyGenerated TestAccIapAppEngineServiceIamPolicyGenerated_withCondition TestAccAppEngineFlexibleAppVersion_update

Affected Resource(s)

• google_app_engine_standard_app_version

Failure rates

• 100% since 2024-06-14/13

Message(s)

------- Stdout: -------
=== RUN   TestAccIapAppEngineServiceIamBindingGenerated
=== PAUSE TestAccIapAppEngineServiceIamBindingGenerated
=== CONT  TestAccIapAppEngineServiceIamBindingGenerated
    vcr_utils.go:152: Step 1/4 error: Error running apply: exit status 1
        Error: Error waiting to create StandardAppVersion: Error waiting for Creating StandardAppVersion: Error code 13, message: Failed to create cloud build: com.google.net.rpc3.client.RpcClientException: <eye3 title='/ArgoAdminNoCloudAudit.CreateBuild, FAILED_PRECONDITION'/> APPLICATION_ERROR;google.devtools.cloudbuild.v1/ArgoAdminNoCloudAudit.CreateBuild;invalid bucket "staging.tf-testju2vajhrhc.appspot.com"; service account tf-testju2vajhrhc@appspot.gserviceaccount.com does not have access to the bucket;AppErrorCode=9;StartTimeMs=1722398946450;unknown;ResFormat=uncompressed;ServerTimeSec=0.660572071;LogBytes=256;Non-FailFast;EndUserCredsRequested;EffSecLevel=none;ReqFormat=uncompressed;ReqID=873e667e60d6b7a;GlobalID=0;Server=[2002:a6a:360d:0:b0:37e:a3be:7ad9]:4001.
          with google_app_engine_standard_app_version.version,
          on terraform_plugin_test.tf line 36, in resource "google_app_engine_standard_app_version" "version":
          36: resource "google_app_engine_standard_app_version" "version" {
--- FAIL: TestAccIapAppEngineServiceIamBindingGenerated (137.00s)
FAIL

Nightly build test history

https://hashicorp.teamcity.com/test/3712244440130588461?currentProjectId=TerraformProviders_GoogleCloud_GOOGLE_BETA_NIGHTLYTESTS&expandTestHistoryChartSection=true

b/357622612

[SarahFrench]

This has also been failing on PRs

[SarahFrench]

This looks relevant: https://cloud.google.com/appengine/docs/standard/nodejs/release-notes#July_12_2024

---

Deployments for new projects might be impacted from the following changes to org policies:

Starting in May 2024, Google Cloud enforces secure-by-default organization policies for all organization resources. This policy prevents App Engine from granting the Editor role to the App Engine default services accounts by default.

Starting in June 2024, Cloud Build has changed the default behavior for how Cloud Build uses service accounts in new projects. This change is detailed in Cloud Build Service Account Change. As a result of this change, new projects deploying versions for the first time may be using the default App Engine service account with insufficient permissions for deploying versions.

If you are impacted by this change you can do one of the following:

Grant the Editor role to the App Engine default service account.

Review the Cloud Build guidance on changes to the default service account and opt out of these changes.

[SarahFrench]

This PR attempts to fix the permissions issues but it doesn't avoid the error happening, despite there being a wait to allow the new permissions to propagate: https://github.com/GoogleCloudPlatform/magic-modules/pull/11326

Not sure what the problem is, and I'm not going to proceed with that PR currently. Hoping for input from the service team