IT Failure(google-cloud-compute): ITPaginationTest.testPaginationIterating zone not found

[emmileaf]

Failures observed in GraalVM nightly build and presubmit. This error is happening often (in a number of presubmits and re-triggers), but not consistently:

Problematic test - com.google.cloud.compute.v1.integration.ITPaginationTest.testPaginationIterating

Failures (1):
  JUnit Vintage:ITPaginationTest:testPaginationIterating
    MethodSource [className = 'com.google.cloud.compute.v1.integration.ITPaginationTest', methodName = 'testPaginationIterating', methodParameterTypes = '']
    => java.lang.AssertionError: Zone us-central1-a was not found for cloud-java-ci-test in zones list (size: 65).
       org.junit.Assert.fail(Assert.java:89)
       org.junit.Assert.assertTrue(Assert.java:42)
       com.google.cloud.compute.v1.integration.ITPaginationTest.testPaginationIterating(ITPaginationTest.java:124)
       java.base@17.0.5/java.lang.reflect.Method.invoke(Method.java:568)
       org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
       org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
       org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
       org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
       org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
       org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
       [...]

This could be a similar issue with interfering compute resources from integration tests as observed/addressed in https://github.com/googleapis/google-cloud-java/pull/8260.

[emmileaf]

Update - seems like this is a previously observed flake coming back: https://github.com/googleapis/java-compute/issues/683, https://github.com/googleapis/java-compute/pull/682#issuecomment-1108555217 look relevant.

[emmileaf]

An initial investigation from locally reproducing this error w/ Terraform:

gcloud compute zones list returns 119 zones, which also consistent with the response.iterateAll() size from testList().

In testPaginationIterating(), iterating over zones sometimes gets stuck at around this point:

...
Zone: europe-west9-b
Zone: europe-west9-a
Zone: me-central1-b
Zone: me-central1-c

response.IterateAll() size: 46

And sometimes able to get a full list including us-central1-a (DEFAULT_ZONE):

...
Zone: europe-west9-b
Zone: europe-west9-a
Zone: me-west1-a
Zone: me-west1-c
Zone: me-west1-b
...
Zone: us-central1-a
...

[emmileaf]

The nightly builds are back to green, and I am no longer able to reproduce this error 🤔. Leaving this issue open to track and revisit if it resurfaces again.

[lqiu96]

Closing this issue now. It's been ~3 months since this last flaked. We can re-open if this test flakes again.

[emmileaf]

Observed this flake again in nightly build - reopening to track.