container.snippets.create_cluster_test: test_create_clusters failed

[flaky-bot[bot]]

Note: #8484 was also for this test, but it was closed more than 10 days ago. So, I didn't mark it flaky.

---

commit: 07e8b8145670fc1f9f9ed99e348d8fe28ca2ca7e buildURL: Build Status, Sponge status: failed

Test output

Traceback (most recent call last):
  File "/workspace/container/snippets/.nox/py-3-7/lib/python3.7/site-packages/google/api_core/grpc_helpers.py", line 72, in error_remapped_callable
    return callable_(*args, **kwargs)
  File "/workspace/container/snippets/.nox/py-3-7/lib/python3.7/site-packages/grpc/_channel.py", line 946, in __call__
    return _end_unary_response_blocking(state, call, False, None)
  File "/workspace/container/snippets/.nox/py-3-7/lib/python3.7/site-packages/grpc/_channel.py", line 849, in _end_unary_response_blocking
    raise _InactiveRpcError(state)
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
    status = StatusCode.UNAUTHENTICATED
    details = "Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project."
    debug_error_string = "UNKNOWN:Error received from peer ipv4:173.194.202.95:443 {created_time:"2022-11-30T11:38:47.606427821+00:00", grpc_status:16, grpc_message:"Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project."}"
>

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/workspace/container/snippets/create_cluster_test.py", line 54, in setup_and_tear_down
    wait_for_delete()
  File "/workspace/container/snippets/.nox/py-3-7/lib/python3.7/site-packages/backoff/_sync.py", line 48, in retry
    ret = target(*args, **kwargs)
  File "/workspace/container/snippets/create_cluster_test.py", line 52, in wait_for_delete
    return client.get_operation({"name": op_id}).status
  File "/workspace/container/snippets/.nox/py-3-7/lib/python3.7/site-packages/google/cloud/container_v1/services/cluster_manager/client.py", line 2470, in get_operation
    metadata=metadata,
  File "/workspace/container/snippets/.nox/py-3-7/lib/python3.7/site-packages/google/api_core/gapic_v1/method.py", line 154, in __call__
    return wrapped_func(*args, **kwargs)
  File "/workspace/container/snippets/.nox/py-3-7/lib/python3.7/site-packages/google/api_core/retry.py", line 288, in retry_wrapped_func
    on_error=on_error,
  File "/workspace/container/snippets/.nox/py-3-7/lib/python3.7/site-packages/google/api_core/retry.py", line 190, in retry_target
    return target()
  File "/workspace/container/snippets/.nox/py-3-7/lib/python3.7/site-packages/google/api_core/grpc_helpers.py", line 74, in error_remapped_callable
    raise exceptions.from_grpc_error(exc) from exc
google.api_core.exceptions.Unauthenticated: 401 Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.

[flaky-bot[bot]]

Looks like this issue is flaky. :worried:

I'm going to leave this open and stop commenting.

A human should fix and close this.

---

When run at the same commit (07e8b8145670fc1f9f9ed99e348d8fe28ca2ca7e), this test passed in one build (Build Status, Sponge) and failed in another build (Build Status, Sponge).

[flaky-bot[bot]]

Oops! Looks like this issue is still flaky. It failed again. :grimacing:

I reopened the issue, but a human will need to close it again.

---

commit: 98f37f567772e89cff7ef03f6c008303050c91e1 buildURL: Build Status, Sponge status: failed

Test output

Traceback (most recent call last):
  File "/workspace/container/snippets/.nox/py-3-8/lib/python3.8/site-packages/google/api_core/grpc_helpers.py", line 72, in error_remapped_callable
    return callable_(*args, **kwargs)
  File "/workspace/container/snippets/.nox/py-3-8/lib/python3.8/site-packages/grpc/_channel.py", line 946, in __call__
    return _end_unary_response_blocking(state, call, False, None)
  File "/workspace/container/snippets/.nox/py-3-8/lib/python3.8/site-packages/grpc/_channel.py", line 849, in _end_unary_response_blocking
    raise _InactiveRpcError(state)
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
    status = StatusCode.NOT_FOUND
    details = "Not found: projects/python-docs-samples-tests/zones/us-central1-b/clusters/py-container-repo-test-951a8e41c3."
    debug_error_string = "UNKNOWN:Error received from peer ipv4:74.125.197.95:443 {created_time:"2022-12-05T12:33:26.168698895+00:00", grpc_status:5, grpc_message:"Not found: projects/python-docs-samples-tests/zones/us-central1-b/clusters/py-container-repo-test-951a8e41c3."}"
>

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/workspace/container/snippets/create_cluster_test.py", line 43, in setup_and_tear_down
    op = client.delete_cluster({"name": cluster_name})
  File "/workspace/container/snippets/.nox/py-3-8/lib/python3.8/site-packages/google/cloud/container_v1/services/cluster_manager/client.py", line 2208, in delete_cluster
    response = rpc(
  File "/workspace/container/snippets/.nox/py-3-8/lib/python3.8/site-packages/google/api_core/gapic_v1/method.py", line 113, in __call__
    return wrapped_func(*args, **kwargs)
  File "/workspace/container/snippets/.nox/py-3-8/lib/python3.8/site-packages/google/api_core/retry.py", line 349, in retry_wrapped_func
    return retry_target(
  File "/workspace/container/snippets/.nox/py-3-8/lib/python3.8/site-packages/google/api_core/retry.py", line 191, in retry_target
    return target()
  File "/workspace/container/snippets/.nox/py-3-8/lib/python3.8/site-packages/google/api_core/grpc_helpers.py", line 74, in error_remapped_callable
    raise exceptions.from_grpc_error(exc) from exc
google.api_core.exceptions.NotFound: 404 Not found: projects/python-docs-samples-tests/zones/us-central1-b/clusters/py-container-repo-test-951a8e41c3.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/workspace/container/snippets/create_cluster_test.py", line 54, in setup_and_tear_down
    wait_for_delete()
  File "/workspace/container/snippets/.nox/py-3-8/lib/python3.8/site-packages/backoff/_sync.py", line 48, in retry
    ret = target(*args, **kwargs)
  File "/workspace/container/snippets/create_cluster_test.py", line 52, in wait_for_delete
    return client.get_operation({"name": op_id}).status
NameError: free variable 'op_id' referenced before assignment in enclosing scope

[dandhlee]

Using #8670 in favor, same issue.

[flaky-bot[bot]]

Oops! Looks like this issue is still flaky. It failed again. :grimacing:

I reopened the issue, but a human will need to close it again.

---

commit: 0caa6618920c788cfece5d30e840b19c122ba426 buildURL: Build Status, Sponge status: failed

Test output

Traceback (most recent call last):
  File "/workspace/container/snippets/.nox/py-3-8/lib/python3.8/site-packages/google/api_core/grpc_helpers.py", line 72, in error_remapped_callable
    return callable_(*args, **kwargs)
  File "/workspace/container/snippets/.nox/py-3-8/lib/python3.8/site-packages/grpc/_channel.py", line 946, in __call__
    return _end_unary_response_blocking(state, call, False, None)
  File "/workspace/container/snippets/.nox/py-3-8/lib/python3.8/site-packages/grpc/_channel.py", line 849, in _end_unary_response_blocking
    raise _InactiveRpcError(state)
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
    status = StatusCode.NOT_FOUND
    details = "Not found: projects/python-docs-samples-tests/zones/us-central1-b/clusters/py-container-repo-test-5e3d36da6a."
    debug_error_string = "UNKNOWN:Error received from peer ipv4:74.125.135.95:443 {created_time:"2022-12-06T12:33:15.435792465+00:00", grpc_status:5, grpc_message:"Not found: projects/python-docs-samples-tests/zones/us-central1-b/clusters/py-container-repo-test-5e3d36da6a."}"
>

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/workspace/container/snippets/create_cluster_test.py", line 43, in setup_and_tear_down
    op = client.delete_cluster({"name": cluster_name})
  File "/workspace/container/snippets/.nox/py-3-8/lib/python3.8/site-packages/google/cloud/container_v1/services/cluster_manager/client.py", line 2208, in delete_cluster
    response = rpc(
  File "/workspace/container/snippets/.nox/py-3-8/lib/python3.8/site-packages/google/api_core/gapic_v1/method.py", line 113, in __call__
    return wrapped_func(*args, **kwargs)
  File "/workspace/container/snippets/.nox/py-3-8/lib/python3.8/site-packages/google/api_core/retry.py", line 349, in retry_wrapped_func
    return retry_target(
  File "/workspace/container/snippets/.nox/py-3-8/lib/python3.8/site-packages/google/api_core/retry.py", line 191, in retry_target
    return target()
  File "/workspace/container/snippets/.nox/py-3-8/lib/python3.8/site-packages/google/api_core/grpc_helpers.py", line 74, in error_remapped_callable
    raise exceptions.from_grpc_error(exc) from exc
google.api_core.exceptions.NotFound: 404 Not found: projects/python-docs-samples-tests/zones/us-central1-b/clusters/py-container-repo-test-5e3d36da6a.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/workspace/container/snippets/create_cluster_test.py", line 54, in setup_and_tear_down
    wait_for_delete()
  File "/workspace/container/snippets/.nox/py-3-8/lib/python3.8/site-packages/backoff/_sync.py", line 48, in retry
    ret = target(*args, **kwargs)
  File "/workspace/container/snippets/create_cluster_test.py", line 52, in wait_for_delete
    return client.get_operation({"name": op_id}).status
NameError: free variable 'op_id' referenced before assignment in enclosing scope

[dandhlee]

Same issue as #8670

[flaky-bot[bot]]

Oops! Looks like this issue is still flaky. It failed again. :grimacing:

I reopened the issue, but a human will need to close it again.

---

commit: bcb2b2748eda44ebdac4ec47c4b09570948f9815 buildURL: Build Status, Sponge status: failed

Test output

Traceback (most recent call last):
  File "/workspace/container/snippets/.nox/py-3-8/lib/python3.8/site-packages/google/api_core/grpc_helpers.py", line 72, in error_remapped_callable
    return callable_(*args, **kwargs)
  File "/workspace/container/snippets/.nox/py-3-8/lib/python3.8/site-packages/grpc/_channel.py", line 946, in __call__
    return _end_unary_response_blocking(state, call, False, None)
  File "/workspace/container/snippets/.nox/py-3-8/lib/python3.8/site-packages/grpc/_channel.py", line 849, in _end_unary_response_blocking
    raise _InactiveRpcError(state)
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
    status = StatusCode.NOT_FOUND
    details = "Not found: projects/python-docs-samples-tests/zones/us-central1-b/clusters/py-container-repo-test-ce39209c3a."
    debug_error_string = "UNKNOWN:Error received from peer ipv4:74.125.197.95:443 {grpc_message:"Not found: projects/python-docs-samples-tests/zones/us-central1-b/clusters/py-container-repo-test-ce39209c3a.", grpc_status:5, created_time:"2022-12-08T12:30:39.392550459+00:00"}"
>

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/workspace/container/snippets/create_cluster_test.py", line 43, in setup_and_tear_down
    op = client.delete_cluster({"name": cluster_name})
  File "/workspace/container/snippets/.nox/py-3-8/lib/python3.8/site-packages/google/cloud/container_v1/services/cluster_manager/client.py", line 2208, in delete_cluster
    response = rpc(
  File "/workspace/container/snippets/.nox/py-3-8/lib/python3.8/site-packages/google/api_core/gapic_v1/method.py", line 113, in __call__
    return wrapped_func(*args, **kwargs)
  File "/workspace/container/snippets/.nox/py-3-8/lib/python3.8/site-packages/google/api_core/retry.py", line 349, in retry_wrapped_func
    return retry_target(
  File "/workspace/container/snippets/.nox/py-3-8/lib/python3.8/site-packages/google/api_core/retry.py", line 191, in retry_target
    return target()
  File "/workspace/container/snippets/.nox/py-3-8/lib/python3.8/site-packages/google/api_core/grpc_helpers.py", line 74, in error_remapped_callable
    raise exceptions.from_grpc_error(exc) from exc
google.api_core.exceptions.NotFound: 404 Not found: projects/python-docs-samples-tests/zones/us-central1-b/clusters/py-container-repo-test-ce39209c3a.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/workspace/container/snippets/create_cluster_test.py", line 54, in setup_and_tear_down
    wait_for_delete()
  File "/workspace/container/snippets/.nox/py-3-8/lib/python3.8/site-packages/backoff/_sync.py", line 48, in retry
    ret = target(*args, **kwargs)
  File "/workspace/container/snippets/create_cluster_test.py", line 52, in wait_for_delete
    return client.get_operation({"name": op_id}).status
NameError: free variable 'op_id' referenced before assignment in enclosing scope

[dandhlee]

Hasn't been happening lately, closing for now.