tests.system.TestStreamingPull: test_streaming_pull_callback_error_propagation[grpc-grpc] failed

[flaky-bot[bot]]

Note: #1043 was also for this test, but it was closed more than 10 days ago. So, I didn't mark it flaky.

---

commit: 7cf2d11ce20bf9080c4434b920bdf419faac5d88 buildURL: Build Status, Sponge status: failed

[flaky-bot[bot]]

Looks like this issue is flaky. :worried:

I'm going to leave this open and stop commenting.

A human should fix and close this.

---

When run at the same commit (7cf2d11ce20bf9080c4434b920bdf419faac5d88), this test passed in one build (Build Status, Sponge) and failed in another build (Build Status, Sponge).

[mukund-ananthu]

This occurred due to a 504 deadline exceed:

args = (name: "projects/precise-truck-742/topics/t-1723526603392-streaming-pull-callback-error-propagation"
,)
kwargs = {'metadata': [('x-goog-request-params', 'name=projects/precise-truck-742/topics/t-1723526603392-streaming-pull-callback-error-propagation'), ('x-goog-api-client', 'gl-python/3.12.0 grpc/1.66.0rc3 gax/2.19.1 gccl/2.23.0')], 'timeout': 60.0}

    @functools.wraps(callable_)
    def error_remapped_callable(*args, **kwargs):
        try:
            return callable_(*args, **kwargs)
        except grpc.RpcError as exc:
>           raise exceptions.from_grpc_error(exc) from exc
E           google.api_core.exceptions.DeadlineExceeded: 504 Deadline Exceeded

[.nox/prerelease_deps-protobuf_implementation-upb/lib/python3.12/site-packages/google/api_core/grpc_helpers.py:78](https://cs.corp.google.com/piper///depot/google3/.nox/prerelease_deps-protobuf_implementation-upb/lib/python3.12/site-packages/google/api_core/grpc_helpers.py?l=78): DeadlineExceeded
- generated xml file: /tmpfs/src/github/python-pubsub/system_3.12_sponge_log.xml -
===Flaky Test Report===

which according to the Pub/Sub documentation is a retryable error code: https://cloud.google.com/pubsub/docs/reference/error-codes