googleapis / java-pubsub

Apache License 2.0
127 stars 90 forks source link

Duplicate messages are published and throws TimeoutException #2175

Open sann3 opened 2 weeks ago

sann3 commented 2 weeks ago

Environment details

  1. google-cloud-pubsub
  2. Linux version 5.15.153.1-microsoft-standard-WSL2
  3. Java Corretto-21.0.3.9.1 (build 21.0.3+9-LTS)
  4. version(s): com.google.cloud:google-cloud-pubsub:1.131.0

Steps to reproduce

  1. Some times a message is published multiple times (upto 7 times), since it is not getting the acknowledge back. Finally getting timeout exception.
  2. Not happening always, only happens to some random messages

Code example

    application.yaml
      max-buffered-messages: 1
      initial-retry-delay-in-millis: 100
      retry-delay-multiplier: 2.0
      max-retry-delay-in-seconds: 60
      initial-rpc-timeout-in-seconds: 1
      rpc-timeout-multiplier: 1.0
      max-rpc-timeout-in-seconds: 600
      total-timeout-in-seconds: 600

       RetrySettings defaultRetrySettings =
                RetrySettings.newBuilder()
                        .setInitialRetryDelay(
                                Duration.ofMillis(pubSubProperties.getInitialRetryDelayInMillis()))
                        .setRetryDelayMultiplier(pubSubProperties.getRetryDelayMultiplier())
                        .setMaxRetryDelay(
                                Duration.ofSeconds(pubSubProperties.getMaxRetryDelayInSeconds()))
                        .setInitialRpcTimeout(
                                Duration.ofSeconds(
                                        pubSubProperties.getInitialRpcTimeoutInSeconds()))
                        .setRpcTimeoutMultiplier(pubSubProperties.getRpcTimeoutMultiplier())
                        .setMaxRpcTimeout(
                                Duration.ofSeconds(pubSubProperties.getMaxRpcTimeoutInSeconds()))
                        .setTotalTimeout(
                                Duration.ofSeconds(pubSubProperties.getTotalTimeoutInSeconds()))
                        .build();
        Publisher publisher =
                builder.setEnableMessageOrdering(true)
                        .setRetrySettings(defaultRetrySettings)
                        .build();
        ApiFuture<String> future = publisher.publish(pubsubMessage);
        future.get(5000, TimeUnit.MILLISECONDS);

Stack trace

java.util.concurrent.TimeoutException: Waited 5000 milliseconds (plus 182053 nanoseconds delay) for com.google.api.core.AbstractApiFuture$InternalSettableFuture@4aa80674[status=PENDING]
    at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:533)
    at com.google.api.core.AbstractApiFuture.get(AbstractApiFuture.java:58)
    at com.company.feature.pubsub.PublisherDelegate.publishDataToPubSub(PublisherDelegate.java:91)
    ... 134 common frames omitted