GoogleCloudPlatform / spring-cloud-gcp

New home for Spring Cloud GCP development starting with version 2.0.
Apache License 2.0
414 stars 305 forks source link

Subscribers don't reconnect to the subscription when there is no traffic #2599

Closed ArtEE98 closed 1 week ago

ArtEE98 commented 7 months ago

I am using StreamingPull mechanism to receive messages from the Pub/Sub.

Here is only 2 overriding for default SubscriberFactory settings:

@Bean
    public SubscriberFactory subscriberFactory(
            GcpProjectIdProvider id,
            TransportChannelProvider channelProvider,
            CredentialsProvider credentialsProvider) throws IOException {
        PubSubConfiguration config = new PubSubConfiguration();
        config.initialize(id.getProjectId());
        config.getSubscriber().setParallelPullCount(2); //overriding1
        config.getSubscriber().setMaxDurationPerAckExtension(0); //overriding2
        DefaultSubscriberFactory factory = new DefaultSubscriberFactory(id, config);
        factory.setChannelProvider(channelProvider);
        factory.setPullEndpoint(url());
        factory.setCredentialsProvider(credentialsProvider);
        return factory;
    }

The second overriding helps me to resolve this issue: https://stackoverflow.com/questions/77711978/my-subscriber-receive-a-message-from-gcp-subscription-with-exactly-60-seconds-de

I don't have any other overriding, but when there is no trafic I get this picute of number_of_open_streaming_pull subscription metric:

Screenshot 2024-02-13 at 17 54 46

I know that Pub/Sub prefers to avoid a long-running sticky connection and client library should reopens a StreamingPull connection.. It works well when there is an active traffic. Any advice I can achive a stable reopens even when there is no traffic?

I found some issues about this: https://github.com/spring-attic/spring-cloud-gcp/issues/1005 - resolved as internal pub/sub issue https://github.com/spring-attic/spring-cloud-gcp/issues/2552 - adviced to use Synchronous Pull, but I want to go further with Asynchronous pull because for my app lower latency and higher throughput are very important.

We use spring-cloud-gcp-pubsub:5.0

ldetmer commented 3 weeks ago

Hi @ArtEE98, is this still an issue? Can you share a little more information on the problem please? Is the issue that you want the stream to show up in number_of_open_streaming_pull even if there is no active traffic? Can I ask what your specific use case is?

ldetmer commented 1 week ago

I performed a test where I left a subscriber running for 2 hours without traffic. After 2 hours I was able to send a message and have the subscriber reconnect and receive the message. I will close out for now, but @ArtEE98 if you are able to reproduce please send me additional details about your setup.