Socket Timeout Followed by Unexpected Automatic Retry Despite Retry Being Disabled

[Michele971]

Our bug We are encountering issues in our Java Spring Boot application where a SocketTimeoutException occurs during database operations, followed by an unexpected automatic retry, even though we have explicitly disabled retry in our configuration.

The flow The issue arises in a setup where the frontend (browser) sends a request to the gateway, which then forwards the request to another microservice X that performs database operations. Although the gateway timeout is set to 15 minutes, we observe that the microservice takes 1 minute and 5 seconds to process the request before throwing a SocketTimeoutException. After the timeout error, the system automatically retries the failed request even though we have explicitly disabled retry functionality in our configuration, and an other call is made to the same endpoint.

Tests We try to delay the Oracle database operation (using DBMS_LOCK.SLEEP) to simulate a slow query. We think that use the Thread.sleep() java method is not useful.

The error ERROR org.springframework.web.server.adapter.HttpWebHandlerAdapter [https-jsse-nio-443-exec-5] [3adb5fc8] Error [java.net.SocketTimeoutException] for HTTP GET "/getData", but ServerHttpResponse already committed (200 OK)

Expected Behavior At least no automatic retry should occur unless explicitly configured. Since we have disabled retry, the request should fail after the timeout without retrying. Moreover, by default from Spring Cloud Gateway, the retry should be disabled!

Question The microservice X still returns a 200 OK response to the client, even though the request encountered a SocketTimeoutException. Why? How can we fix this bug? We would like to avoid the retry and also avoid the socket timeout exception. How can we simulate this behaviour in our local enviroment? Any idea?

Thanks in advance!

[spencergibb]

Please provide more context. You have mentioned various settings, please provide the properties and values set as well as your route configuration. Please provide the full stack trace for your error.

[Michele971]

In my Gateway application.properties I tried to disable the retry functionality globally through this configuration properties:

spring.autoconfigure.exclude=org.springframework.cloud.client.loadbalancer.RetryAutoConfiguration
spring.cloud.config.retry.initial-interval = 0
spring.cloud.config.retry.max-attempts = 0 
spring.cloud.config.retry.max-interval = 0
spring.cloud.config.retry.multiplier = 0
spring.cloud.gateway.filter.retry.enabled=false
spring.cloud.loadbalancer.retry.enabled=false

What do you mean for "route configuration"? I already provide the full stack trace. I don't have any error in my microserice X, only in my gateway I can find the Socket Timeout Exception...

Thanks again @spencergibb!

[spencergibb]

You said this is a gateway application. The basic function of the gateway is routing. How is yours configured?

Also what versions of boot and spring cloud are you using?

[Michele971]

Oh yes, got it, spring boot version is 2.7.5 spring cloud gateway version is 3.1.8

The route configuration is something like:

@SpringBootApplication
@EnableDiscoveryClient
public class MyGatewayApplication {

    public static void main(String[] args) {
        SpringApplication.run(MyGatewayApplication.class, args);
    }

    @Bean
    public RouteLocator customRouteLocator(RouteLocatorBuilder builder) {
        return builder.routes()
                .route("microserviceX", r -> r.path("/microserviceX/**")
                        .filters(f -> f.stripPrefix(1)
                                .removeResponseHeader("Sensitive-Header")
                                .removeResponseHeader("Another-Sensitive-Header")
                                .removeRequestHeader("Request-Specific-Header")
                                .metadata("responseTimeout", 5000)  
                                .metadata("connectionTimeout", 2000)) 
                        .uri("lb://microserviceX"))
                .route("microserviceY", r -> r.path("/microserviceY/**")
                        .filters(f -> f.stripPrefix(1)
                                .removeResponseHeader("Sensitive-Header")
                                .removeResponseHeader("Another-Sensitive-Header")
                                .removeRequestHeader("Request-Specific-Header"))
                        .uri("microserviceY"))
                .build();
    }
}

[spencergibb]

the system automatically retries the failed request even though we have explicitly disabled retry functionality in our configuration, and an other call is made to the same endpoint.

How do you know this? and how do you know it was gateway that did it? Various http clients will retry by default.

Gateway does not retry without adding the retry filter.

Unfortunately, the versions mentioned are no longer supported.

[spring-cloud-issues]

If you would like us to look at this issue, please provide the requested information. If the information is not provided within the next 7 days this issue will be closed.

[spring-cloud-issues]

Closing due to lack of requested feedback. If you would like us to look at this issue, please provide the requested information and we will re-open the issue.