OpenEntityManagerInViewFilter + StreamingResponseBody: connection leak (HikariCP)

[PierreMardon]

Hi,

I faced a HikariCP leak that I thought was caused by my code but it appears that the use of StreamingResponseBody causes it by itself.

After removing all unneeded code, I come to a pretty straight forward reproduction case.

With StreamingResponseBody, one connection stays active forever in the CP. This connection is opened in one of my filter OncePerRequestFilter to fill a request-scoped component.

[UPDATE] see the next comment for a reproduction scenario (project zip included).

In a controller I directly call this service function:

    @Override
    @Transactional
    public ResponseEntity<StreamingResponseBody> download() {

        StreamingResponseBody stream = (o) -> {}

        HttpHeaders headers = new HttpHeaders();

        return new ResponseEntity(stream, headers, HttpStatus.OK);

    }

com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - After adding stats (total=6, active=1, idle=5, waiting=0)

When I just replace the type of response body to String, everything is ok:

    @Override
    @Transactional
    public ResponseEntity download() {

        HttpHeaders headers = new HttpHeaders();

        return new ResponseEntity("", headers, HttpStatus.OK);

    }

com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - After cleanup  stats (total=5, active=0, idle=5, waiting=0)

Here is my pom:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>com.someproject</groupId>
    <artifactId>central</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <packaging>jar</packaging>

    <name>central</name>
    <description>Central service for someproject Project</description>

    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.0.3.RELEASE</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
        <java.version>1.8</java.version>
        <spring-cloud.version>Finchley.RELEASE</spring-cloud.version>
    </properties>

    <repositories>
        <repository>
            <id>repo.com.someproject.common-lib.releases</id>
            <name>AWS Release Repository</name>
            <url>s3://repo.com.someproject.common-lib/releases</url>
        </repository>
        <repository>
            <id>repo.com.someproject.common-lib.snapshots</id>
            <name>AWS Snapshot Repository</name>
            <url>s3://repo.com.someproject.common-lib/snapshots</url>
        </repository>
    </repositories>

    <dependencyManagement>
      <dependencies>
        <dependency>
          <groupId>com.amazonaws</groupId>
          <artifactId>aws-java-sdk-bom</artifactId>
          <version>1.11.297</version>
          <type>pom</type>
          <scope>import</scope>
        </dependency>
          <dependency>
              <groupId>org.springframework.cloud</groupId>
              <artifactId>spring-cloud-dependencies</artifactId>
              <version>${spring-cloud.version}</version>
              <type>pom</type>
              <scope>import</scope>
          </dependency>
      </dependencies>
    </dependencyManagement>

    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter</artifactId>
        </dependency>

        <!-- For HTTP Endpoints -->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-aop</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-openfeign</artifactId>
        </dependency>
        <!-- For Model -->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-jpa</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.security</groupId>
            <artifactId>spring-security-test</artifactId>
            <version>5.1.2.RELEASE</version>
            <scope>test</scope>
        </dependency>

        <!--For Mail -->
        <dependency>
            <groupId>javax.mail</groupId>
            <artifactId>mail</artifactId>
            <version>1.4.7</version>
        </dependency>

        <dependency>
            <groupId>mysql</groupId>
            <artifactId>mysql-connector-java</artifactId>
            <version>5.1.40</version>
        </dependency>

        <!-- For database migration -->
        <dependency>
            <groupId>org.flywaydb</groupId>
            <artifactId>flyway-core</artifactId>
        </dependency>

        <dependency>
            <groupId>com.amazonaws</groupId>
            <artifactId>aws-java-sdk-s3</artifactId>
        </dependency>

        <dependency>
            <groupId>com.amazonaws</groupId>
            <artifactId>aws-java-sdk-sns</artifactId>
        </dependency>

        <dependency>
            <groupId>commons-codec</groupId>
            <artifactId>commons-codec</artifactId>
            <version>1.11</version>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>com.auth0</groupId>
            <artifactId>java-jwt</artifactId>
            <version>3.3.0</version>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-config</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-oauth2</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-sleuth</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-zipkin</artifactId>
        </dependency>
        <dependency>
            <groupId>org.modelmapper</groupId>
            <artifactId>modelmapper</artifactId>
            <version>2.3.1</version>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-hystrix</artifactId>
            <version>1.1.5.RELEASE</version>
        </dependency>
        <dependency>
            <groupId>io.springfox</groupId>
            <artifactId>springfox-swagger2</artifactId>
            <version>2.9.2</version>
            <scope>compile</scope>
        </dependency>
        <dependency>
            <groupId>io.springfox</groupId>
            <artifactId>springfox-swagger-ui</artifactId>
            <version>2.9.2</version>
            <scope>compile</scope>
        </dependency>
        <dependency>
            <groupId>com.someproject</groupId>
            <artifactId>common-lib</artifactId>
            <version>0.0.6-SNAPSHOT</version>
        </dependency>
        <dependency>
            <groupId>org.flywaydb.flyway-test-extensions</groupId>
            <artifactId>flyway-spring-test</artifactId>
            <version>5.2.1</version>
            <scope>test</scope>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
        <finalName>legacy-service</finalName>
        <extensions>
            <extension>
                <groupId>org.springframework.build</groupId>
                <artifactId>aws-maven</artifactId>
                <version>5.0.0.RELEASE</version>
            </extension>
        </extensions>
    </build>

</project>

Some config:

@Configuration
@EnableTransactionManagement
public class AppConfig implements WebMvcConfigurer {

    @Bean(name = "uploadTaskExecutor")
    public TaskExecutor specificTaskExecutor() {
        ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
        executor.setCorePoolSize(4);
        executor.setMaxPoolSize(4);
        executor.setThreadNamePrefix("Upload-");
        executor.initialize();
        return executor;
    }

    /**
     * Fix: submitting a multipart request (multipart/form-data) using PUT.
     * Normally, Spring just only support POST for submitting a multipart request.
     */
    @Bean
    public MultipartResolver multipartResolver() {
        return new StandardServletMultipartResolver() {
            @Override
            public boolean isMultipart(HttpServletRequest request) {
                String method = request.getMethod().toLowerCase();
                //By default, only POST is allowed. Since this is an 'update' we should accept PUT.
                if (!Arrays.asList("put", "post").contains(method)) {
                    return false;
                }
                String contentType = request.getContentType();
                return (contentType != null &&contentType.toLowerCase().startsWith("multipart/"));
            }
        };
    }

    @Bean
    JpaTransactionManager transactionManager(EntityManagerFactory entityManagerFactory) {
        JpaTransactionManager transactionManager = new JpaTransactionManager();
        transactionManager.setEntityManagerFactory(entityManagerFactory);
        return transactionManager;
    }

    @Bean
    public FilterRegistrationBean registerOpenSessionInViewFilterBean() {
        FilterRegistrationBean registrationBean = new FilterRegistrationBean();
        OpenEntityManagerInViewFilter filter = new OpenEntityManagerInViewFilter();
        registrationBean.setFilter(filter);
        registrationBean.setOrder(5);
        return registrationBean;
    }

    @Bean
    public FilterRegistrationBean requestContextFilterRegistration(){
        FilterRegistrationBean filter = new FilterRegistrationBean();
        filter.setFilter(requestContextFilter());
        filter.setOrder(0);
        return filter;
    }

    @Bean
    public RequestContextFilter requestContextFilter(){
        return new RequestContextFilter();
    }

    @Bean
    public MessageSource messageSource() {
        ResourceBundleMessageSource messageSource = new ResourceBundleMessageSource();
        messageSource.setBasename("message/error_messages");
        messageSource.setDefaultEncoding("UTF-8");
        return messageSource;
    }

    @Override
    public void configureContentNegotiation(ContentNegotiationConfigurer configurer) {
        configurer.defaultContentType(MediaType.APPLICATION_XML);
    }

    @Override
    public void addCorsMappings(CorsRegistry registry) {

        //TODO: to modify after other dev
       registry.addMapping("/**").allowedOrigins("*").allowedMethods("PUT", "DELETE","PATCH","GET","POST");
    }
}

@Configuration
@EnableResourceServer
@EnableGlobalMethodSecurity(prePostEnabled = true)
@Profile("!test")
public class ResourceServerConfiguration extends BaseResourceServerConfiguration {

    @Value("${auth-server.url}")
    private String authEndpoint;

    @Value("${security.oauth2.client.client-id}")
    private String clientId;

    @Value("${security.oauth2.client.client-secret}")
    private String clientSecret;

    @Override
    public void configure(ResourceServerSecurityConfigurer resources) throws Exception {
        resources.resourceId("ms/legacy");
    }

    @Bean
    public ResourceServerTokenServices tokenService() {
        RemoteTokenServices tokenServices = new RemoteTokenServices();
        tokenServices.setClientId(clientId);
        tokenServices.setClientSecret(clientSecret);
        tokenServices.setCheckTokenEndpointUrl(authEndpoint + "/uaa/oauth/check_token");
        return tokenServices;
    }
}

My yml props:

server:
  port: 8086
  servlet:
    context-path: /
spring:
  profiles: local
  application:
    name: legacy-central-service
  cloud:
    config:
      allow-override: true
      override-none: true
  zipkin:
    enabled: true
    base-url: http://localhost:8087/
  sleuth:
    sampler:
      probability: 1.0
    feign:
      enabled: true
  # JPA Configuration
  jpa:
    hibernate:
      ddl-auto: validate
    properties:
      hibernate:
        dialect: org.hibernate.dialect.MySQL5InnoDBDialect
  datasource:
    url: jdbc:mysql://localhost:3306/legacy_service?autoReconnect=true&useSSL=false
    username: username
    password: password
    hikari:
      maximum-pool-size: 20
      minimum-idle: 5
      idle-timeout: 600000
      max-lifetime: 1800000
      leak-detection-threshold: 60000
  thymeleaf:
    cache: false
    mode: HTML
    encoding: UTF-8
  flyway:
    baseline-on-migrate: true

  # For Upload File - the max support file size is 500MB
  servlet:
    multipart:
      enabled: true
      max-file-size: 500MB
      max-request-size: 500MB
      location: /tmp
  http:
    multipart:
      enabled: false

proj:
  central:
    mail:
      from: contact@proj.com
      host: localhost
      port: 25
      username:
      password:
      debug: false
    video:
      renderscore_tool_location: /tmp

eureka:
  instance:
    hostname: localhost
    port: 8081
  client:
    registerWithEureka: true
    fetchRegistry: true
    serviceUrl:
      defaultZone: http://${eureka.instance.hostname}:${eureka.instance.port}/eureka/
  server:
    wait-time-in-ms-when-sync-empty: 3000

auth-server:
  url: http://localhost:8083

security:
  oauth2:
    client:
      client-id: proj
      client-secret: proj
    resource:
      user-info-uri: http://localhost:8082/uaa
  jackson:
    property-naming-strategy: SNAKE_CASE
    serialization:
      indent-output: true

authorization-service:
  ribbon:
    listOfServers: http://localhost:8083
legacy-central-service:
  ribbon:
    listOfServers: http://localhost:8086
static-data-service:
  ribbon:
    listOfServers: http://localhost:7071

[PierreMardon]

I removed most of the classes of my project so that you can easily reproduce and inspect the relevant code only.

sample-bug.zip

EDIT: the leak is caused by the use of OpenEntityManagerInViewFilter that fails to release the connection when StreamingResponseBody is used. Commenting it in AppConfig.java solves the leak. Still, I need this filter :P

To be able to start the app, you have to create an empty schema test_schema in a local MySQL database (configure your connection in application.yml).

• issue a GET request to http://localhost:8086/ws3/file/data/?file=ANY_STRING.
• the GeneralFilter class is the only one using JPA and calling the MyUserRepository, it's the only place where HikaryCP is asked for a connection.
• you'll see the hikary logs:

2019-01-27 18:54:01.350 DEBUG 18739 --- [l-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Before cleanup stats (total=6, active=1, idle=5, waiting=0)
2019-01-27 18:54:01.351 DEBUG 18739 --- [l-1 housekeeper] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - After cleanup  stats (total=6, active=1, idle=5, waiting=0)

And later the leak detection:

java.lang.Exception: Apparent connection leak detected
    at com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:128) ~[HikariCP-2.7.9.jar:na]
    at org.hibernate.engine.jdbc.connections.internal.DatasourceConnectionProviderImpl.getConnection(DatasourceConnectionProviderImpl.java:122) ~[hibernate-core-5.2.17.Final.jar:5.2.17.Final]
    at org.hibernate.internal.NonContextualJdbcConnectionAccess.obtainConnection(NonContextualJdbcConnectionAccess.java:35) ~[hibernate-core-5.2.17.Final.jar:5.2.17.Final]
    at org.hibernate.resource.jdbc.internal.LogicalConnectionManagedImpl.acquireConnectionIfNeeded(LogicalConnectionManagedImpl.java:106) ~[hibernate-core-5.2.17.Final.jar:5.2.17.Final]
    at org.hibernate.resource.jdbc.internal.LogicalConnectionManagedImpl.getPhysicalConnection(LogicalConnectionManagedImpl.java:136) ~[hibernate-core-5.2.17.Final.jar:5.2.17.Final]
    at org.hibernate.engine.jdbc.internal.StatementPreparerImpl.connection(StatementPreparerImpl.java:47) ~[hibernate-core-5.2.17.Final.jar:5.2.17.Final]
    at org.hibernate.engine.jdbc.internal.StatementPreparerImpl$5.doPrepare(StatementPreparerImpl.java:145) ~[hibernate-core-5.2.17.Final.jar:5.2.17.Final]
    at org.hibernate.engine.jdbc.internal.StatementPreparerImpl$StatementPreparationTemplate.prepareStatement(StatementPreparerImpl.java:171) ~[hibernate-core-5.2.17.Final.jar:5.2.17.Final]
    at org.hibernate.engine.jdbc.internal.StatementPreparerImpl.prepareQueryStatement(StatementPreparerImpl.java:147) ~[hibernate-core-5.2.17.Final.jar:5.2.17.Final]
    at org.hibernate.loader.Loader.prepareQueryStatement(Loader.java:1985) ~[hibernate-core-5.2.17.Final.jar:5.2.17.Final]
    at org.hibernate.loader.Loader.executeQueryStatement(Loader.java:1915) ~[hibernate-core-5.2.17.Final.jar:5.2.17.Final]
    at org.hibernate.loader.Loader.executeQueryStatement(Loader.java:1893) ~[hibernate-core-5.2.17.Final.jar:5.2.17.Final]
    at org.hibernate.loader.Loader.doQuery(Loader.java:938) ~[hibernate-core-5.2.17.Final.jar:5.2.17.Final]
    at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:341) ~[hibernate-core-5.2.17.Final.jar:5.2.17.Final]
    at org.hibernate.loader.Loader.doList(Loader.java:2692) ~[hibernate-core-5.2.17.Final.jar:5.2.17.Final]
    at org.hibernate.loader.Loader.doList(Loader.java:2675) ~[hibernate-core-5.2.17.Final.jar:5.2.17.Final]
    at org.hibernate.loader.Loader.listIgnoreQueryCache(Loader.java:2507) ~[hibernate-core-5.2.17.Final.jar:5.2.17.Final]
    at org.hibernate.loader.Loader.list(Loader.java:2502) ~[hibernate-core-5.2.17.Final.jar:5.2.17.Final]
    at org.hibernate.loader.hql.QueryLoader.list(QueryLoader.java:502) ~[hibernate-core-5.2.17.Final.jar:5.2.17.Final]
    at org.hibernate.hql.internal.ast.QueryTranslatorImpl.list(QueryTranslatorImpl.java:392) ~[hibernate-core-5.2.17.Final.jar:5.2.17.Final]
    at org.hibernate.engine.query.spi.HQLQueryPlan.performList(HQLQueryPlan.java:216) ~[hibernate-core-5.2.17.Final.jar:5.2.17.Final]
    at org.hibernate.internal.SessionImpl.list(SessionImpl.java:1490) ~[hibernate-core-5.2.17.Final.jar:5.2.17.Final]
    at org.hibernate.query.internal.AbstractProducedQuery.doList(AbstractProducedQuery.java:1445) ~[hibernate-core-5.2.17.Final.jar:5.2.17.Final]
    at org.hibernate.query.internal.AbstractProducedQuery.list(AbstractProducedQuery.java:1414) ~[hibernate-core-5.2.17.Final.jar:5.2.17.Final]
    at org.hibernate.query.internal.AbstractProducedQuery.getSingleResult(AbstractProducedQuery.java:1463) ~[hibernate-core-5.2.17.Final.jar:5.2.17.Final]
    at org.hibernate.query.criteria.internal.compile.CriteriaQueryTypeQueryAdapter.getSingleResult(CriteriaQueryTypeQueryAdapter.java:107) ~[hibernate-core-5.2.17.Final.jar:5.2.17.Final]
    at org.springframework.data.jpa.repository.query.JpaQueryExecution$SingleEntityExecution.doExecute(JpaQueryExecution.java:214) ~[spring-data-jpa-2.0.8.RELEASE.jar:2.0.8.RELEASE]
    at org.springframework.data.jpa.repository.query.JpaQueryExecution.execute(JpaQueryExecution.java:91) ~[spring-data-jpa-2.0.8.RELEASE.jar:2.0.8.RELEASE]
    at org.springframework.data.jpa.repository.query.AbstractJpaQuery.doExecute(AbstractJpaQuery.java:136) ~[spring-data-jpa-2.0.8.RELEASE.jar:2.0.8.RELEASE]
    at org.springframework.data.jpa.repository.query.AbstractJpaQuery.execute(AbstractJpaQuery.java:125) ~[spring-data-jpa-2.0.8.RELEASE.jar:2.0.8.RELEASE]
    at org.springframework.data.repository.core.support.RepositoryFactorySupport$QueryExecutorMethodInterceptor.doInvoke(RepositoryFactorySupport.java:590) ~[spring-data-commons-2.0.8.RELEASE.jar:2.0.8.RELEASE]
    at org.springframework.data.repository.core.support.RepositoryFactorySupport$QueryExecutorMethodInterceptor.invoke(RepositoryFactorySupport.java:578) ~[spring-data-commons-2.0.8.RELEASE.jar:2.0.8.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185) ~[spring-aop-5.0.7.RELEASE.jar:5.0.7.RELEASE]
    at org.springframework.data.projection.DefaultMethodInvokingMethodInterceptor.invoke(DefaultMethodInvokingMethodInterceptor.java:59) ~[spring-data-commons-2.0.8.RELEASE.jar:2.0.8.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185) ~[spring-aop-5.0.7.RELEASE.jar:5.0.7.RELEASE]
    at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:294) ~[spring-tx-5.0.7.RELEASE.jar:5.0.7.RELEASE]
    at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98) ~[spring-tx-5.0.7.RELEASE.jar:5.0.7.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185) ~[spring-aop-5.0.7.RELEASE.jar:5.0.7.RELEASE]
    at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:139) ~[spring-tx-5.0.7.RELEASE.jar:5.0.7.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185) ~[spring-aop-5.0.7.RELEASE.jar:5.0.7.RELEASE]
    at org.springframework.data.jpa.repository.support.CrudMethodMetadataPostProcessor$CrudMethodMetadataPopulatingMethodInterceptor.invoke(CrudMethodMetadataPostProcessor.java:135) ~[spring-data-jpa-2.0.8.RELEASE.jar:2.0.8.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185) ~[spring-aop-5.0.7.RELEASE.jar:5.0.7.RELEASE]
    at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92) ~[spring-aop-5.0.7.RELEASE.jar:5.0.7.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185) ~[spring-aop-5.0.7.RELEASE.jar:5.0.7.RELEASE]
    at org.springframework.data.repository.core.support.SurroundingTransactionDetectorMethodInterceptor.invoke(SurroundingTransactionDetectorMethodInterceptor.java:61) ~[spring-data-commons-2.0.8.RELEASE.jar:2.0.8.RELEASE]
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185) ~[spring-aop-5.0.7.RELEASE.jar:5.0.7.RELEASE]
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) ~[spring-aop-5.0.7.RELEASE.jar:5.0.7.RELEASE]
    at com.sun.proxy.$Proxy96.findFirstByUuid(Unknown Source) ~[na:na]
    at com.myproject.filter.GeneralFilter.addRequestInfo(GeneralFilter.java:39) ~[classes/:na]
...

You can experiment that changing the ResponseEntity body type to String and returning an empty string fixes the leak.

[PierreMardon]

I'm sorry for this bug report, in fact adding @Async annotation on my service function solves the issue, so it's not a bug, my bad !

[PierreMardon]

Ok so indeed it would solve the leak but also any real IO transfert would fail (try to stream any resource in the body to see it). So bad solution, still a bug, sorry for the premature closure!

[wilkinsona]

Thanks for the sample but there's still quite a lot to it. If you have time to strip it down further, including replacing the need for MySQL with an in-memory database (H2 or HSQLDB), it will increase the changes of someone having the time to investigate.

[PierreMardon]

I'm sorry to be that guy, but I prefer to tell you directly that I won't be able to take time to enhance the sample.

[wilkinsona]

The problem is caused by a mis-configuration of the OpenEntityManagerInViewFilter. Its dispatcher types have not been customised so it is only called on request dispatch and is not called on async dispatch. As a result, the session that it opens is never closed which causes the database connection to leak.

[alexpartsch]

So was this fixed? Still running in CP leakage when using StreamingRequestBody with JPA. Not sure, since the status invalid was applied on this issue.

[wilkinsona]

@alexpartsch As far as we could determine, there was nothing to fix in Spring Boot as the problem was caused by misconfiguration in the application. Please see my comment immediately above yours for details.

[alexpartsch]

@wilkinsona I read your comment, but was under assumption that the part of Spring Boot handling the StreamingRequestBody threading has to "customise the dispatcher types" (due to my lack of understanding of OpenEntityManagerInViewFilter I'm not aware of what these dispatcher types are and where one can customise them, but trying to find out now.), but as I understand from your last comment, the framework user is probably in charge of that.

[wilkinsona]

The dispatcher types are part of the servlet spec and they govern the types of dispatch for which a filter is invoked. When you're using a StreamingResponseBody an async dispatch is performed so you need to be sure that the filter's been registered for async dispatches otherwise it won't get a chance to clean up properly once the response has been sent. You can configure the dispatcher types using a FilterRegistrationBean. Note that if you are using Spring Boot 2.3, the OpenEntityManagerInViewFilter, which is a OncePerRequestFilter, will be registered for async dispatches by default due to the changes made in https://github.com/spring-projects/spring-boot/issues/18953.

[alexpartsch]

Thanks for the infos! I understand more know.

I'm running Spring Boot 2.3.3, but it seems no OpenEntityManagerInViewFilter is registered or at least executed on any request. I tried to define it myself but fail with a LazyInitialisationException due to no session being open in the async requests. Here is my filter bean definition:

    @Bean
    @Primary
    public FilterRegistrationBean<OpenEntityManagerInViewFilter> openEntityManagerInViewFilterFilterRegistrationBean() {
        var filter = new OpenEntityManagerInViewFilter();
        var registrationBean = new FilterRegistrationBean<>(filter);
        registrationBean.setAsyncSupported(true);
        registrationBean.addUrlPatterns("/*");
        registrationBean.setDispatcherTypes(DispatcherType.REQUEST, DispatcherType.ASYNC);
        return registrationBean;
    }

I use the default EntityManagerFactory configuration with hibernate so I don't think I need to configure more 🤔 ... but it still seems to not be executed. Since you say 2.3 already defines it as ASNY my leak can be caused by something else then 🙈

[wilkinsona]

my leak can be caused by something else then 🙈

It certainly sounds like it. If you can provide us with a minimal sample project that reproduces the leak, please open a new issue and we can take a look.

[alexpartsch]

Okay, so after defining my Bean as described above another error due to my transaction management was rissen. Fixing that, the new OpenEntityManagerInViewFilter seems to work fine now. Are you sure that DispatcherType.ASYNC is enabled by default in Spring Boot 2.3.3?

[wilkinsona]

As sure as I can be. There's a test that verifies that's the case:

https://github.com/spring-projects/spring-boot/blob/602e62998eb2254517d695fa67246399ba40b8bf/spring-boot-project/spring-boot/src/test/java/org/springframework/boot/web/servlet/FilterRegistrationBeanTests.java#L85-L92

[alexpartsch]

Okay, found the issue, just for further reference: When testing with MockMvc and StreamingRequestBody one should use the asyncDispatch RequestBuilder for retrieving and properly handling the response (see here). I removed my OpenEntityManagerInViewFilter filter registration and it still worked 🙆‍♂️

Thanks @wilkinsona for the help!