search

spring-cloud / spring-cloud-stream-binder-kafka

Spring Cloud Stream binders for Apache Kafka and Kafka Streams
Apache License 2.0
331 stars 302 forks source link

Live producer threads and dynamic-destination-cache-size property #1198

Closed ferblaca closed 2 years ago

ferblaca commented 2 years ago

We have detected a problem in the increase of resources when there are configured in an application more bindings out (producers) which value has the property spring.cloud.stream.dynamic-destination-cache-size (default 10).

The problem is that the created threads grow without control and are not released, and in some cases consuming memory and even occupying all the descriptors of the system causing the application to fail.

Is this an expected behavior? the only solution is to increase the value of the spring.cloud.stream.dynamic-destination-cache-size property?

Attached is an application that reproduces the case we explained and we add a screenshot with the threads created are only 2 binding and spring.cloud.stream.dynamic-destination-cache-size=1:

visualvm-threads

demoDynamicDestinationCache.zip

ferblaca commented 2 years ago

We have noticed that updating to new version of Spring Cloud 2021.0.1 the issue https://github.com/spring-cloud/spring-cloud-stream/issues/2266 has been resolved, which I believe indicates the same behavior as this issue.

In fact we have verified that when defining the producers created with StreamBridge in the spring.cloud.stream.output-bindings property the number of threads does NOT grow.

On the other hand, it would be nice if release notes 2021.0.1 would indicate that there are updates to the spring-cloud-stream and spring-cloud-function library.

sobychacko commented 2 years ago

@ferblaca Good to hear that the issue has been resolved once you upgraded. We will see if we can update the release notes. In the meantime, can we close this issue? Is there anything more to be done on this?