search

awslabs / amazon-kinesis-client

Client library for Amazon Kinesis
Apache License 2.0
635 stars 461 forks source link

OutOfMemory due to huge number of 'org.apache.http.impl.conn.PoolingHttpClientConnectionManager' instances referenced by 'idle-connection-reaper' thread #1292

Closed mahudees closed 3 months ago

mahudees commented 3 months ago

We are facing OOM error in a customer setup with AWS java sdk, checking the heap dump, looks like there are huge number of org.apache.http.impl.conn.PoolingHttpClientConnectionManager instances are referenced by idle-connection-reaper thread. Checked the code, clients are created and closed properly (checked this in https://github.com/aws/aws-sdk-java-v2/issues/1679).

There is one similar issue related to this, https://github.com/awslabs/amazon-kinesis-client/issues/981, which was observed when using WebIdentityTokenFileCredentialsProvider, is this applicable for InstanceProfileCredentialsProvider as well?

Our application creates a new client and use for connecting to AWS, when a predefined time is passed (10 minutes), the exisiting clients is closed and a new client is created. Tried to reproduce, but the issue is not reproduced in local setup.

mahudees commented 3 months ago

Closing here as opened a new discussion https://github.com/aws/aws-sdk-java-v2/discussions/5083 in aws-sdk-java-v2.