aws-sdk/client-kinesis consuming too much memory under load

[Guru3101]

Checkboxes for prior research

• [X] I've gone through Developer Guide and API reference
• [X] I've checked AWS Forums and StackOverflow.
• [X] I've searched for previous similar issues and didn't find any solution.

Describe the bug

In one of our app we are using kinesis to push our platform feeds to Kinesis , under some load (3000 to 3500 requests/min max) the heap consumption spikes gradually heavily and app is getting crashed . we took the heapdump when memory is spiking and looks like the kinesis sdk is not flushing the ClientRequest Object and holding it in the memory at least when big volume comes in, attaching the heap screenshot for reference

13K such ClientRequest Object are sitting in the memory when we took heap dump and here is one such memory spike of our container

Here is quick snippets of our script

const { KinesisClient, PutRecordCommand} = require(@aws-sdk/client-kinesis);
const { NodeHttpHandler } = require("@smithy/node-http-handler");
const https = require('https');

//Initialization
let kinesis = new KinesisClient({
    region: 'us-east-1',
    retryMode: 'standard',
    requestHandler: new NodeHttpHandler({
        httpsAgent: new https.Agent({
            keepAlive:true,
            maxSockets: 200
        })
    })
})

//pushing to kinesis
await kinesis.send(new PutRecordCommand(params));

this app consumes the feeds at the rate of ~100req/sec from Kafka and pushes to kinesis so to match that i added a http agent with max socket value as 200 and keepalive as true to reuse connection.

SDK version number

@aws-sdk/client-kinesis@3.606.0

Which JavaScript Runtime is this issue in?

Node.js

Details of the browser/Node.js/ReactNative version

node v18.20.4

Reproduction Steps

const { KinesisClient, PutRecordCommand} = require(@aws-sdk/client-kinesis);
const { NodeHttpHandler } = require("@smithy/node-http-handler");
const https = require('https');

//Initialization
let kinesis = new KinesisClient({
    region: 'us-east-1',
    retryMode: 'standard',
    requestHandler: new NodeHttpHandler({
        httpsAgent: new https.Agent({
            keepAlive:true,
            maxSockets: 200
        })
    })
})

//pushing to kinesis
await kinesis.send(new PutRecordCommand(params));

Observed Behavior

App crashes due to heavy heap usage

under some load (3000 to 3500 requests/min max) the heap consumption spikes gradually heavily and app is getting crashed . we took the heapdump when memory is spiking and looks like the kinesis sdk is not flushing the ClientRequest Object and holding it in the memory at least when big volume comes in, attaching the heap screenshot for reference

13K such ClientRequest Object are sitting in the memory when we took heap dump and here is one such memory spike of our container

Here is quick snippets of our script

const { KinesisClient, PutRecordCommand} = require(@aws-sdk/client-kinesis);
const { NodeHttpHandler } = require("@smithy/node-http-handler");
const https = require('https');

//Initialization
let kinesis = new KinesisClient({
    region: 'us-east-1',
    retryMode: 'standard',
    requestHandler: new NodeHttpHandler({
        httpsAgent: new https.Agent({
            keepAlive:true,
            maxSockets: 200
        })
    })
})

//pushing to kinesis
await kinesis.send(new PutRecordCommand(params));

this app consumes the feeds at the rate of ~100req/sec from Kafka and pushes to kinesis so to match that i added a http agent with max socket value as 200 and keepalive as true to reuse connection.

Expected Behavior

Memory consumption to be stable

Possible Solution

No response

Additional Information/Context

No response

[aBurmeseDev]

Hi @Guru3101 - thanks for reaching out. I apologize for the inconvenience you're experiencing.

To better understand and troubleshoot the problem, it would be helpful if you could provide us with more details about your environment and workflow. Specifically, please share the following information:

• Are you running your application on AWS Lambda?
• Could you describe your workflow involving Apache Kafka?
• Is this issue occurring continuously, or does it happen intermittently?

Having a clear understanding of your setup and the frequency of the issue will greatly assist us in diagnosing the root cause and providing an effective solution. Please provide as much relevant information as possible to help us investigate further.

[github-actions[bot]]

This issue has not received a response in 1 week. If you still think there is a problem, please leave a comment to avoid the issue from automatically closing.

[github-actions[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs and link to relevant comments in this thread.