search

awslabs / amazon-kinesis-scaling-utils

The Kinesis Scaling Utility is designed to give you the ability to scale Amazon Kinesis Streams in the same way that you scale EC2 Auto Scaling groups – up or down by a count or as a percentage of the total fleet. You can also simply scale to an exact number of Shards. There is no requirement for you to manage the allocation of the keyspace to Shards when using this API, as it is done automatically.
Apache License 2.0
338 stars 95 forks source link

Infinite loop attempting to get a report for stream with more than 1000 shards. #106

Open creechy opened 3 years ago

creechy commented 3 years ago

I have encountered a problem trying to list the shards of a stream with over 1000 shards using the "report" function. When I ran 

java -Xmx2g -cp KinesisScalingUtils-.9.8.1-complete.jar -Dstream-name=my-stream-name -Dscaling-action=report ScalingClient

The utility hung for a while then finally crashed with 

02:07:08.021 [main] INFO  c.a.s.k.scaling.StreamScalingUtils - Listing Stream my-stream-name
Exception in thread "main" java.lang.OutOfMemoryError: GC overhead limit exceeded
        at java.util.Collections.unmodifiableMap(Collections.java:1435)

This stream had 1281 active shards in it. After some debugging, I believe I found the problem. The utility seemed to be retrieving the same chunk of the first 1000 shards. I fixed this with the following change in StreamScalingUtils.java I changed code around line 163:

        req = ListShardsRequest.builder().nextToken(result.nextToken()).build();

to

        builder = ListShardsRequest.builder();
        req = builder.nextToken(result.nextToken()).build();

I guess you need to update the builder because its used in subsequent iterations of the loop to get more chunks.

IanMeyers commented 3 years ago

Sorry - this is very ugly and would never have worked. Please download .9.8.6 where this should be fixed?