search

amazon-archives / kinesis-storm-spout

Kinesis spout for Storm
Other
106 stars 64 forks source link

spout slows down #10

Closed asharif closed 9 years ago

asharif commented 9 years ago

not sure what I'm doing wrong. I have my topology configured with 10 as the parallel hint for the spout. I create a stream with 10 shards to test. I've tried having the producer partition the records out in the following ways:

UUID - so lots of partitions % 10 - so that it is distributed evenly among shards % 30 % 2

No matter what partition key I choose the data seems to scream into storm...I'll get ~5-8k records in and then it slows down to a halt. When I say halt I mean 10s-20s of records. Then it will burst again for only a couple hundred and slow down again.

I feel like I'm not understanding something. Any help would be appreciated.

asharif commented 9 years ago

Going to close this issue. It was actually my fault. Error was in another one of my bolts not in this spout.