not sure what I'm doing wrong. I have my topology configured with 10 as the parallel hint for the spout. I create a stream with 10 shards to test. I've tried having the producer partition the records out in the following ways:
UUID - so lots of partitions
% 10 - so that it is distributed evenly among shards
% 30
% 2
No matter what partition key I choose the data seems to scream into storm...I'll get ~5-8k records in and then it slows down to a halt. When I say halt I mean 10s-20s of records. Then it will burst again for only a couple hundred and slow down again.
I feel like I'm not understanding something. Any help would be appreciated.
not sure what I'm doing wrong. I have my topology configured with 10 as the parallel hint for the spout. I create a stream with 10 shards to test. I've tried having the producer partition the records out in the following ways:
UUID - so lots of partitions % 10 - so that it is distributed evenly among shards % 30 % 2
No matter what partition key I choose the data seems to scream into storm...I'll get ~5-8k records in and then it slows down to a halt. When I say halt I mean 10s-20s of records. Then it will burst again for only a couple hundred and slow down again.
I feel like I'm not understanding something. Any help would be appreciated.