The Plenario sender should randomize its Kinesis partition key. Observations with the same key are pushed into the same shard. That is helpful for use cases that need some data to be grouped together (like if we wanted to do rolling window operations on a given sensor type). (More info here for the curious.)
But we do no such operations. If we send the same key every time, we can't scale up by adding shards. Randomizing the partition key will make sure observations are evenly distributed across however many shards we need to spin up.
The Plenario sender should randomize its Kinesis partition key. Observations with the same key are pushed into the same shard. That is helpful for use cases that need some data to be grouped together (like if we wanted to do rolling window operations on a given sensor type). (More info here for the curious.)
But we do no such operations. If we send the same key every time, we can't scale up by adding shards. Randomizing the partition key will make sure observations are evenly distributed across however many shards we need to spin up.