search

twitchscience / kinsumer

Native Go consumer for AWS Kinesis streams.
Other
134 stars 35 forks source link

Skip shard ID cache on first run #18

Closed dcelasun closed 5 years ago

dcelasun commented 5 years ago

On startup, if the metadata table is being created for the first time, loadShardIDsFromDynamo() call in refreshShards() will always return an empty shard list.

This causes the consumer to wait until the next leader election to start receiving events, because that's the only time loadShardIDsFromKinesis() is called. By default, this happens once a minute, which means the consumer basically sleeps for 1 minute on first run.

This commit fixes that by adding a first run flag and skipping the cache if it's true.

Fixes #17.

dcelasun commented 5 years ago

Travis failure is an unrelated Go 1.6 context issue.

dcelasun commented 5 years ago

Closing in favour of #22, which has the correct branch on my fork.