search

salesforce / storm-dynamic-spout

A framework for building spouts for Apache Storm and a Kafka based spout for dynamically skipping messages to be processed later.
BSD 3-Clause "New" or "Revised" License
40 stars 13 forks source link

Switch to using ConsumerPartition for sideline persistence #72

Closed stanlemon closed 6 years ago

stanlemon commented 6 years ago

The goal here is to not have to call out to the kafka config for the topic by relying on ConsumerPartition directly. Long term we could actually use ConsumerState wholesale here and let the specific persistence adapter loop over the partitions - if they need to.

Addresses comment #46