[SIDELINE] Trigger reference implementation

salesforce / storm-dynamic-spout

A framework for building spouts for Apache Storm and a Kafka based spout for dynamically skipping messages to be processed later.

BSD 3-Clause "New" or "Revised" License

40 stars 13 forks source link

[SIDELINE] Trigger reference implementation #46

Closed stanlemon closed 6 years ago

stanlemon commented 6 years ago

Create a reference implementation of SidelineTrigger using Zookeeper watches (it's our preferred state machine at the moment).

stanlemon commented 6 years ago

@Crim as for a namespace for this, how does com.salesforce.storm.spout.sideline.trigger.example seem? Or should we do something else? Example makes sense, but I'm also afraid it might discourage people from using it out of the box, which is what I'm hoping they'll do.

Crim commented 6 years ago

@stanlemon To remove the dependency on kafka consumer from Sideline, I think we should do one of the following in SidelineSpoutHandler::loadSidelines():

Remove final String topic = (String) getSpoutConfig().get(KafkaConsumerConfig.KAFKA_TOPIC); and replace with its own configuration value.

Store the topic/namespace on the sideline request, and load it from there.

thoughts?

stanlemon commented 6 years ago

@Crim WIP over on #72 for that item. Because the reference implementation is done I am going to go ahead and close this issue now.