search

salesforce / storm-dynamic-spout

A framework for building spouts for Apache Storm and a Kafka based spout for dynamically skipping messages to be processed later.
BSD 3-Clause "New" or "Revised" License
40 stars 13 forks source link

SidelineSpoutHandler should periodically check its state and recover #47

Closed stanlemon closed 6 years ago

stanlemon commented 6 years ago

VirtualSpouts can die. They shouldn't, but they do. FilterChains can get out of whack. They shouldn't, but they do. The SidelineSpoutHandler needs to periodically check its state and reload things to get them in the correct state. This should be as simple as calling loadSidelines() periodically, but we need to add test coverage that jacking with the state of the spouts running and the various filter chains resumes correctly with subsequent calls. Then when all is said and done we can call loadSildelines() on an interval and we should be good to go.

stanlemon commented 6 years ago

@Crim Are we good to go on this one now?

Crim commented 6 years ago

Added some comments here. I think there are some concerns around multi-threading in the loadSidelines() method that we need to be sure of before merging this in.

stanlemon commented 6 years ago

@Crim I think I've responded to everything that was actionable, let me know if I have missed anything.