search

salesforce / storm-dynamic-spout

A framework for building spouts for Apache Storm and a Kafka based spout for dynamically skipping messages to be processed later.
BSD 3-Clause "New" or "Revised" License
40 stars 13 forks source link

Just compares the differences between CoordinatorFacades <--> CoordinatorRefactor branches #56

Closed Crim closed 6 years ago

Crim commented 6 years ago

see #55 don't merge this one.

Crim commented 6 years ago

So I think you may be onto something here Lemon.

Looking at what this class is doing, I don't see the reason why we should start a background thread, who's whole purpose is to start new VirtualSpouts. With triggers de-coupled from the spout, I feel like we can just have addVirtualSpout fire it up synchronously. This removes a fair bit of complexity (both in the multi-threaded code, and in our tests).

If we go that route and the background thread is no longer starting VirtualSpouts, that thread is now only occasionally polling for status and publishing metrics.