search

JasperFx / wolverine

Supercharged .NET server side development!
https://wolverinefx.net
MIT License
1.23k stars 131 forks source link

"Sticky" Listeners that only run on one Node #784

Closed jeremydmiller closed 6 months ago

jeremydmiller commented 6 months ago

What if Wolverine had the ability to make the message listener for an endpoint (a Rabbit queue, an Azure Service Bus queue, a Kafka topic) only active on a single node within a Wolverine cluster? This potentially gives Wolverine the ability for:

  1. Strictly ordered message handling at a single endpoint. You can't get that with competing consumers.
  2. A form of virtual actors for Wolverine. Not sure how aggressively we'd build on top of that, but it's possible
lcorneliussen commented 6 months ago

One active node per queue could be a start, but wouldn’t wolverine need to route to a certain node based on the actor id? (either via a partitioning scheme, or via a persisted registry of which key is running where?)

jeremydmiller commented 6 months ago

@lcorneliussen Cruder than that. If wolverine is publishing to an endpoint where only the one sticky listener is handling it, you're de facto a poor man's actor.

jeremydmiller commented 6 months ago

Oh, and Wolverine does have the ability to let you define your own distribution of "sticky" agents across a cluster of nodes. Not documented right now, but used under the covers for some of the inbox/outbox stuff and the "Critter Stack Pro" commercial tooling that distributes asynchronous projections over a cluster

jeremydmiller commented 6 months ago

Tasks