satabin
commented
5 years ago Hi,
I do not think this is something that belongs to this connector, it looks more related to parallel processing in streams. For instance, using the Balance connector in step 6 to dispatch to several substreams is a possibility. Or if you just want to have one single step processed in parallel, you can have a look at mapAsync and mapAsyncUnordered. If you want consistent hashing, you can use Partition operator. If you want your parallelism to be dynamic, you may have a look at the hubs. Nothing prevents that in the plugin. If you open several streams, they are uncorrelated, by design each stream is independent, so you cannot achieve this using several streams and I think the aforementioned stream operators are the best solutions to to what you want.
May be, this is more of a question rather than issue. Is there any plans to support competing consumers part of persistenceQuery plugin?
Below is a quick peek of what I'm trying to do...When I run multiple instances of the "persistence reader actor", all the actors (# 6 below) receive the same message. Ideally, I would want one of the actors to receive the message vs all. Is it possible to implement "competing consumers" so that I can avoid the same message being processed by multiple persistence reader actors?
I did go thru akka "smallest-mailbox-pool" to implement competing consumers but wanted to make sure that there is nothing inbuilt in persistenceQuery plugin before handling competing consumers with my own code.
Or please suggest if there is a better design for this..