eventuate-foundation / eventuate-messaging-kafka

Other
13 stars 11 forks source link

SwimlaneDispatcher configurable - hashing algorithm, threadpool size #57

Open cer opened 2 years ago

cer commented 2 years ago

Currently,

Proposal:

cer commented 2 years ago

To configure the SwimlaneBasedDispatcher, configure an @Bean that implements TopicPartitionToSwimLaneMapping.

For example, see SwimlanePerTopicPartition

cer commented 2 years ago

One more thing to consider: pass the message key to toSwimLane(TopicPartition topicPartition) to enable parallelism within a TopicPartition.

cer commented 2 years ago

I believe that MultipleSwimlanesPerTopicPartitionMapping is unnecessarily complicated. A simpler implementation could simply hash(topic and messageKey) % desiredNumberOfSwimlanes. This will preserve ordering of all messages published to a topic (e.g. Aggregate type) with the same message key (e.g. aggregate ID)