search

EventStore / replicator

Real-time replication tool
https://replicator.eventstore.org
Apache License 2.0
20 stars 13 forks source link

Partitioning versus persistent subscription #59

Closed poostwoud closed 2 years ago

poostwoud commented 2 years ago

Hi,

In the docs on sink partitioning it is written that "If you don’t care much about events order in $all, you can configure Replicator to use concurrent writers, which will increase performance."

In the comments of issue 41 it is written that "The reason for replicator not to use persistent subscriptions is that: 1) No guarantee for events on $all to be processed in order, which is often the requirement; 2) Additional IO on the source database leader as it produces checkpoint events."

Does this mean that, if the order of $all events is not important, it makes to sense to consider replicator using partitioning similar to using multiple consumers of a persistent subscription?

Also, would the same count for using Kafka as this could probably cause the order of $all to differ from the source as well?

Curious on your thoughts.

Cheers,

Peter

alexeyzimarev commented 2 years ago

I'd prefer if you use Discussions for questions, not issues. Thanks.

alexeyzimarev commented 2 years ago

Does this mean that, if the order of $all events is not important, it makes to sense to consider replicator using partitioning similar to using multiple consumers of a persistent subscription?

Yes

Also, would the same count for using Kafka

Using Kafka for what exactly, I don't understand the question.