search

uber / uReplicator

Improvement of Apache Kafka Mirrormaker
Apache License 2.0
917 stars 199 forks source link

Federated uReplicator? #125

Closed balboah closed 6 years ago

balboah commented 6 years ago

While looking for metrics to use for monitoring lagging replication, I noticed the offset monitor which seem to only exist in Federated uReplicator branch. Is this branch something usable and why are these features not merged back into master?

Appreciate pointers to which metrics that are most useful for monitoring replication status

@xhl1988

xhl1988 commented 6 years ago

We are still testing Federated uReplicator in our staging environment. After we migrate to Federated uReplicator, we will merge it back to master.

In our experience, the most important ones are MaxLag on worker side and NumNoProgressPartitions on controller side. Other metrics like BytesPerSec from Producer/Consumer itself are also very useful.

balboah commented 6 years ago

thanks for the update, appreciated

dungnt081191 commented 5 years ago

Hi @xhl1988 , at this time , Federated uReplicator mode is working or not ? Do you have any document for this mode ?

xhl1988 commented 5 years ago

@dungnt081191 federation mode is working and we are using it in production. You can find the user guide here and meetup talk here.

dungnt081191 commented 5 years ago

Hi @xhl1988 , do you have clear introduction about federation mode ? on README.md just have a few information about it ?