search

uber / uReplicator

Improvement of Apache Kafka Mirrormaker
Apache License 2.0
917 stars 199 forks source link

How to monitor the progress of replication from the source cluster to the target cluster？ #317

Closed yucheng19930906 closed 4 years ago

yucheng19930906 commented 4 years ago

We deployed ureplicator in the poc environment, and can restore the source cluster information to the target cluster normally. Thank you very much for being able to use ureplicator, but we don’t know how to check the replication progress，Any tools here？

yangy0000 commented 4 years ago

you can use graphite to montior the replication progress. in order to do that, you need to specify -graphite_host and -graphite_port in the ureplicator startup command , some of the worker metrics are direct export from jmx, you can find an jmx config example in https://github.com/uber/uReplicator/blob/master/config/jmxtrans.xml

In practice, the most useful metrics to monitor replication status are the NumNoProgressPartitions from uReplicator controller and ConsumerLag from uReplicator Worker.

huangty0213 commented 4 years ago

How to use jmx to monitor the replication progress for uReplicator? Is there any jmx using example? Thanks.

yangy0000 commented 4 years ago

yes, the example is in the user guide