search

apache / rocketmq-externals

Mirror of Apache RocketMQ (Incubating)
4.58k stars 3.07k forks source link

Any wiki for replicator deployment? #804

Open zhuozl opened 3 years ago

lizhiboo commented 3 years ago

是否有 rocketmq-replicator更详细的部署文档,根据官方文档部署不生效.

  1. 创建俩套rocketmq 版本V4.5.1
  2. 源mq 创建topic如下 connector-cluster-topic connector-config-topic connector-offset-topic connector-position-topic replicatorTopic
  3. 配置文件
#current cluster node uniquely identifies
workerId=DEFAULT_WORKER_1

## Http port for user to access REST API
httpPort=8081

# Rocketmq namesrvAddr
namesrvAddr=192.168.254.131:9876

# Source or sink connector jar file dir,The default value is rocketmq-connect-sample
#pluginPaths=/usr/local/connector-plugins
pluginPaths=/home/zhuozl/rocketmq/connector-plugins

# Local file dir for config store
storePathRootDir=/home/zhuozl/rocketmq/storeRoot
  1. 启动 sh ./run_worker.sh
2021-08-31 20:33:56 INFO pool-3-thread-1 - Source task producer shutdown.
2021-08-31 20:33:56 INFO pool-3-thread-2 - Source task producer shutdown.
2021-08-31 20:33:59 ERROR RmqSourceReplicator-SourceWatcher-1 - Fetch topic list error.
org.apache.rocketmq.remoting.exception.RemotingConnectException: connect to <null> failed
  at org.apache.rocketmq.remoting.netty.NettyRemotingClient.invokeSync(NettyRemotingClient.java:392) ~[rocketmq-replicator-0.1.0-SNAPSHOT-jar-with-dependencies.jar:na]
  at org.apache.rocketmq.client.impl.MQClientAPIImpl.getTopicListFromNameServer(MQClientAPIImpl.java:1358) ~[rocketmq-replicator-0.1.0-SNAPSHOT-jar-with-dependencies.jar:na]
  at org.apache.rocketmq.tools.admin.DefaultMQAdminExtImpl.fetchAllTopicList(DefaultMQAdminExtImpl.java:246) ~[rocketmq-replicator-0.1.0-SNAPSHOT-jar-with-dependencies.jar:na]
  at org.apache.rocketmq.tools.admin.DefaultMQAdminExt.fetchAllTopicList(DefaultMQAdminExt.java:212) ~[rocketmq-replicator-0.1.0-SNAPSHOT-jar-with-dependencies.jar:na]
  at org.apache.rocketmq.replicator.RmqSourceReplicator.buildRoute(RmqSourceReplicator.java:235) ~[rocketmq-replicator-0.1.0-SNAPSHOT-jar-with-dependencies.jar:na]
  at org.apache.rocketmq.replicator.RmqSourceReplicator$1.run(RmqSourceReplicator.java:146) ~[rocketmq-replicator-0.1.0-SNAPSHOT-jar-with-dependencies.jar:na]
  at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[na:na]
  at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) ~[na:na]
  at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) ~[na:na]
  at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[na:na]
  at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[na:na]
  at java.base/java.lang.Thread.run(Thread.java:829) ~[na:na]
2021-08-31 20:33:59 INFO NettyClientSelector_1 - closeChannel: close the connection to remote address[] result: true
2021-08-31 20:33:59 ERROR RmqSourceReplicator-SourceWatcher-1 - Fetch topic list error.
org.apache.rocketmq.remoting.exception.RemotingConnectException: connect to <null> failed

2021-08-31 20:33:56 WARN pool-3-thread-2 - Rocketmq replicator task is not started.
2021-08-31 20:33:56 WARN pool-3-thread-1 - Rocketmq replicator task is not started.
2021-08-31 20:33:56 WARN pool-3-thread-2 - Rocketmq replicator task is not started.
2021-08-31 20:33:56 WARN pool-3-thread-1 - Rocketmq replicator task is not started.
2021-08-31 20:33:56 WARN pool-3-thread-2 - Rocketmq replicator task is not started.
2021-08-31 20:33:56 WARN pool-3-thread-1 - Rocketmq replicator task is not started.
2021-08-31 20:33:56 WARN pool-3-thread-2 - Rocketmq replicator task is not started.
2021-08-31 20:33:56 WARN pool-3-thread-1 - Rocketmq replicator task is not started.
2021-08-31 20:33:56 WARN pool-3-thread-2 - Rocketmq replicator task is not started.
2021-08-31 20:33:56 WARN pool-3-thread-1 - Rocketmq replicator task is not started.
2021-08-31 20:33:56 WARN pool-3-thread-2 - Rocketmq replicator task is not started.
2021-08-31 20:33:56 WARN pool-3-thread-1 - Rocketmq replicator task is not started.
2021-08-31 20:33:56 WARN pool-3-thread-2 - Rocketmq replicator task is not started.

IMO, maybe namesrv address is not correct. Make sure rocketmq cluster works.

zhuozl commented 3 years ago

@lizhiboo connect.conf 的namesrvAddr 要配置为目的的namesrvAddr可以了.不过会丢失 tag,key

odbozhou commented 2 years ago

rocketmq-connect already has an independent repository, you can go to this repository to submit PR, issue. https://github.com/apache/rocketmq-connect