search

ksprojects / zkcopy

ZooKeeper copy utililty
Apache License 2.0
237 stars 94 forks source link

zkcopy occured error: #22

Closed calipeng closed 2 years ago

calipeng commented 3 years ago

1、run cmd: java -jar target/zkcopy.jar --source xxxx:2181/clickhouse --target xxxx:2181/clickhouse

2021-09-07 21:47:33,542 [main] INFO com.github.ksprojects.ZkCopy - using 10 concurrent workers to copy data 2021-09-07 21:47:33,542 [main] INFO com.github.ksprojects.ZkCopy - delete nodes = true 2021-09-07 21:47:33,542 [main] INFO com.github.ksprojects.ZkCopy - ignore ephemeral nodes = true 2021-09-07 21:47:33,543 [main] INFO com.github.ksprojects.zkcopy.reader.Reader - Reading /clickhouse from 10.201.226.32:2181 2021-09-07 21:47:34,590 [main] INFO com.github.ksprojects.zkcopy.reader.Reader - Processing, total=12417, processed=3142 2021-09-07 21:47:35,655 [main] INFO com.github.ksprojects.zkcopy.reader.Reader - Processing, total=12417, processed=6059 2021-09-07 21:47:36,655 [main] INFO com.github.ksprojects.zkcopy.reader.Reader - Processing, total=12417, processed=9172 2021-09-07 21:47:37,655 [main] INFO com.github.ksprojects.zkcopy.reader.Reader - Processing, total=12497, processed=12497 2021-09-07 21:47:37,657 [main] INFO com.github.ksprojects.zkcopy.reader.Reader - Completed. 2021-09-07 21:47:37,687 [main] INFO com.github.ksprojects.zkcopy.writer.Writer - Writing data... 2021-09-07 21:47:38,338 [main] INFO com.github.ksprojects.zkcopy.writer.Writer - Committing transaction 2021-09-07 21:47:38,954 [main] INFO com.github.ksprojects.zkcopy.writer.Writer - Committing transaction Exception in thread "main" picocli.CommandLine$ExecutionException: Error while calling command (com.github.ksprojects.ZkCopy@7960847b): java.lang.RuntimeException: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss at picocli.CommandLine.execute(CommandLine.java:458) at picocli.CommandLine.access$300(CommandLine.java:134) at picocli.CommandLine$RunLast.handleParseResult(CommandLine.java:538) at picocli.CommandLine.parseWithHandlers(CommandLine.java:656) at picocli.CommandLine.call(CommandLine.java:883) at picocli.CommandLine.call(CommandLine.java:834) at com.github.ksprojects.ZkCopy.main(ZkCopy.java:69) Caused by: java.lang.RuntimeException: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss at com.github.ksprojects.zkcopy.writer.AutoCommitTransactionWrapper.maybeCommitTransaction(AutoCommitTransactionWrapper.java:74) at com.github.ksprojects.zkcopy.writer.AutoCommitTransactionWrapper.create(AutoCommitTransactionWrapper.java:39) at com.github.ksprojects.zkcopy.writer.Writer.upsertNode(Writer.java:147) at com.github.ksprojects.zkcopy.writer.Writer.update(Writer.java:96) at com.github.ksprojects.zkcopy.writer.Writer.update(Writer.java:104) at com.github.ksprojects.zkcopy.writer.Writer.update(Writer.java:104) at com.github.ksprojects.zkcopy.writer.Writer.update(Writer.java:104) at com.github.ksprojects.zkcopy.writer.Writer.update(Writer.java:104) at com.github.ksprojects.zkcopy.writer.Writer.update(Writer.java:104) at com.github.ksprojects.zkcopy.writer.Writer.write(Writer.java:65) at com.github.ksprojects.ZkCopy.call(ZkCopy.java:86) at com.github.ksprojects.ZkCopy.call(ZkCopy.java:14) at picocli.CommandLine.execute(CommandLine.java:456) ... 6 more Caused by: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss at org.apache.zookeeper.KeeperException.create(KeeperException.java:99) at org.apache.zookeeper.ZooKeeper.multiInternal(ZooKeeper.java:935) at org.apache.zookeeper.ZooKeeper.multi(ZooKeeper.java:915) at org.apache.zookeeper.Transaction.commit(Transaction.java:61) at com.github.ksprojects.zkcopy.writer.AutoCommitTransactionWrapper.maybeCommitTransaction(AutoCommitTransactionWrapper.java:70) ... 18 more

mossad-zika commented 2 years ago

I succeeded in k8s with this manifest

apiVersion: batch/v1
kind: Job
metadata:
  name: zk-copy
  namespace: zoo1ns
spec:
  template:
    spec:
      containers:
        - name: zkcopy
          image: ksprojects/zkcopy
          args: ["--source",  "zookeepers:2181/clickhouse", "--target", "zookeepers-clickhouse:2181/clickhouse"]
      restartPolicy: Never
  backoffLimit: 1
kshchepanovskyi commented 2 years ago

Looks like some temporary connectivity issue caused connection loss. In such cases retry should be sufficient.