search

ksprojects / zkcopy

ZooKeeper copy utililty
Apache License 2.0
237 stars 94 forks source link

Exception copying data: java.io.IOException: Packet len5361140 is out of range! #12

Closed the-smooth-operator closed 7 years ago

the-smooth-operator commented 7 years ago

I'm trying to copy data between 2 zookeeper clusters, and getting the next errors and exceptions mentioned below. The process exits in few seconds and doesn't copy the data. I'm running zkcopy using the docker image. Do you have any idea where can be the problem?

2017-05-23 10:07:24,130 [main] INFO  com.github.ksprojects.zkcopy.reader.Reader - Processing, total=93351, processed=443 
2017-05-23 10:07:25,131 [main] INFO  com.github.ksprojects.zkcopy.reader.Reader - Processing, total=93351, processed=443 
2017-05-23 10:07:25,547 [main-SendThread(xxxxx.xxxxx:2181)] WARN  org.apache.zookeeper.ClientCnxn - Session 0x15bcd5d59f159e7 for server xxxxx.xxxxx/yyy.yyy.yyy.yyy:2181, unexpected error, closing socket connection and attempting reconnect
 java.io.IOException: Packet len179414030 is out of range!     
     at org.apache.zookeeper.ClientCnxnSocket.readLength(ClientCnxnSocket.java:112)
     at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:79)     
     at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:366)
     at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081) 
2017-05-23 10:07:25,654 [Thread-0] ERROR com.github.ksprojects.zkcopy.reader.NodeReader - Could not read from remote server org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /data/path     at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
     at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
     at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
     at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1500
     at com.github.ksprojects.zkcopy.reader.NodeReader.run(NodeReader.java:49)
     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
     at java.lang.Thread.run(Thread.java:745) 
2017-05-23 10:07:26,131 [main] INFO  com.github.ksprojects.zkcopy.reader.Reader - Processing, total=93375, processed=93366 
2017-05-23 10:07:27,131 [main] INFO  com.github.ksprojects.zkcopy.reader.Reader - Processing, total=93375, processed=93366 
2017-05-23 10:07:27,625 [Thread-1-SendThread(xxxxx.xxxxx:2181)] WARN  org.apache.zookeeper.ClientCnxn - Session 0x15bcd5d59f159ea for server xxxxx.xxxxx/yyy.yyy.yyy.yyy:2181, unexpected error, closing socket connection and attempting reconnect 
java.io.IOException: Packet len5361140 is out of range!
     at org.apache.zookeeper.ClientCnxnSocket.readLength(ClientCnxnSocket.java:112)
     at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:79)
     at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:366)
     at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081) 
2017-05-23 10:07:27,725 [Thread-3] ERROR com.github.ksprojects.zkcopy.reader.NodeReader - Could not read from remote server org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /another/data/path
     at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
     at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)     at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
     at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1500)
     at com.github.ksprojects.zkcopy.reader.NodeReader.run(NodeReader.java:49)
     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
     at java.lang.Thread.run(Thread.java:745)
2017-05-23 10:07:28,131 [main] INFO  com.github.ksprojects.zkcopy.reader.Reader - Processing, total=93392, processed=93392 
2017-05-23 10:07:28,132 [main] INFO  com.github.ksprojects.zkcopy.reader.Reader - Completed.
2017-05-23 10:07:28,132 [main] ERROR com.github.ksprojects.ZkCopy - FAILED
kshchepanovskyi commented 7 years ago

I think it is related to https://stackoverflow.com/questions/10249579/zookeeper-cli-failing-ioexception-packet-len12343123123-is-out-of-range

Try to add -Djute.maxbuffer=536870912 to parameters.

the-smooth-operator commented 7 years ago

That fixed the problem. Thanks a lot!