vert-x3 / vertx-zookeeper

Zookeeper based cluster manager implementation
Other
72 stars 67 forks source link

zookeeper cluster error:java.io.IOException: Packet len4194475 is out of range #76

Open ifrozenice opened 5 years ago

ifrozenice commented 5 years ago

when i use zookeeper cluster,i got the error: java.io.IOException: Packet len4194475 is out of range image

why asyncMultiMap node has so many sub nodes?ZK node default max size is 4M,asyncMultiMap node was larger than 4M. image

when i create a eventbus consumer or create a weboscket,asyncMultiMap will add a sub node.... does the sub node really nesscessary?

stream-iori commented 5 years ago

Hi the nodes under __vertx.subs is the addresses of eventBus , so the more eventBus the more address in this node.

this error is make by zk, how many children node in __vertx.subs? how does you using eventBus and websocket?

You can also set system properties of zookeeper by System.setProperty("jute.maxbuffer", 4096 * 1024 * 10 + ""); 40MB to avoid such error, i prefer to decrease the number of __vertx.subs

ifrozenice commented 5 years ago

@stream1984 Thank you for reply! I used SockJS event bus bridge to do a chat room-like function. Each user will create a websocket and an eventbus consumer. If the amount of users is large, the size of the node exceeds the limit of zk. If the size of the node is set to 40MB, will the message transmission of the websocket have performance problems?

ifrozenice commented 5 years ago

The frequent establishment and shutdown of websockets leads to frequent zookeeper operations (building child nodes and deleting child nodes). If the amount of users is large, the zookeeper's IO will be very large. If the zookeeper is also the registry center of other services, it will probably affect other services. The asyncMultiMap node does not record the consumer address of websocket and eventbus. I don't know if it is feasible?

stream-iori commented 5 years ago

Hi This looked like you make one user with one eventBus, that is not good way to using sockJS with event bus bridge. You have to maintain all user with one eventBus.