A zookeeper command is limited to 8KB.

[addyess]

When issuing a command, the output on the socket is truncated at 8KB.

zk = KazooClient()
buf = zk.command("stat")

Perhaps you could read from the socket until the socket is drained rather than truncating to only 8KB of data from the socket? https://github.com/python-zk/kazoo/blob/4d268adf9837836f05dde5ec81be0d7bbd759e78/kazoo/client.py#L653

[jeffwidman]

@hannosch do you recall why you arbitrarily set it at 8KB rather than draining the socket?

[hannosch]

IIRC 8 KB doesn't have a special meaning here. It's just a "should-be-big-enough" default value.

After reading the four-letter command list, I probably assumed their output would never be particular large.

For normal Zookeeper actions we have a different limit somewhere. And IIRC those are also limited by the jute.maxBuffer server setting.

I'm fine with either using what we use elsewhere or just draining the socket.

[brouberol]

I stumbled upon the same issue today, as I was trying to parse the cons output, and was missing some data.

To bypass the 8192 char limit, I overrode my client command method with

def command(self, cmd='ruok'):
    """Sends a commmand to the ZK node.

    Overrides methode defined at
    https://github.com/python-zk/kazoo/blob/release/2.4/kazoo/client.py#L637
    as it could leave some data unread from the socket.

    """
    if not self._live.is_set():
        raise ConnectionLoss("No connection to server")
    out = []
    peer = self._connection._socket.getpeername()
    sock = self.handler.create_connection(
        peer, timeout=self._session_timeout / 1000.0)
    sock.sendall(cmd)
    while True:
        data = sock.recv(8192)
        if not data:
            break
        out.append(data)

    sock.close()
    result = b''.join(out)
    return result.decode('utf-8', 'replace')

[brouberol]

For reference, I have a fairly used ZK server, and the cons output is ~45k char.

$  echo 'cons' | nc localhost 2181 | wc -c
44378

[zerogtw]

I have encountered a problem like @brouberol mentioned, I hope this can be resolved as soon as possible.

$ echo "mntr" |nc 127.0.0.1 2181 |wc -c
16801

[zbentley]

Any chance of a fix here, or would you accept PRs for this? Seems pretty bad.

[StephenSorriaux]

Hi,

Yes we would definitely accept PRs for this. I guess the addition of a new kwarg to https://github.com/python-zk/kazoo/blob/master/kazoo/client.py#L705 in order to make the size adjustable, with a default value to the current one, seems safe. But any better ideas are welcomed.