hydro-project / anna

A low-latency, cloud-native KVS
Apache License 2.0
702 stars 106 forks source link

Python client in Anna Cluster mode #41

Closed authwork closed 4 years ago

authwork commented 4 years ago

@vsreekanti I enconuter a new issue. When I use the same configuration with #38 and python client client.py (from #27 ) like

from anna.client import AnnaTcpClient
from anna.lattices import SumLattice
client = AnnaTcpClient("10.1.2.66", "10.1.2.66", False)
#client = AnnaTcpClient("10.1.2.66", "10.1.2.66", True)
val = 32*[1.0]
lattice = LWWPairLattice(0, b'value')
client.put('key', lattice)

When I first run python3 client.py, it works fine. But it hooked when I run this command again. When I ctrl+c, it shows:

^CTraceback (most recent call last):
  File "client.py", line 7, in <module>
    client.put('key', lattice)
  File "/usr/local/lib/python3.6/dist-packages/anna/client.py", line 165, in put
    worker_address = self._get_worker_address(key)
  File "/usr/local/lib/python3.6/dist-packages/anna/client.py", line 233, in _get_worker_address
    addresses = self._query_routing(key, port)
  File "/usr/local/lib/python3.6/dist-packages/anna/client.py", line 266, in _query_routing
    KeyAddressResponse)[0]
  File "/usr/local/lib/python3.6/dist-packages/anna/zmq_util.py", line 27, in recv_response
    resp = rcv_sock.recv()
  File "zmq/backend/cython/socket.pyx", line 791, in zmq.backend.cython.socket.Socket.recv
  File "zmq/backend/cython/socket.pyx", line 827, in zmq.backend.cython.socket.Socket.recv
  File "zmq/backend/cython/socket.pyx", line 186, in zmq.backend.cython.socket._recv_copy
  File "zmq/backend/cython/checkrc.pxd", line 13, in zmq.backend.cython.checkrc._check_rc
KeyboardInterrupt
vsreekanti commented 4 years ago

This looks like it's hanging on querying the routing node. That could either be because there's an error on the routing node side or because the ports aren't configured correctly (i.e., the port that the client is listening on for a response from the routing node is not open).

authwork commented 4 years ago

@vsreekanti Refer to the cluster configuration in #38, I do not configure the ports, does it means the system use the default ports?

vsreekanti commented 4 years ago

Yes, it uses the default ports, but I was wondering if those ports were accessible in the network configuration on your machines?

authwork commented 4 years ago

@vsreekanti When RF<=4, the 64 cpp clients work normal. Does the python clients use different ports by default? I have discovered two further points:

  1. In the CPP clients, I give all IPs of the rounters (a.k.a. 10.1.2.66-10.1.2.69) to each client, while I only give the IP address of one rounter to the Python client (a.k.a. 10.1.2.66).
  2. The performance of RF=4 seems to be worse than RF=3 -> Maybe I need to increase the number of clients (The test system has 64 clients currently).