search

vesoft-inc / nebula-python

Client API of Nebula Graph in Python
194 stars 78 forks source link

client 从外部连接 docker-compose 部署的 nebula-storaged 连接不上 #345

Closed GG2002 closed 6 months ago

GG2002 commented 6 months ago

General Question

我的 nebula 是使用 docker-compose 部署的，我刚刚尝试了使用 python client 连接 nebula。需要事先声明的是，按照 https://www.siwei.io/nebula-python-storage-docker-guide/ 中在 docker 里开一个 Jupyter Notebook 的方法我成功执行了以下代码，但是在外部访问 docker 就会出现下列的错。

实际上我也尝试过了 Rust client https://github.com/vesoft-inc/nebula-rust/issues/18 ，这些问题在两个 client 上都出现让我怀疑是我访问的方式不对.

这个过程中出现了两个错误：

  1. 首先是下面这段代码： 
    
from nebula3.mclient import MetaCache, HostAddr
from nebula3.sclient.GraphStorageClient import GraphStorageClient

the metad servers's address

meta_cache = MetaCache([('192.168.1.156', 54923), ('192.168.1.156', 54970), ('192.168.1.156', 54926)], 50000)

option 1 metad usually discover the storage address automatically

graph_storage_client = GraphStorageClient(meta_cache)

它会报下面的错：
<details>

socket error connecting to host metad1, port 9559 (None): gaierror(11001, 'getaddrinfo failed') Update meta data failed: Transport not open Traceback (most recent call last): File "C:\Program Files\Anaconda3\lib\site-packages\nebula3\mclient__init.py", line 305, in _load_all spaces = self._meta_client.list_spaces() File "C:\Program Files\Anaconda3\lib\site-packages\nebula3\mclient__init__.py", line 142, in list_spaces resp = self._connection.listSpaces(req) File "C:\Program Files\Anaconda3\lib\site-packages\nebula3\meta\MetaService.py", line 17615, in listSpaces self.send_listSpaces(req) File "C:\Program Files\Anaconda3\lib\site-packages\nebula3\meta\MetaService.py", line 17624, in send_listSpaces self._oprot.trans.flush() File "C:\Program Files\Anaconda3\lib\site-packages\nebula3\fbthrift\transport\TTransport.py", line 193, in flush self.trans.write(out) File "C:\Program Files\Anaconda3\lib\site-packages\nebula3\fbthrift\transport\TSocket.py", line 323, in write raise TTransportException(TTransportException.NOT_OPEN, nebula3.fbthrift.transport.TTransport.TTransportException: Transport not open

File C:\Program Files\Anaconda3\lib\site-packages\nebula3\sclient\GraphStorageClient.py:45, in GraphStorageClient.init(self, meta_cache, storage_addrs, time_out) 43 self._time_out = time_out 44 self._connections = [] ---> 45 self._create_connection()

File C:\Program Files\Anaconda3\lib\site-packages\nebula3\sclient\GraphStorageClient.py:84, in GraphStorageClient._create_connection(self) 82 self._storage_addrs = self._meta_cache.get_all_storage_addrs() 83 if len(self._storage_addrs) == 0: ---> 84 raise RuntimeError('Get storage address from meta cache is empty') 85 try: 86 for addr in self._storage_addrs:

RuntimeError: Get storage address from meta cache is empty

</details>

2. 而在我补全了storaged地址后，下面这个`GraphStorageClient`就不会报错了：
```python3
# option 2 manually specify the storage address
storage_addrs = [HostAddr(host='192.168.1.156', port=54935),
                 HostAddr(host='192.168.1.156', port=54929),
                 HostAddr(host='192.168.1.156', port=54930)]
graph_storage_client = GraphStorageClient(meta_cache, storage_addrs)

resp = graph_storage_client.scan_vertex(
        space_name='test_kv',
        tag_name='kv')
while resp.has_next():
    result = resp.next()
    for vertex_data in result:
        print(vertex_data)

取而代之的是和我使用rust连接时一样的错：

``` socket error connecting to host metad1, port 9559 (None): gaierror(11001, 'getaddrinfo failed') Update meta data failed: Transport not open Traceback (most recent call last): File "C:\Program Files\Anaconda3\lib\site-packages\nebula3\mclient\__init__.py", line 305, in _load_all spaces = self._meta_client.list_spaces() File "C:\Program Files\Anaconda3\lib\site-packages\nebula3\mclient\__init__.py", line 142, in list_spaces resp = self._connection.listSpaces(req) File "C:\Program Files\Anaconda3\lib\site-packages\nebula3\meta\MetaService.py", line 17615, in listSpaces self.send_listSpaces(req) File "C:\Program Files\Anaconda3\lib\site-packages\nebula3\meta\MetaService.py", line 17624, in send_listSpaces self._oprot.trans.flush() File "C:\Program Files\Anaconda3\lib\site-packages\nebula3\fbthrift\transport\TTransport.py", line 193, in flush self.__trans.write(out) File "C:\Program Files\Anaconda3\lib\site-packages\nebula3\fbthrift\transport\TSocket.py", line 323, in write raise TTransportException(TTransportException.NOT_OPEN, nebula3.fbthrift.transport.TTransport.TTransportException: Transport not open Update meta data failed: Transport not open Traceback (most recent call last): File "C:\Program Files\Anaconda3\lib\site-packages\nebula3\mclient\__init__.py", line 305, in _load_all spaces = self._meta_client.list_spaces() File "C:\Program Files\Anaconda3\lib\site-packages\nebula3\mclient\__init__.py", line 142, in list_spaces resp = self._connection.listSpaces(req) File "C:\Program Files\Anaconda3\lib\site-packages\nebula3\meta\MetaService.py", line 17615, in listSpaces self.send_listSpaces(req) File "C:\Program Files\Anaconda3\lib\site-packages\nebula3\meta\MetaService.py", line 17624, in send_listSpaces self._oprot.trans.flush() File "C:\Program Files\Anaconda3\lib\site-packages\nebula3\fbthrift\transport\TTransport.py", line 193, in flush self.__trans.write(out) File "C:\Program Files\Anaconda3\lib\site-packages\nebula3\fbthrift\transport\TSocket.py", line 323, in write raise TTransportException(TTransportException.NOT_OPEN, nebula3.fbthrift.transport.TTransport.TTransportException: Transport not open SpaceNotFoundException Traceback (most recent call last) Cell In[2], line 19 14 storage_addrs = [HostAddr(host='192.168.1.156', port=54935), 15 HostAddr(host='192.168.1.156', port=54929), 16 HostAddr(host='192.168.1.156', port=54930)] 17 graph_storage_client = GraphStorageClient(meta_cache, storage_addrs) ---> 19 resp = graph_storage_client.scan_vertex( 20 space_name='test_kv', 21 tag_name='kv') 22 while resp.has_next(): 23 result = resp.next() File C:\Program Files\Anaconda3\lib\site-packages\nebula3\sclient\GraphStorageClient.py:132, in GraphStorageClient.scan_vertex(self, space_name, tag_name, prop_names, limit, start_time, end_time, where, only_latest_version, enable_read_from_follower, partial_success) 102 def scan_vertex( 103 self, 104 space_name, (...) 113 partial_success=False, 114 ): 115 """scan vertex with the specified space_name, tag_name, 116 if the prop_names is empty, will return all properties of the tag 117 (...) 130 :return: ScanResult 131 """ --> 132 part_leaders = self._meta_cache.get_part_leaders(space_name) 133 return self._scan_vertex( 134 space_name, 135 part_leaders, (...) 144 partial_success, 145 ) File C:\Program Files\Anaconda3\lib\site-packages\nebula3\mclient\__init__.py:445, in MetaCache.get_part_leaders(self, space_name) 443 self._load_all() 444 if space_name not in self._storage_leader.keys(): --> 445 raise SpaceNotFoundException(space_name) 446 return self._storage_leader[space_name] SpaceNotFoundException: test_kv ```

下面是我的 docker 信息：

CONTAINER ID   IMAGE                           COMMAND                   CREATED      STATUS                 PORTS                                                                                                  NAMES
37990c0a3422   vesoft/nebula-console:v3.5      "sh -c 'for i in `se…"   2 days ago   Up 2 hours
                                                                                           nebula-docker-compose-console-1
c6900f46e667   vesoft/nebula-graphd:v3.6.0     "/usr/local/nebula/b…"   2 days ago   Up 2 hours (healthy)   0.0.0.0:55085->9669/tcp, 0.0.0.0:55086->19669/tcp, 0.0.0.0:55087->19670/tcp                            nebula-docker-compose-graphd1-1
020f6b27e2ec   vesoft/nebula-graphd:v3.6.0     "/usr/local/nebula/b…"   2 days ago   Up 2 hours (healthy)   0.0.0.0:9669->9669/tcp, 0.0.0.0:55083->19669/tcp, 0.0.0.0:55084->19670/tcp                             nebula-docker-compose-graphd-1
252e335fee11   vesoft/nebula-graphd:v3.6.0     "/usr/local/nebula/b…"   2 days ago   Up 2 hours (healthy)   0.0.0.0:55082->9669/tcp, 0.0.0.0:55080->19669/tcp, 0.0.0.0:55081->19670/tcp                            nebula-docker-compose-graphd2-1
39cd245ed938   vesoft/nebula-storaged:v3.6.0   "/usr/local/nebula/b…"   2 days ago   Up 2 hours (healthy)   9777-9778/tcp, 9780/tcp, 0.0.0.0:54930->9779/tcp, 0.0.0.0:54931->19779/tcp, 0.0.0.0:54932->19780/tcp   nebula-docker-compose-storaged2-1
8ce54cc547cc   vesoft/nebula-storaged:v3.6.0   "/usr/local/nebula/b…"   2 days ago   Up 2 hours (healthy)   9777-9778/tcp, 9780/tcp, 0.0.0.0:54929->9779/tcp, 0.0.0.0:54927->19779/tcp, 0.0.0.0:54928->19780/tcp   nebula-docker-compose-storaged1-1
e2b4ef403ad3   vesoft/nebula-storaged:v3.6.0   "/usr/local/nebula/b…"   2 days ago   Up 2 hours (healthy)   9777-9778/tcp, 9780/tcp, 0.0.0.0:54935->9779/tcp, 0.0.0.0:54933->19779/tcp, 0.0.0.0:54934->19780/tcp   nebula-docker-compose-storaged0-1
a9cffd7788b9   vesoft/nebula-metad:v3.6.0      "/usr/local/nebula/b…"   2 days ago   Up 2 hours (healthy)   9560/tcp, 0.0.0.0:54923->9559/tcp, 0.0.0.0:54921->19559/tcp, 0.0.0.0:54922->19560/tcp                  nebula-docker-compose-metad0-1
a588f0e51768   vesoft/nebula-metad:v3.6.0      "/usr/local/nebula/b…"   2 days ago   Up 2 hours (healthy)   9560/tcp, 0.0.0.0:54970->9559/tcp, 0.0.0.0:54968->19559/tcp, 0.0.0.0:54969->19560/tcp                  nebula-docker-compose-metad1-1
115a9d985a7b   vesoft/nebula-metad:v3.6.0      "/usr/local/nebula/b…"   2 days ago   Up 2 hours (healthy)   9560/tcp, 0.0.0.0:54926->9559/tcp, 0.0.0.0:54924->19559/tcp, 0.0.0.0:54925->19560/tcp                  nebula-docker-compose-metad2-1

我的部署方式完全按照 https://docs.nebula-graph.io/3.6.0/2.quick-start/1.quick-start-workflow/ 这个进行，我想知道的是我的部署方式或者访问方式有什么问题吗？

wey-gu commented 6 months ago

因为 storaged client 需要从 meta 获取 host list，这个host list 在docker 部署情况下，是内部的域名，外部无法访问，可以把 python 跑在同一个容器网络哈

GG2002 commented 6 months ago

因为 storaged client 需要从 meta 获取 host list，这个host list 在docker 部署情况下，是内部的域名，外部无法访问，可以把 python 跑在同一个容器网络哈

我猜测也是这样