Open AbelRubick opened 2 years ago
@Aiee do you have more ideas on this please, it seems version verification failed even when 3.0 console talking to 3.0 nebula graphD.
@AbelRubick could you help double-check its server-side version?
bin]# ./nebula-graphd --version
nebula-graphd version 3.1.0, Git: 33fd35e, Build Time: Apr 23 2022 22:03:50
(root@nebula) [(none)]> show hosts graph
+-----------+------+----------+---------+--------------+---------+
| Host | Port | Status | Role | Git Info Sha | Version |
+-----------+------+----------+---------+--------------+---------+
| "graphd" | 9669 | "ONLINE" | "GRAPH" | "33fd35e" | "3.1.0" |
| "graphd1" | 9669 | "ONLINE" | "GRAPH" | "33fd35e" | "3.1.0" |
| "graphd2" | 9669 | "ONLINE" | "GRAPH" | "33fd35e" | "3.1.0" |
+-----------+------+----------+---------+--------------+---------+
@wey-gu This seems to be the typical error due to the version mismatch between the client-side and server-side.
I've tested it and the version is 3.0.0
@Aiee do you have more ideas on this please, it seems version verification failed even when 3.0 console talking to 3.0 nebula graphD.
@AbelRubick could you help double-check its server-side version?
- from its binary like nebula-graphd :
bin]# ./nebula-graphd --version nebula-graphd version 3.1.0, Git: 33fd35e, Build Time: Apr 23 2022 22:03:50
- show graph from its studio console if you have it
(root@nebula) [(none)]> show hosts graph +-----------+------+----------+---------+--------------+---------+ | Host | Port | Status | Role | Git Info Sha | Version | +-----------+------+----------+---------+--------------+---------+ | "graphd" | 9669 | "ONLINE" | "GRAPH" | "33fd35e" | "3.1.0" | | "graphd1" | 9669 | "ONLINE" | "GRAPH" | "33fd35e" | "3.1.0" | | "graphd2" | 9669 | "ONLINE" | "GRAPH" | "33fd35e" | "3.1.0" | +-----------+------+----------+---------+--------------+---------+
I've tested it and all of the version is 3.0.0
I've tested it and all of the version is 3.0.0
@Aiee where could go wrong then, please? Anything that @AbelRubick could provide to look into this?
Is it possible to be a network issue just with the wrong error being raised by the console?
Sorry for the late reply. The information you provided has not been reproduced locally. Do you have any new information you can provide us to reproduce? Thanks a lot for your contribution.
We have noticed that the issue you created hasn’t been updated for nearly a month, so we have to close it for now. If you have any new updates, you are welcome to reopen this issue anytime. Thanks a lot for your contribution.
How to solve this problem?
How to solve this problem?
Have you encountered this, too? this error refers to version-hand-shake failure, which normally could be caused by two reasons:
could you double-check against the above cases?
root@1f69b77c5572:/usr/local/nebula/bin# ./nebula-graphd --version nebula-graphd version 3.4.0, Git: db3c1b3, Build Time: Feb 9 2023 12:09:22 This source code is licensed under Apache 2.0 License.
nebula-console-linux-amd64-v3.4.0 My nebula db and console are both 3.4.0.
@fxb392 could you help double confirm:
nebula-console-3.4 -addr 192.168.8.128 -port 9669 -user root -p nebula
and errorss -plunt | grep 9669
[INFO] nebula-metad(db3c1b3): Exited [INFO] nebula-graphd(db3c1b3): Running as 565, Listening on 9669 [WARN] nebula-storaged after v3.0.0 will not start service until it is added to cluster. [WARN] See Manage Storage hosts:ADD HOSTS in https://docs.nebula-graph.io/ [INFO] nebula-storaged(db3c1b3): Running as 611, Listening on 9779
2023/05/08 06:16:38 Fail to initialize the connection pool, host: 192.168.8.128, port: 9669, failed to open connection, error: failed to open transport, error: dial tcp 192.168.8.128:9669: i/o timeout panic: Fail to initialize the connection pool, host: 192.168.8.128, port: 9669, failed to open connection, error: failed to open transport, error: dial tcp 192.168.8.128:9669: i/o timeout
goroutine 1 [running]: log.Panicf(0xc0000aa0c0, 0xb5, 0x0, 0x0, 0x0) /opt/hostedtoolcache/go/1.16.4/x64/src/log/log.go:361 +0xc5 main.main() /home/runner/work/nebula-console/nebula-console/main.go:532 +0xb10
root@1f69b77c5572:/nebula# ss -plunt | grep 9669 tcp LISTEN 0 128 0.0.0.0:19669 0.0.0.0:* users:(("nebula-graphd",pid=1012,fd=28))
Is there a problem with my installation?
It's related the metad not being up and running, could you check metad's log?
[INFO] nebula-metad(db3c1b3): Exited
Three error logs: 1、nebula-metad.ERROR Log file created at: 2023/05/08 14:23:52 Running on machine: 7fdf26acdc79 Running duration (h:mm:ss): 0:00:00 Log line format: [IWEF]yyyymmdd hh:mm:ss.uuuuuu threadid file:line] msg F20230508 14:23:52.975773 55 DiskManager.cpp:39] DataPath invalid: boost::filesystem::status: Function not implemented: "/usr/local/nebula/data/meta"
2、nebula-graphd.ERROR Log file created at: 2023/05/08 14:23:55 Running on machine: 7fdf26acdc79 Running duration (h:mm:ss): 0:00:00 Log line format: [IWEF]yyyymmdd hh:mm:ss.uuuuuu threadid file:line] msg E20230508 14:23:55.987087 203 MetaClient.cpp:772] Send request to "127.0.0.1":9559, exceed retry limit E20230508 14:23:55.987252 203 MetaClient.cpp:773] RpcResponse exception: apache::thrift::transport::TTransportException: Dropping unsent request. Connection closed after: apache::thrift::transport::TTransportException: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused E20230508 14:23:55.987308 122 MetaClient.cpp:112] Heartbeat failed, status:RPC failure in MetaClient: apache::thrift::transport::TTransportException: Dropping unsent request. Connection closed after: apache::thrift::transport::TTransportException: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connect E20230508 14:24:08.997056 216 MetaClient.cpp:772] Send request to "127.0.0.1":9559, exceed retry limit E20230508 14:24:08.997126 216 MetaClient.cpp:773] RpcResponse exception: apache::thrift::transport::TTransportException: Dropping unsent request. Connection closed after: apache::thrift::transport::TTransportException: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused E20230508 14:24:08.998440 122 MetaClient.cpp:112] Heartbeat failed, status:RPC failure in MetaClient: apache::thrift::transport::TTransportException: Dropping unsent request. Connection closed after: apache::thrift::transport::TTransportException: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connect E20230508 14:24:22.001811 218 MetaClient.cpp:772] Send request to "127.0.0.1":9559, exceed retry limit E20230508 14:24:22.001852 218 MetaClient.cpp:773] RpcResponse exception: apache::thrift::transport::TTransportException: Dropping unsent request. Connection closed after: apache::thrift::transport::TTransportException: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused E20230508 14:24:22.001892 122 MetaClient.cpp:112] Heartbeat failed, status:RPC failure in MetaClient: apache::thrift::transport::TTransportException: Dropping unsent request. Connection closed after: apache::thrift::transport::TTransportException: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connect E20230508 14:24:35.006040 220 MetaClient.cpp:772] Send request to "127.0.0.1":9559, exceed retry limit E20230508 14:24:35.006086 220 MetaClient.cpp:773] RpcResponse exception: apache::thrift::transport::TTransportException: Dropping unsent request. Connection closed after: apache::thrift::transport::TTransportException: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused E20230508 14:24:35.006141 122 MetaClient.cpp:112] Heartbeat failed, status:RPC failure in MetaClient: apache::thrift::transport::TTransportException: Dropping unsent request. Connection closed after: apache::thrift::transport::TTransportException: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connect E20230508 14:24:38.009608 283 MetaClient.cpp:772] Send request to "127.0.0.1":9559, exceed retry limit E20230508 14:24:38.009657 283 MetaClient.cpp:773] RpcResponse exception: apache::thrift::transport::TTransportException: Dropping unsent request. Connection closed after: apache::thrift::transport::TTransportException: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused E20230508 14:24:38.009712 122 MetaClient.cpp:150] RPC failure in MetaClient: apache::thrift::transport::TTransportException: Dropping unsent request. Connection closed after: apache::thrift::transport::TTransportException: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connect E20230508 14:24:38.009747 122 GraphService.cpp:49] Failed to wait for meta service ready synchronously. E20230508 14:24:38.009774 122 GraphServer.cpp:39] Failed to wait for meta service ready synchronously. E20230508 14:24:38.011329 122 GraphDaemon.cpp:156] The graph server start failed
3、nebula-storaged.ERROR Log file created at: 2023/05/08 14:23:55 Running on machine: 7fdf26acdc79 Running duration (h:mm:ss): 0:00:00 Log line format: [IWEF]yyyymmdd hh:mm:ss.uuuuuu threadid file:line] msg E20230508 14:23:55.998018 210 MetaClient.cpp:772] Send request to "127.0.0.1":9559, exceed retry limit E20230508 14:23:55.998123 210 MetaClient.cpp:773] RpcResponse exception: apache::thrift::transport::TTransportException: Dropping unsent request. Connection closed after: apache::thrift::transport::TTransportException: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connection refused): Connection refused E20230508 14:23:55.998186 165 MetaClient.cpp:112] Heartbeat failed, status:RPC failure in MetaClient: apache::thrift::transport::TTransportException: Dropping unsent request. Connection closed after: apache::thrift::transport::TTransportException: AsyncSocketException: connect failed, type = Socket not open, errno = 111 (Connect E20230508 14:24:09.003250 217 MetaClient.cpp:772] Send request to "127.0.0.1":9559, exceed retry limit ......
what should i do next?
boost::filesystem::status: Function not implemented
@fxb392 it's a known issue https://github.com/vesoft-inc/nebula/issues/4205 may I know what OS are you using?
uname -a
For now, I suggest using one with a newer version of the linux kernel.
@wey-gu uname -a Linux 7fdf26acdc79 3.10.0-1160.36.2.el7.x86_64 #1 SMP Wed Jul 21 11:57:15 UTC 2021 x86_64 GNU/Linux I run this in docker.
3.10.0
The root cause is the upstream boost::filesystem::status()
doesn't have implementation under 3.10.0 kernal, which was released 10 years ago T__T.
Thank you very much for your help. Can this problem be avoided if Nebula db 3.3.0 is used?
No, it's been there for more than one year as I observed, sorry about this, could you spawn some VM with a higher version of kernel?
Ok.If i want upgrade the kernal version,which kernal version is suitable.
Ok.If i want upgrade the kernal version,which kernal version is suitable.
According to similar issues
4.11 onward would be good enough for x86_64, but I haven't tried it yet
It's strange enough that someone with a 4.15 kernel is with this issue, but I could see one of my machines with an old kernel worked fine Linux nebula1 3.10.0-1160.76.1.el7.x86_64
It's highly recommended to try spawning a new VM, which should be working fine.
I used version 3.0.2 and successfully installed it. root@83311f6a550a:/opt/nebula# /usr/local/nebula/scripts/nebula.service status all [INFO] nebula-metad(4805f8c): Running as 144, Listening on 9559 [INFO] nebula-graphd(4805f8c): Running as 212, Listening on 9669 [INFO] nebula-storaged(4805f8c): Running as 251, Listening on 9779
But i encountered another problem... root@83311f6a550a:/opt/nebula# dpkg -i nebula-graph-studio-3.2.5.x86_64.deb (Reading database ... 5855 files and directories currently installed.) Preparing to unpack nebula-graph-studio-3.2.5.x86_64.deb ... Unpacking nebula-graph-studio (3.2.5) over (3.2.5) ... Setting up nebula-graph-studio (3.2.5) ... ERROR:systemctl: dbus.service: Executable path is not absolute, ignoring: @/usr/bin/dbus-daemon @dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation --syslog-only
ok, that's a quite outdated version though :)
ERROR:systemctl: dbus.service: Executable path is not absolute, ignoring: @/usr/bin/dbus-daemon @dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation --syslog-only @wey-gu Does this error report have any impact?I can open the studio now.
ERROR:systemctl: dbus.service: Executable path is not absolute, ignoring: @/usr/bin/dbus-daemon @dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation --syslog-only @wey-gu Does this error report have any impact?I can open the studio now.
This is a problem with systemd's dependency on dbus. Perhaps you don't have dbus installed. You can manually start the studio process without using systemd.
-- 这是 systemd 对 dbus 的依赖的问题,是不是你没有 dbus,你可以手动启动 studio 进程,不用 systemd。
这是 systemd 对 dbus 的依赖的问题,是不是你没有 dbus,你可以手动启动 studio 进程,不用 systemd。
好的,不是nebula studio的问题就行,我现在能打开nebula studio的ui界面。非常感谢您这两天的帮助!!!
My version of nebula is:3.0.0 My version of console is:3.0.0 and 2.6.0(Have tested for the same error to occur) error: 2022/06/03 22:11:29 Fail to initialize the connection pool, host: 127.0.0.1, port: 9669, failed to open connection, error: failed to verify client version: verifyClientVersion failed: wrong method name panic: Fail to initialize the connection pool, host: 127.0.0.1, port: 9669, failed to open connection, error: failed to verify client version: verifyClientVersion failed: wrong method name
goroutine 1 [running]: log.Panicf(0xc0003a6000, 0xb6, 0x0, 0x0, 0x0) /opt/hostedtoolcache/go/1.16.4/x64/src/log/log.go:361 +0xc5 main.main() /home/runner/work/nebula-console/nebula-console/main.go:535 +0xb10