search

spectre-project / rusty-spectre

Spectre full-node, wallet and related libraries in the Rust programming language.
ISC License
15 stars 7 forks source link

spectred exits with tonic::transport::Error(Transport, hyper::Error(Listen, Os { code: 98, kind: AddrInUse, message: "Address already in use" })) #3

Closed whileunless closed 4 months ago

whileunless commented 4 months ago

systemctl status rusty-spectre :

● rusty-spectre.service - rusty-spectred
     Loaded: loaded (/etc/systemd/system/rusty-spectre.service; enabled; vendor>
     Active: activating (auto-restart) (Result: exit-code) since Wed 2024-06-19>
    Process: 100318 ExecStart=/home/j/.cargo/bin/spectred --utxoindex (code=exi>
   Main PID: 100318 (code=exited, status=1/FAILURE)
        CPU: 76ms

journalctl -u rusty-spectre:

Jun 19 10:02:52 slim systemd[1]: Started rusty-spectred.
Jun 19 10:02:52 slim spectred[109282]: 2024-06-19 10:02:52.403+02:00 [INFO ] spectred v0.3.14
Jun 19 10:02:52 slim spectred[109282]: 2024-06-19 10:02:52.403+02:00 [INFO ] Application directory: /home/j/.rusty-spectre
Jun 19 10:02:52 slim spectred[109282]: 2024-06-19 10:02:52.403+02:00 [INFO ] Data directory: /home/j/.rusty-spectre/spectre-mainnet/datadir
Jun 19 10:02:52 slim spectred[109282]: 2024-06-19 10:02:52.403+02:00 [INFO ] Logs directory: /home/j/.rusty-spectre/spectre-mainnet/logs
Jun 19 10:02:52 slim spectred[109282]: 2024-06-19 10:02:52.403+02:00 [INFO ] Utxoindex Data directory /home/j/.rusty-spectre/spectre-mainnet/datadir/utxoindex
Jun 19 10:02:52 slim spectred[109282]: 2024-06-19 10:02:52.469+02:00 [INFO ] [UPnP] Attempting to register upnp... (to disable run the node with --disable-upnp)
Jun 19 10:03:02 slim spectred[109282]: 2024-06-19 10:03:02.554+02:00 [WARN ] [UPnP] Error adding port mapping: IO error: Resource temporarily unavailable (os error 11)
Jun 19 10:03:02 slim spectred[109282]: 2024-06-19 10:03:02.557+02:00 [INFO ] GRPC Server starting on: 127.0.0.1:18110
Jun 19 10:03:02 slim spectred[109282]: 2024-06-19 10:03:02.557+02:00 [INFO ] P2P Server starting on: 0.0.0.0:18111
Jun 19 10:03:02 slim spectred[109282]: 2024-06-19 10:03:02.558+02:00 [ERROR] thread 'tokio-runtime-worker' panicked at rpc/grpc/server/src/connection_handler.rs:160:29: GRPC Server 127.0.0.1:18110 stopped with error: tonic::transport::Error(Transport, hyper::Error(Listen, Os { code: 98, kind: AddrInUse, message: "Address already in use" }))
Jun 19 10:03:02 slim spectred[109282]: thread 'tokio-runtime-worker' panicked at rpc/grpc/server/src/connection_handler.rs:160:29:
Jun 19 10:03:02 slim spectred[109282]: GRPC Server 127.0.0.1:18110 stopped with error: tonic::transport::Error(Transport, hyper::Error(Listen, Os { code: 98, kind: AddrInUse, message: "Address already in use" }))
Jun 19 10:03:02 slim spectred[109282]: note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Jun 19 10:03:02 slim spectred[109282]: Exiting...
Jun 19 10:03:02 slim spectred[109282]: 2024-06-19 10:03:02.558+02:00 [ERROR] thread 'tokio-runtime-worker' panicked at protocol/p2p/./src/core/connection_handler.rs:91:29: P2P, Server 0.0.0.0:18111 stopped with error: tonic::transport::Error(Transport, hyper::Error(Listen, Os { code: 98, kind: AddrInUse, message: "Address already in use" }))
Jun 19 10:03:02 slim spectred[109282]: thread 'tokio-runtime-worker' panicked at protocol/p2p/./src/core/connection_handler.rs:91:2024-06-19 10:03:02.558+02:0029 [:
Jun 19 10:03:02 slim spectred[109282]: INFOP2P, Server 0.0.0.0:18111 stopped with error: tonic::transport::Error(Transport, hyper::Error(Listen, Os { code: 98, kind: AddrInUse, message: "Address already in use" }))
Jun 19 10:03:02 slim spectred[109282]: ] Querying DNS seeder mainnet-dnsseed-2.spectre-network.org
Jun 19 10:03:02 slim spectred[109282]: Exiting...
Jun 19 10:03:02 slim systemd[1]: rusty-spectre.service: Main process exited, code=exited, status=1/FAILURE
Jun 19 10:03:02 slim systemd[1]: rusty-spectre.service: Failed with result 'exit-code'.
Jun 19 10:04:42 slim systemd[1]: rusty-spectre.service: Scheduled restart job, restart counter is at 25.
Jun 19 10:04:42 slim systemd[1]: Stopped rusty-spectred.
Jun 19 10:04:42 slim systemd[1]: Started rusty-spectred.
Jun 19 10:04:42 slim spectred[110632]: 2024-06-19 10:04:42.637+02:00 [INFO ] spectred v0.3.14
Jun 19 10:04:42 slim spectred[110632]: 2024-06-19 10:04:42.637+02:00 [INFO ] Application directory: /home/j/.rusty-spectre
Jun 19 10:04:42 slim spectred[110632]: 2024-06-19 10:04:42.637+02:00 [INFO ] Data directory: /home/j/.rusty-spectre/spectre-mainnet/datadir
Jun 19 10:04:42 slim spectred[110632]: 2024-06-19 10:04:42.637+02:00 [INFO ] Logs directory: /home/j/.rusty-spectre/spectre-mainnet/logs
Jun 19 10:04:42 slim spectred[110632]: 2024-06-19 10:04:42.637+02:00 [INFO ] Utxoindex Data directory /home/j/.rusty-spectre/spectre-mainnet/datadir/utxoindex
Jun 19 10:04:42 slim spectred[110632]: 2024-06-19 10:04:42.701+02:00 [INFO ] [UPnP] Attempting to register upnp... (to disable run the node with --disable-upnp)
Jun 19 10:04:52 slim spectred[110632]: 2024-06-19 10:04:52.890+02:00 [WARN ] [UPnP] Error adding port mapping: IO error: Resource temporarily unavailable (os error 11)
Jun 19 10:04:52 slim spectred[110632]: 2024-06-19 10:04:52.892+02:00 [INFO ] GRPC Server starting on: 127.0.0.1:18110
Jun 19 10:04:52 slim spectred[110632]: 2024-06-19 10:04:52.892+02:00 [INFO ] P2P Server starting on: 0.0.0.0:18111
Jun 19 10:04:52 slim spectred[110632]: 2024-06-19 10:04:52.892+02:00 [ERROR] thread 'tokio-runtime-worker' panicked at rpc/grpc/server/src/connection_handler.rs:160:29: GRPC Server 127.0.0.1:18110 stopped with error: tonic::transport::Error(Transport, hyper::Error(Listen, Os { code: 98, kind: AddrInUse, message: "Address already in use" }))
Jun 19 10:04:52 slim spectred[110632]: thread 'tokio-runtime-worker' panicked at rpc/grpc/server/src/connection_handler.rs:160:29:
Jun 19 10:04:52 slim spectred[110632]: GRPC Server 127.0.0.1:18110 stopped with error: tonic::transport::Error(Transport, hyper::Error(Listen, Os { code: 98, kind: AddrInUse, message: "Address already in use" }))
Jun 19 10:04:52 slim spectred[110632]: note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Jun 19 10:04:52 slim spectred[110632]: 2024-06-19 10:04:52.892+02:00 [ERROR] thread 'tokio-runtime-worker' panicked at protocol/p2p/./src/core/connection_handler.rs:91:29: P2P, Server 0.0.0.0:18111 stopped with error: tonic::transport::Error(Transport, hyper::Error(Listen, Os { code: 98, kind: AddrInUse, message: "Address already in use" }))
Jun 19 10:04:52 slim spectred[110632]: Exiting...
Jun 19 10:04:52 slim spectred[110632]: thread 'tokio-runtime-worker' panicked at protocol/p2p/./src/core/connection_handler.rs:91:29:
Jun 19 10:04:52 slim spectred[110632]: P2P, Server 0.0.0.0:18111 stopped with error: tonic::transport::Error(Transport, hyper::Error(Listen, Os { code: 98, kind: AddrInUse, message: "Address already in use" }))
Jun 19 10:04:52 slim spectred[110632]: Exiting...
Jun 19 10:04:52 slim spectred[110632]: 2024-06-19 10:04:52.893+02:00 [INFO ] Querying DNS seeder mainnet-dnsseed-1.spectre-network.org
Jun 19 10:04:52 slim systemd[1]: rusty-spectre.service: Main process exited, code=exited, status=1/FAILURE
Jun 19 10:04:52 slim systemd[1]: rusty-spectre.service: Failed with result 'exit-code'.
Jun 19 10:06:33 slim systemd[1]: rusty-spectre.service: Scheduled restart job, restart counter is at 26.
Jun 19 10:06:33 slim systemd[1]: Stopped rusty-spectred.
Jun 19 10:06:33 slim systemd[1]: Started rusty-spectred.
Jun 19 10:06:33 slim spectred[112005]: 2024-06-19 10:06:33.125+02:00 [INFO ] spectred v0.3.14
Jun 19 10:06:33 slim spectred[112005]: 2024-06-19 10:06:33.125+02:00 [INFO ] Application directory: /home/j/.rusty-spectre
Jun 19 10:06:33 slim spectred[112005]: 2024-06-19 10:06:33.125+02:00 [INFO ] Data directory: /home/j/.rusty-spectre/spectre-mainnet/datadir
Jun 19 10:06:33 slim spectred[112005]: 2024-06-19 10:06:33.125+02:00 [INFO ] Logs directory: /home/j/.rusty-spectre/spectre-mainnet/logs
Jun 19 10:06:33 slim spectred[112005]: 2024-06-19 10:06:33.125+02:00 [INFO ] Utxoindex Data directory /home/j/.rusty-spectre/spectre-mainnet/datadir/utxoindex
Jun 19 10:06:33 slim spectred[112005]: 2024-06-19 10:06:33.199+02:00 [INFO ] [UPnP] Attempting to register upnp... (to disable run the node with --disable-upnp)
Jun 19 10:06:43 slim spectred[112005]: 2024-06-19 10:06:43.226+02:00 [WARN ] [UPnP] Error adding port mapping: IO error: Resource temporarily unavailable (os error 11)
Jun 19 10:06:43 slim spectred[112005]: 2024-06-19 10:06:43.229+02:00 [INFO ] GRPC Server starting on: 127.0.0.1:18110
Jun 19 10:06:43 slim spectred[112005]: 2024-06-19 10:06:43.229+02:00 [INFO ] P2P Server starting on: 0.0.0.0:18111
Jun 19 10:06:43 slim spectred[112005]: 2024-06-19 10:06:43.229+02:00 [ERROR] thread 'tokio-runtime-worker' panicked at protocol/p2p/./src/core/connection_handler.rs:91:29: P2P, Server 0.0.0.0:18111 stopped with error: tonic::transport::Error(Transport, hyper::Error(Listen, Os { code: 98, kind: AddrInUse, message: "Address already in use" }))
Jun 19 10:06:43 slim spectred[112005]: thread 'tokio-runtime-worker' panicked at protocol/p2p/./src/core/connection_handler.rs:91:29:
Jun 19 10:06:43 slim spectred[112005]: P2P, Server 0.0.0.0:18111 stopped with error: tonic::transport::Error(Transport, hyper::Error(Listen, Os { code: 98, kind: AddrInUse, message: "Address already in use" }))
Jun 19 10:06:43 slim spectred[112005]: note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Jun 19 10:06:43 slim spectred[112005]: 2024-06-19 10:06:43.229+02:00 [ERROR] thread 'tokio-runtime-worker' panicked at rpc/grpc/server/src/connection_handler.rs:160:29: GRPC Server 127.0.0.1:18110 stopped with error: tonic::transport::Error(Transport, hyper::Error(Listen, Os { code: 98, kind: AddrInUse, message: "Address already in use" }))
Jun 19 10:06:43 slim spectred[112005]: thread 'tokio-runtime-worker' panicked at rpc/grpc/server/src/connection_handler.rs:160:29:
Jun 19 10:06:43 slim spectred[112005]: GRPC Server 127.0.0.1:18110 stopped with error: tonic::transport::Error(Transport, hyper::Error(Listen, Os { code: 98, kind: AddrInUse, message: "Address already in use" }))
Jun 19 10:06:43 slim spectred[112005]: 2024-06-19 10:06:43.229+02:00 [INFO ] Querying DNS seeder mainnet-dnsseed-1.spectre-network.org
Jun 19 10:06:43 slim spectred[112005]: Exiting...
Jun 19 10:06:43 slim spectred[112005]: Exiting...
Jun 19 10:06:43 slim systemd[1]: rusty-spectre.service: Main process exited, code=exited, status=1/FAILURE
Jun 19 10:06:43 slim systemd[1]: rusty-spectre.service: Failed with result 'exit-code'.

It has synced negligible amount of data as evidenced by the output of du, du -sh /home/j/.spectred/spectre-mainnet/*:

2.8M    /home/j/.rusty-spectre/spectre-mainnet/datadir
88K /home/j/.rusty-spectre/spectre-mainnet/logs

The remaining logs, /home/j/.rusty-spectre/spectre-mainnet/logs/rusty-spectre_err.log:

2024-06-19 10:08:33.565+02:00 [ERROR] thread 'tokio-runtime-worker' panicked at protocol/p2p/./src/core/connection_handler.rs:91:29: P2P, Server 0.0.0.0:18111 stopped with error: tonic::transport::Error(Transport, hyper::Error(Listen, Os { code: 98, kind: AddrInUse, message: "Address already in use" }))
2024-06-19 10:08:33.565+02:00 [ERROR] thread 'tokio-runtime-worker' panicked at rpc/grpc/server/src/connection_handler.rs:160:29: GRPC Server 127.0.0.1:18110 stopped with error: tonic::transport::Error(Transport, hyper::Error(Listen, Os { code: 98, kind: AddrInUse, message: "Address already in use" }))
2024-06-19 10:10:23.902+02:00 [WARN ] [UPnP] Error adding port mapping: IO error: Resource temporarily unavailable (os error 11)
2024-06-19 10:10:23.905+02:00 [ERROR] thread 'tokio-runtime-worker' panicked at protocol/p2p/./src/core/connection_handler.rs:91:29: P2P, Server 0.0.0.0:18111 stopped with error: tonic::transport::Error(Transport, hyper::Error(Listen, Os { code: 98, kind: AddrInUse, message: "Address already in use" }))
2024-06-19 10:10:23.905+02:00 [ERROR] thread 'tokio-runtime-worker' panicked at rpc/grpc/server/src/connection_handler.rs:160:29: GRPC Server 127.0.0.1:18110 stopped with error: tonic::transport::Error(Transport, hyper::Error(Listen, Os { code: 98, kind: AddrInUse, message: "Address already in use" }))

and /home/j/.rusty-spectre/spectre-mainnet/logs/rusty-spectre.log:

2024-06-19 10:10:23.904+02:00 [INFO ] GRPC Server starting on: 127.0.0.1:18110
2024-06-19 10:10:23.904+02:00 [INFO ] P2P Server starting on: 0.0.0.0:18111
2024-06-19 10:10:23.905+02:00 [ERROR] thread 'tokio-runtime-worker' panicked at protocol/p2p/./src/core/connection_handler.rs:91:29: P2P, Server 0.0.0.0:18111 stopped with error: tonic::transport::Error(Transport, hyper::Error(Listen, Os { code: 98, kind: AddrInUse, message: "Address already in use" }))
2024-06-19 10:10:23.905+02:00 [ERROR] thread 'tokio-runtime-worker' panicked at rpc/grpc/server/src/connection_handler.rs:160:29: GRPC Server 127.0.0.1:18110 stopped with error: tonic::transport::Error(Transport, hyper::Error(Listen, Os { code: 98, kind: AddrInUse, message: "Address already in use" }))
2024-06-19 10:10:23.905+02:00 [INFO ] Querying DNS seeder mainnet-dnsseed-1.spectre-network.org

I disabled upnp to isolate the issue, RUST_BACKTRACE=full /home/j/.cargo/bin/spectred --utxoindex --disable-upnp:

2024-06-19 10:37:13.399+02:00 [INFO ] spectred v0.3.14
2024-06-19 10:37:13.399+02:00 [INFO ] Application directory: /home/j/.rusty-spectre
2024-06-19 10:37:13.399+02:00 [INFO ] Data directory: /home/j/.rusty-spectre/spectre-mainnet/datadir
2024-06-19 10:37:13.399+02:00 [INFO ] Logs directory: /home/j/.rusty-spectre/spectre-mainnet/logs
2024-06-19 10:37:13.399+02:00 [INFO ] Utxoindex Data directory /home/j/.rusty-spectre/spectre-mainnet/datadir/utxoindex
2024-06-19 10:37:13.469+02:00 [INFO ] GRPC Server starting on: 127.0.0.1:18110
2024-06-19 10:37:13.469+02:00 [INFO ] P2P Server starting on: 0.0.0.0:18111
2024-06-19 10:37:13.469+02:00 [ERROR] thread 'tokio-runtime-worker' panicked at rpc/grpc/server/src/connection_handler.rs:160:29: GRPC Server 127.0.0.1:18110 stopped with error: tonic::transport::Error(Transport, hyper::Error(Listen, Os { code: 98, kind: AddrInUse, message: "Address already in use" }))
thread 'tokio-runtime-worker' panicked at rpc/grpc/server/src/connection_handler.rs:160:29:
GRPC Server 127.0.0.1:18110 stopped with error: tonic::transport::Error(Transport, hyper::Error(Listen, Os { code: 98, kind: AddrInUse, message: "Address already in use" }))
stack backtrace:
2024-06-19 10:37:13.469+02:00 [ERROR] thread 'tokio-runtime-worker' panicked at protocol/p2p/./src/core/connection_handler.rs:91:29: P2P, Server 0.0.0.0:18111 stopped with error: tonic::transport::Error(Transport, hyper::Error(Listen, Os { code: 98, kind: AddrInUse, message: "Address already in use" }))
thread 'tokio-runtime-worker' panicked at protocol/p2p/./src/core/connection_handler.rs:91:29:
P2P, Server 0.0.0.0:18111 stopped with error: tonic::transport::Error(Transport, hyper::Error(Listen, Os { code: 98, kind: AddrInUse, message: "Address already in use" }))
2024-06-19 10:37:13.470+02:00 [INFO ] Querying DNS seeder mainnet-dnsseed-1.spectre-network.org
   0:     0x63883bfcf6a0 - <unknown>
   1:     0x63883addc28b - <unknown>
   2:     0x63883bfcbb03 - <unknown>
   3:     0x63883bfcf41e - <unknown>
   4:     0x63883bfd0cbe - <unknown>
   5:     0x63883bfd0996 - <unknown>
   6:     0x63883b522acb - <unknown>
   7:     0x63883bfd12ed - <unknown>
   8:     0x63883bfd10a4 - <unknown>
   9:     0x63883bfcfb69 - <unknown>
  10:     0x63883bfd0df7 - <unknown>
  11:     0x63883ac39a73 - <unknown>
  12:     0x63883b7da5ff - <unknown>
  13:     0x63883b7d20b2 - <unknown>
  14:     0x63883b83db63 - <unknown>
  15:     0x63883c00728a - <unknown>
  16:     0x63883c006809 - <unknown>
  17:     0x63883bffafea - <unknown>
  18:     0x63883bffd9e1 - <unknown>
  19:     0x63883c005fc8 - <unknown>
  20:     0x63883c000f73 - <unknown>
  21:     0x63883bff8f90 - <unknown>
  22:     0x63883bfffd02 - <unknown>
  23:     0x63883bfe547a - <unknown>
  24:     0x63883bff5176 - <unknown>
  25:     0x63883bfd598b - <unknown>
  26:     0x71ca2ea94ac3 - <unknown>
  27:     0x71ca2eb26850 - <unknown>
  28:                0x0 - <unknown>
Exiting...
stack backtrace:
   0:     0x63883bfcf6a0 - <unknown>
   1:     0x63883addc28b - <unknown>
   2:     0x63883bfcbb03 - <unknown>
   3:     0x63883bfcf41e - <unknown>
   4:     0x63883bfd0cbe - <unknown>
   5:     0x63883bfd0996 - <unknown>
   6:     0x63883b522acb - <unknown>
   7:     0x63883bfd12ed - <unknown>
   8:     0x63883bfd10a4 - <unknown>
   9:     0x63883bfcfb69 - <unknown>
  10:     0x63883bfd0df7 - <unknown>
  11:     0x63883ac39a73 - <unknown>
  12:     0x63883bad8eaa - <unknown>
  13:     0x63883bad4de2 - <unknown>
  14:     0x63883bbb2f63 - <unknown>
  15:     0x63883c00728a - <unknown>
  16:     0x63883c006809 - <unknown>
  17:     0x63883bffafea - <unknown>
  18:     0x63883bffd9e1 - <unknown>
  19:     0x63883c005fc8 - <unknown>
  20:     0x63883c000f73 - <unknown>
  21:     0x63883bff8f90 - <unknown>
  22:     0x63883bfffd02 - <unknown>
  23:     0x63883bfe547a - <unknown>
  24:     0x63883bff5176 - <unknown>
  25:     0x63883bfd598b - <unknown>
  26:     0x71ca2ea94ac3 - <unknown>
  27:     0x71ca2eb26850 - <unknown>
  28:                0x0 - <unknown>
Exiting...
0xA001113 commented 4 months ago

The error shows already what you should do, stop the processes listening already on 18110 and 18111, so please check "Address already in use". Simple way: netstat -npla | grep -E '18110|18111' | grep LISTEN as privileged user and stop it. Most probably a previous spectred from Go is running.

whileunless commented 4 months ago

[...] Most probably a previous spectred from Go is running.

Yes indeed. Many thanks. Stopping Golang's spectred solved this issue.