search

GenesysGo / dagger-testnet-issues

A place to report any issues experienced during the DAGGER Testnet waves. No duplicate issues please. Use the search function before submitting an issue. Duplicate issues will be closed.
Apache License 2.0
7 stars 1 forks source link

[ERROR] dagger_p2p - failed connect Unspecified #42

Open sgimborn opened 7 months ago

sgimborn commented 7 months ago

I get multiple errors per second since today. Last finalize was at 2024-02-02T19:43:08.777833349+01:00

Now I get 2024-02-02T23:22:16.883270034+01:00 [ERROR] dagger_p2p - failed connect Unspecified { source: Location { file: "/.cargo/registry/src/index.crates.io-6f17d22bba15001f/s2n-quic-transport-0.31.0/src/endpoint/connect.rs", line: 148, col: 40 } } 2024-02-02T23:22:16.883294591+01:00 [ERROR] dagger_p2p - failed connect Unspecified { source: Location { file: "/.cargo/registry/src/index.crates.io-6f17d22bba15001f/s2n-quic-transport-0.31.0/src/endpoint/connect.rs", line: 148, col: 40 } } 2024-02-02T23:22:16.883315410+01:00 [ERROR] dagger_p2p - failed connect Unspecified { source: Location { file: "/.cargo/registry/src/index.crates.io-6f17d22bba15001f/s2n-quic-transport-0.31.0/src/endpoint/connect.rs", line: 148, col: 40 } } 2024-02-02T23:22:16.884219590+01:00 [ERROR] dagger_p2p - failed connect Unspecified { source: Location { file: "/.cargo/registry/src/index.crates.io-6f17d22bba15001f/s2n-quic-transport-0.31.0/src/endpoint/connect.rs", line: 148, col: 40 } } 2024-02-02T23:22:16.884251942+01:00 [ERROR] dagger_p2p - failed connect Unspecified { source: Location { file: "/.cargo/registry/src/index.crates.io-6f17d22bba15001f/s2n-quic-transport-0.31.0/src/endpoint/connect.rs", line: 148, col: 40 } } 2024-02-02T23:22:16.884270588+01:00 [ERROR] dagger_p2p - failed connect Unspecified { source: Location { file: "/.cargo/registry/src/index.crates.io-6f17d22bba15001f/s2n-quic-transport-0.31.0/src/endpoint/connect.rs", line: 148, col: 40 } } 2024-02-02T23:22:16.884289314+01:00 [ERROR] dagger_p2p - failed connect Unspecified { source: Location { file: "/.cargo/registry/src/index.crates.io-6f17d22bba15001f/s2n-quic-transport-0.31.0/src/endpoint/connect.rs", line: 148, col: 40 } } 2024-02-02T23:22:16.884309722+01:00 [ERROR] dagger_p2p - failed connect Unspecified { source: Location { file: "/.cargo/registry/src/index.crates.io-6f17d22bba15001f/s2n-quic-transport-0.31.0/src/endpoint/connect.rs", line: 148, col: 40 } } 2024-02-02T23:22:16.885293024+01:00 [ERROR] dagger_p2p - failed connect Unspecified { source: Location { file: "/.cargo/registry/src/index.crates.io-6f17d22bba15001f/s2n-quic-transport-0.31.0/src/endpoint/connect.rs", line: 148, col: 40 } } 2024-02-02T23:22:16.885416040+01:00 [ERROR] dagger_p2p - failed connect Unspecified { source: Location { file: "/.cargo/registry/src/index.crates.io-6f17d22bba15001f/s2n-quic-transport-0.31.0/src/endpoint/connect.rs", line: 148, col: 40 } } 2024-02-02T23:22:16.885443753+01:00 [ERROR] dagger_p2p - failed connect Unspecified { source: Location { file: "/.cargo/registry/src/index.crates.io-6f17d22bba15001f/s2n-quic-transport-0.31.0/src/endpoint/connect.rs", line: 148, col: 40 } } 2024-02-02T23:22:16.885466987+01:00 [ERROR] dagger_p2p - failed connect Unspecified { source: Location { file: "/.cargo/registry/src/index.crates.io-6f17d22bba15001f/s2n-quic-transport-0.31.0/src/endpoint/connect.rs", line: 148, col: 40 } } 2024-02-02T23:22:16.885500411+01:00 [ERROR] dagger_p2p - failed connect Unspecified { source: Location { file: "/.cargo/registry/src/index.crates.io-6f17d22bba15001f/s2n-quic-transport-0.31.0/src/endpoint/connect.rs", line: 148, col: 40 } } 2024-02-02T23:22:16.886216100+01:00 [ERROR] dagger_p2p - failed connect Unspecified { source: Location { file: "/.cargo/registry/src/index.crates.io-6f17d22bba15001f/s2n-quic-transport-0.31.0/src/endpoint/connect.rs", line: 148, col: 40 } } 2024-02-02T23:22:16.886291404+01:00 [ERROR] dagger_p2p - failed connect Unspecified { source: Location { file: "/.cargo/registry/src/index.crates.io-6f17d22bba15001f/s2n-quic-transport-0.31.0/src/endpoint/connect.rs", line: 148, col: 40 } } 2024-02-02T23:22:16.886337272+01:00 [ERROR] dagger_p2p - failed connect Unspecified { source: Location { file: "/.cargo/registry/src/index.crates.io-6f17d22bba15001f/s2n-quic-transport-0.31.0/src/endpoint/connect.rs", line: 148, col: 40 } } 2024-02-02T23:22:16.886377027+01:00 [ERROR] dagger_p2p - failed connect Unspecified { source: Location { file: "/.cargo/registry/src/index.crates.io-6f17d22bba15001f/s2n-quic-transport-0.31.0/src/endpoint/connect.rs", line: 148, col: 40 } } 2024-02-02T23:22:16.886410081+01:00 [ERROR] dagger_p2p - failed connect Unspecified { source: Location { file: "/.cargo/registry/src/index.crates.io-6f17d22bba15001f/s2n-quic-transport-0.31.0/src/endpoint/connect.rs", line: 148, col: 40 } } 2024-02-02T23:22:16.887210542+01:00 [ERROR] dagger_p2p - failed connect Unspecified { source: Location { file: "/.cargo/registry/src/index.crates.io-6f17d22bba15001f/s2n-quic-transport-0.31.0/src/endpoint/connect.rs", line: 148, col: 40 } } 2024-02-02T23:22:16.887244949+01:00 [ERROR] dagger_p2p - failed connect Unspecified { source: Location { file: "/.cargo/registry/src/index.crates.io-6f17d22bba15001f/s2n-quic-transport-0.31.0/src/endpoint/connect.rs", line: 148, col: 40 } } 2024-02-02T23:22:16.887265547+01:00 [ERROR] dagger_p2p - failed connect Unspecified { source: Location { file: "/.cargo/registry/src/index.crates.io-6f17d22bba15001f/s2n-quic-transport-0.31.0/src/endpoint/connect.rs", line: 148, col: 40 } } 2024-02-02T23:22:16.887284424+01:00 [ERROR] dagger_p2p - failed connect Unspecified { source: Location { file: "/.cargo/registry/src/index.crates.io-6f17d22bba15001f/s2n-quic-transport-0.31.0/src/endpoint/connect.rs", line: 148, col: 40 } } 2024-02-02T23:22:16.887302439+01:00 [ERROR] dagger_p2p - failed connect Unspecified { source: Location { file: "/.cargo/registry/src/index.crates.io-6f17d22bba15001f/s2n-quic-transport-0.31.0/src/endpoint/connect.rs", line: 148, col: 40 } }

sgimborn commented 7 months ago

btw after restarting it its finalizing again but of course i dropped out of the queue because of this issue :(

tracy-codes commented 7 months ago

Thank you for reporting this. Can you please provide the following information:

1) your node's ID 2) what server hosting provider you're using (if using one, if not, just say residential or if its collocated in a datacenter please provide the datacenter information) 3) geographic location of your node 4) wield version currently running

sgimborn commented 7 months ago
  1. 7suZGsUkF3kgm8AeHzfGLUqiDUKHCFVPfLQPT4i5WjKU
  2. netcup
  3. ANX84, Nürnberg, Germany
  4. 0.8.1
sgimborn commented 7 months ago

Had it again... Anything I could do against it?

2024-02-15T20:55:03.777951159+01:00 [ERROR] dagger_p2p - failed connect Unspecified { source: Location { file: "/.cargo/registry/src/index.crates.io-6f17d22bba15001f/s2n-quic-transport-0.31.0/src/endpoint/connect.rs", line: 148, col: 40 } }

sgimborn commented 7 months ago

here the most recent error logs before the failed connect Unspecified started 2024-02-15T20:28:19.465164907+01:00 [ERROR] dagger::p2p::graph::event - Failed to save event in database: IO error: While open a file for random read: /mnt/dag/historydb/071436.sst: Too many open files 2024-02-15T20:28:19.476750797+01:00 [ERROR] log_panics - thread 'wield-metrics-agent' panicked at 'Client::new(): reqwest::Error { kind: Builder, source: Os { code: 24, kind: Uncategorized, message: "Too many open files" } }': /.cargo/registry/src/index.crates.io-6f17d22bba15001f/reqwest-0.11.22/src/blocking/client.rs:877 2024-02-15T20:28:19.483476175+01:00 [ERROR] log_panics - thread 'controller' panicked at 'called Result::unwrap() on an Err value: "SendError(..)"': dagger-logger/src/metrics.rs:439 2024-02-15T20:28:20.078407417+01:00 [ERROR] log_panics - thread 'comms' panicked at 'controller will not drop receiver: SendError { .. }': core/src/p2p/message/handlers/client.rs:434 2024-02-15T20:28:20.342267552+01:00 [ERROR] log_panics - thread 'comms' panicked at 'controller will not drop receiver: SendError { .. }': core/src/p2p/message/handlers/client.rs:434 2024-02-15T20:28:21.326915707+01:00 [ERROR] log_panics - thread '' panicked at 'called Result::unwrap() on an Err value: "SendError(..)"': dagger-logger/src/metrics.rs:439 2024-02-15T20:29:20.110292926+01:00 [ERROR] dagger::p2p::graph::event - Failed to save event in database: IO error: While open a file for random read: /mnt/dag/historydb/071436.sst: Too many open files 2024-02-15T20:29:20.110476537+01:00 [ERROR] dagger::p2p::graph::event - Failed to save event in database: IO error: While open a file for random read: /mnt/dag/historydb/071436.sst: Too many open files 2024-02-15T20:29:20.110685015+01:00 [ERROR] dagger::p2p::graph::event - Failed to save event in database: IO error: While open a file for random read: /mnt/dag/historydb/071436.sst: Too many open files 2024-02-15T20:29:20.110859188+01:00 [ERROR] dagger::p2p::graph::event - Failed to save event in database: IO error: While open a file for random read: /mnt/dag/historydb/071436.sst: Too many open files 2024-02-15T20:29:20.111156898+01:00 [ERROR] dagger::p2p::graph::event - Failed to save event in database: IO error: While open a file for random read: /mnt/dag/historydb/071436.sst: Too many open files 2024-02-15T20:29:20.111176975+01:00 [ERROR] dagger::p2p::graph::event - Failed to save event in database: IO error: While open a file for random read: /mnt/dag/historydb/071436.sst: Too many open files 2024-02-15T20:29:20.111426253+01:00 [ERROR] dagger::p2p::graph::event - Failed to save event in database: IO error: While open a file for random read: /mnt/dag/historydb/071436.sst: Too many open files 2024-02-15T20:29:20.112759301+01:00 [ERROR] dagger::p2p::graph::event - Failed to save event in database: IO error: While open a file for random read: /mnt/dag/historydb/071436.sst: Too many open files 2024-02-15T20:29:20.112838562+01:00 [ERROR] dagger::p2p::graph::event - Failed to save event in database: IO error: While open a file for random read: /mnt/dag/historydb/071436.sst: Too many open files 2024-02-15T20:29:20.112854723+01:00 [ERROR] dagger::p2p::graph::event - Failed to save event in database: IO error: While open a file for random read: /mnt/dag/historydb/071436.sst: Too many open files 2024-02-15T20:29:20.113021471+01:00 [ERROR] dagger::p2p::graph::event - Failed to save event in database: IO error: While open a file for random read: /mnt/dag/historydb/071436.sst: Too many open files 2024-02-15T20:29:20.113409453+01:00 [ERROR] dagger::p2p::graph::event - Failed to save event in database: IO error: While open a file for random read: /mnt/dag/historydb/071436.sst: Too many open files 2024-02-15T20:29:20.113448699+01:00 [ERROR] dagger::p2p::graph::event - Failed to save event in database: IO error: While open a file for random read: /mnt/dag/historydb/071436.sst: Too many open files 2024-02-15T20:29:20.113502521+01:00 [ERROR] dagger::p2p::graph::event - Failed to save event in database: IO error: While open a file for random read: /mnt/dag/historydb/071436.sst: Too many open files 2024-02-15T20:29:20.113610117+01:00 [ERROR] dagger::p2p::graph::event - Failed to save event in database: IO error: While open a file for random read: /mnt/dag/historydb/071436.sst: Too many open files 2024-02-15T20:29:20.118126020+01:00 [ERROR] log_panics - thread 'dagger-toplevel' panicked at 'graph -> controller bundle sender panic: SendError { .. }': core/src/p2p/graph/dagger.rs:2127 2024-02-15T20:29:20.119164345+01:00 [ERROR] log_panics - thread '' panicked at 'graph module should never drop sender: RecvError': core/src/p2p/graph/dagger.rs:4184 2024-02-15T20:29:20.507027014+01:00 [ERROR] dagger_p2p - failed connect Unspecified { source: Location { file: "/.cargo/registry/src/index.crates.io-6f17d22bba15001f/s2n-quic-transport-0.31.0/src/endpoint/connect.rs", line: 148, col: 40 } } 2024-02-15T20:29:20.507185407+01:00