Failures were observed for transactions submitted to mainnet, and they were failing with broken pipe errors, and Hedera-specific errors, but only a small percentage of transactions were affected. The root cause was node-specific.
This adds logging, with the node id (realm/shard/number) and index (into the client's Vec of known nodes) at the debug level for requests, and success responses, and at the warn level for failure responses.
Notes for reviewer:
Example log entries:
Execution of hedera::ping_query::PingQuery on node at index 7 / node id 0.0.15 will continue due to GrpcStatus(Status { code: Unavailable, message: "error trying to connect: tcp connect error: Broken pipe (os error 32)", source: Some(tonic::transport::Error(Transport, hyper::Error(Connect, ConnectError("tcp connect error", Os { code: 32, kind: BrokenPipe, message: "Broken pipe" })))) })
Backing off for 1255ms after failure of attempt 3 due to QueryPreCheckStatus { status: Ok, transaction_id: "0.0.XXX@XXX" }
Description:
Failures were observed for transactions submitted to mainnet, and they were failing with broken pipe errors, and Hedera-specific errors, but only a small percentage of transactions were affected. The root cause was node-specific.
This adds logging, with the node id (realm/shard/number) and index (into the client's
Vecof known nodes) at thedebuglevel for requests, and success responses, and at thewarnlevel for failure responses.Notes for reviewer:
Example log entries:
Checklist