koinos / koinos-p2p

The p2p microservice orchestrates the distribution of blocks and transactions between peers.
MIT License
6 stars 4 forks source link

[BUG]: Local RPC timeout causes disconnect #192

Closed mvandeberg closed 2 years ago

mvandeberg commented 2 years ago

Is there an existing issue for this?

Current behavior

2022-02-15 23:10:00.957249 (p2p.Koinos) [node/node.go:176] <info>: Publishing block - Height: 35071 ID: 0x1220197cda56de9af8a15f9c194649c409ab7ca283d75d8661773d8534f03526b07e Prev: 0x122034e52417b378b852354dd64ef576105be1687fafc8ee34f8e114ddca9220739d

2022-02-15 23:10:01.917393 (p2p.Koinos) [koinos-mq-golang@v0.0.0-20211026183607-441fb7106dd3/client.go:265] <warn>: RPC error: context deadline exceeded

2022-02-15 23:10:01.917440 (p2p.Koinos) [p2p/error_handler.go:81] <info>: Encountered peer error: XXXXXXXXXXXXXXXXX, block application failed: local RPC request timed out ApplyBlock, context deadline exceeded. Current error score: 101738

2022-02-15 23:10:01.918091 (p2p.Koinos) [koinos-mq-golang@v0.0.0-20211026183607-441fb7106dd3/client.go:266] <warn>: Trying again in 1 seconds

2022-02-15 23:10:01.919130 (p2p.Koinos) [p2p/connection_manager.go:191] <info>: Disconnected from peer: /ip4/XXX.XXX.XXX.XXX/tcp/8888/p2p/XXXXXXXXXXXXXXXXX

Local RPC timeout is adding the full error score to a peer, when it is not the peer's fault, eventually resulting in disconnect and slowing sync time.

Expected behavior

Syncing should continue working when the peer is not misbehaving.

Steps to reproduce

v3

Environment

- OS: Docker

Anything else?

No response