paritytech / polkadot-sdk

The Parity Polkadot Blockchain SDK
https://polkadot.com/
1.89k stars 696 forks source link

network/litep2p: Update litep2p network backend to version 0.8.1 #6484

Closed lexnv closed 1 day ago

lexnv commented 2 days ago

This PR updates the litep2p backend to version 0.8.1 from 0.8.0.

The v0.8.1 release includes key fixes that enhance the stability and performance of the litep2p library. The focus is on long-running stability and improvements to polling mechanisms.

Long Running Stability Improvements

This issue caused long-running nodes to reject all incoming connections, impacting overall stability.

Addressed a bug in the connection limits functionality that incorrectly tracked connections due for rejection.

This issue caused an artificial increase in inbound peers, which were not being properly removed from the connection limit count.

This fix ensures more accurate tracking and management of peer connections #286.

Polling implementation fixes

This release provides multiple fixes to the polling mechanism, improving how connections and events are processed:

Fixed

Testing Done

Started kusama nodes running side by side with a higher number of inbound and outbound connections (500). We previously tested with peers bounded at 50. This testing filtered out the fixes included in the latest release.

With this high connection testing setup, litep2p outperforms libp2p in almost every domain, from performance to the warnings / errors encountered while operating the nodes.

TLDR: this is the version we need to test on kusama validators next

Repo Count Level Triage report
polkadot-sdk 409 warn Report .: . to .. Reason: .. Banned, disconnecting. ( Peer disconnected with inflight after backoffs. Banned, disconnecting. )
litep2p 128 warn Refusing to add known address that corresponds to a different peer ID
litep2p 54 warn inbound identify substream opened for peer who doesn't exist
polkadot-sdk 7 error 💔 Called on_validated_block_announce with a bad peer ID .*
polkadot-sdk 1 warn ❌ Error while dialing .: .
polkadot-sdk 1 warn Report .: . to .. Reason: .. Banned, disconnecting. ( Invalid justification. Banned, disconnecting. )
Repo Count Level Triage report
polkadot-sdk 1023 warn 💔 Ignored block (#. -- .) announcement from .* because all validation slots are occupied.
polkadot-sdk 472 warn Report .: . to .. Reason: .. Banned, disconnecting. ( Unsupported protocol. Banned, disconnecting. )
polkadot-sdk 379 error 💔 Called on_validated_block_announce with a bad peer ID .*
polkadot-sdk 163 warn Report .: . to .. Reason: .. Banned, disconnecting. ( Invalid justification. Banned, disconnecting. )
polkadot-sdk 116 warn Report .: . to .. Reason: .. Banned, disconnecting. ( Peer disconnected with inflight after backoffs. Banned, disconnecting. )
polkadot-sdk 83 warn Report .: . to .. Reason: .. Banned, disconnecting. ( Same block request multiple times. Banned, disconnecting. )
polkadot-sdk 4 warn Re-finalized block #. (.) in the canonical chain, current best finalized is #.*
polkadot-sdk 2 warn Report .: . to .. Reason: .. Banned, disconnecting. ( Genesis mismatch. Banned, disconnecting. )
polkadot-sdk 2 warn Report .: . to .. Reason: .. Banned, disconnecting. ( Not requested block data. Banned, disconnecting. )
polkadot-sdk 2 warn Can't listen on . because: .
polkadot-sdk 1 warn ❌ Error while dialing .: .
paritytech-cmd-bot-polkadot-sdk[bot] commented 1 day ago

Created backport PR for stable2407:

Please cherry-pick the changes locally and resolve any conflicts.

git fetch origin backport-6484-to-stable2407
git worktree add --checkout .worktree/backport-6484-to-stable2407 backport-6484-to-stable2407
cd .worktree/backport-6484-to-stable2407
git reset --hard HEAD^
git cherry-pick -x 8bea091ef131c8d962f45d89c28e17ece17bc5b2
git push --force-with-lease
paritytech-cmd-bot-polkadot-sdk[bot] commented 1 day ago

Created backport PR for stable2409:

Please cherry-pick the changes locally and resolve any conflicts.

git fetch origin backport-6484-to-stable2409
git worktree add --checkout .worktree/backport-6484-to-stable2409 backport-6484-to-stable2409
cd .worktree/backport-6484-to-stable2409
git reset --hard HEAD^
git cherry-pick -x 8bea091ef131c8d962f45d89c28e17ece17bc5b2
git push --force-with-lease