Remove the need of updating listen_addresses and external_addresses on every single call to NetworkWorker::next_action.
This is very inefficient, as it involves creating new vectors from hashsets on every call, even if nothing changed in the address set. It turns out to be time consuming when testing in Versi, under a heavy load - about 5-8% of the total polkadot CPU utilisation.
This is a tricky problem to solve, since this data needs to be accessible from a NetworkService instance, which does not have direct access to the Swarm.
Changed the underlying data store from a Vec to a HashSet, as it's more efficient for inserts/removes (which is also what libp2p uses for this data). It's converted to a vector only when needed (rarely).
For listen_addresses, we now simply alter the HashSet as a reaction to FromSwarm::NewListenAddr and FromSwarm::ExpiredListenAddr, which is returned by Swarm::poll_next_event(), whenever there is a new listen address.
For external_addresses it's a bit different, because they are actually updated either from a network Behaviour or from the NetworkWorker itself. This PR introduces an ExternalAddress struct, that is defined on the PeerInfo network behaviour. It reacts to external address swarm events and updates the shared view of the addresses whenever needed.
Checklist
[ ] My PR includes a detailed description as outlined in the "Description" section above
[ ] My PR follows the labeling requirements of this project (at minimum one label for each A, B, C and D required)
[ ] I have made corresponding changes to the documentation (if applicable)
[ ] I have added tests that prove my fix is effective or that my feature works (if applicable)
[ ] If this PR alters any external APIs or interfaces used by Polkadot, the corresponding Polkadot PR is ready as well as the corresponding Cumulus PR (optional)
You can remove the "Checklist" section once all have been checked. Thank you for your contribution!
Motivation
Remove the need of updating
listen_addresses
andexternal_addresses
on every single call toNetworkWorker::next_action
.This is very inefficient, as it involves creating new vectors from hashsets on every call, even if nothing changed in the address set. It turns out to be time consuming when testing in Versi, under a heavy load - about 5-8% of the total polkadot CPU utilisation.
Discussion about motivation here: https://github.com/libp2p/rust-libp2p/discussions/3840#discussioncomment-6575512
Fix
This is a tricky problem to solve, since this data needs to be accessible from a
NetworkService
instance, which does not have direct access to theSwarm
.Changed the underlying data store from a
Vec
to aHashSet
, as it's more efficient for inserts/removes (which is also what libp2p uses for this data). It's converted to a vector only when needed (rarely).For
listen_addresses
, we now simply alter theHashSet
as a reaction toFromSwarm::NewListenAddr
andFromSwarm::ExpiredListenAddr
, which is returned bySwarm::poll_next_event()
, whenever there is a new listen address.For
external_addresses
it's a bit different, because they are actually updated either from a networkBehaviour
or from theNetworkWorker
itself. This PR introduces anExternalAddress
struct, that is defined on thePeerInfo
network behaviour. It reacts to external address swarm events and updates the shared view of the addresses whenever needed.Checklist
A
,B
,C
andD
required)You can remove the "Checklist" section once all have been checked. Thank you for your contribution!