networking/litep2p: Node running litep2p seems to be leaking memory

[alexggh]

Looking over the dashboards on our kusama validators the memory on node that is running litep2p seems to be constantly increasing it is now at 12GiB, all other nodes are around 3-4 GiB and constant.

https://grafana.teleport.parity.io/goto/Uoh4CQmHg?orgId=1

cc: @paritytech/networking

[lexnv]

Thanks Alex for raising this! 🙏

We also had this issue with addresses, although the memory consumed here is in the order of GiB (https://github.com/paritytech/polkadot-sdk/pull/5998).

There are a few places that come to mind where to look at next:

• Transport manager peers: we don't seem to ever remove old/disconnected entries
• Address store: this is intertwined at the moment with the above peer state, which might grow indefinely as well. There's already work in this area to have an address store eviction algorithm to bound the growth of internal structures:
  • https://github.com/paritytech/litep2p/pull/250

I would start by looking at litep2p and then move to substrate code

[lexnv]

Initial testing with debug logs built on top of branch lexnv/holistic-litep2p-test-dhtandpeerset shows memory leaks in litep2p:

• transport manager peer state (from above)
• TCP / WebSocket state tracking

Will let my node running over night and followup with patches

[lexnv]

We need metrics to filter out potential leaks (ie monotonically increasing state tracking is a concern).

We have around 3 separate leaks:

• https://github.com/paritytech/litep2p/issues/268
• https://github.com/paritytech/litep2p/issues/269
• https://github.com/paritytech/litep2p/issues/270 (grouped together the cancellation state by connection ID with the AbortHandler since I believe they are similar)

1. Transport Manager State Leak

• https://github.com/paritytech/litep2p/issues/268
• we are increasing state tracking of peers in an unbounded way
• state tracking can be a bit smarter and similar to a garbage collector removing the state of disconnected peers once every N minutes

2. TCP/WebSocket Pending Dials Leak

• https://github.com/paritytech/litep2p/issues/269
• we are leaking pending dials state tracking, i expect we are not removing this state properly

3. TCP/WebSocket Cancellation Logic Leak

• https://github.com/paritytech/litep2p/issues/270
• cancel dials leak connection IDs
• cancel dials leak AbortableHandlers for dialing futures

[lexnv]

Litep2p PRs

For more details and explained edge-cases when the leaks happen see:

• https://github.com/paritytech/litep2p/pull/271
• https://github.com/paritytech/litep2p/pull/272

[lexnv]

Lower severity memory leaks in the ping and identify protocols:

• https://github.com/paritytech/litep2p/pull/274
• https://github.com/paritytech/litep2p/pull/273