search

ipfs / kubo

An IPFS implementation in Go
https://docs.ipfs.tech/how-to/command-line-quick-start/
Other
15.99k stars 3k forks source link

Prevent Routing.Type=auto from enabling dhtserver mode if have too low of limits #9548

Open BigLep opened 1 year ago

BigLep commented 1 year ago

Done Criteria

A Kubo node with Routing.Type=auto doesn't attempt to be a DHT server if the following additional conditions aren't met:

  1. Swarm.ResourceMgr.System.ConnsInbound > 800 AND
  2. Swarm.ResourceMgr.System.StreamsInbound > 800 AND
  3. Swarm.ResourceMgr.Protocol.kaddht.StreamsInbound > 800 AND
  4. Swarm.ConnMgr is enabled

(Existing logic of ensuring the node is publicly dialable will still apply.)

Why Important

If a "dhtserver" node advertises itself to the DHT but has too-low of hard Swarm.ResourceMgr limits, then it won't be able to function and be of value to the DHT and instead degrades performance.

Notes

lidel commented 1 year ago

@Jorropo this will be only applied when custom Swarm.ResourceMgr.Limits is present in the config, right?

I assume we don't want to apply this to people running with the implicit defaults, otherwise many current DHT servers like mine will disappear when they updade, causing higher load on remaining DHT servers.

Example: I run my 0.18.0-rc2 node with implicit defaults ConnMgr (32/96) and ResourceMgr defaults as well.

Have 32GB RAM and ResourceMgr calculated ConnsInbound to be ~558, not 800:

{
  "Conns": 1000000000,
  "ConnsInbound": 558,
  "ConnsOutbound": 1000000000,
  "FD": 524288,
  "Memory": 8300000000,
  "Streams": 1000000000,
  "StreamsInbound": 8939,
  "StreamsOutbound": 1000000000
}
Jorropo commented 1 year ago

@lidel because of https://github.com/ipfs/kubo/pull/9555 the default scalling config will always be above thoses limit, so yes this indeed only applies to manually configured nodes. (note the checks are >= not >)

BigLep commented 1 year ago

@ajnavarro : per 2023-01-17 standup, base your work on https://github.com/ipfs/kubo/pull/9555

lidel commented 1 year ago

@Jorropo i think you've put wrong isssue in commit message of https://github.com/ipfs/kubo/commit/73ebad1892611a4b45b35798fe2d36cba2ec3bf6 :upside_down_face: – reopening, as this will be fixed by https://github.com/ipfs/kubo/pull/9561 instead.

ajnavarro commented 1 year ago

Wrong comment on commit