[Connection Manager] Improve fallback mechanism when remote peer rejects connection

fryorcraken commented 1 year ago

This is a feature request

Problem

The reported problem is that js-waku fails to connect to nodes in the wakuv2.prod fleet. This is because some nodes have reached maximum connection number (confirmed, 150 connection for wakuv2.prod and 300 for wakuv2.test, all maxed out).

Currently, RAILGUN tries to use wakuv2.prod, if it fails, it tries to wakuv2.test. They also run their own nwaku node, ensure yet how they pass this.

Proposed Solution

[x] 1. Ensure we can pass several enrtree + multiaddr to waku node for bootstrap purposes: https://github.com/waku-org/js-waku/pull/1379
[x] 2. Ensure that we dial several nodes in parallel. Currently, it should be dialling any discovered peer that is a bootstrap peer until the node is connected to maxBootstrapPeerAllowed. Note that this is not the observed behaviour. Reproducible with https://examples.waku.org/relay-js/. Once fixed, we may want to cap the number of peers we dial in parallel: https://github.com/waku-org/js-waku/pull/1380
[x] 3. Ensure we drop connections to peers if we are connected to more bootstrap peers than necessary, once (2) is done: https://github.com/waku-org/js-waku/pull/1380
[x] 4. Make peer exchange work, tracked with #1309.
- Note: need to review this item as we don't want the usage to spill over Status fleet.
[ ] 5. Persist peers/peer store in browser local storage so that at next start, it tries first to connect to known peers (retrieved via peer exchange) instead of bootstrap, still using bootstrap at backup. ~~https://github.com/waku-org/js-waku-examples/pull/262~~ Will be tackled with https://github.com/waku-org/js-waku/issues/1463

Notes

(4) Need to be careful with usage of peer exchange as we do not want the traffic to spill over the Status fleet. The Status fleet will be moved to separate sharded pubsub topics, so peer management on nwaku side needs to ensure that connections to relay nodes that do not subscribes to same topic are dropped.

fryorcraken commented 1 year ago

See for (1) how railgun currently uses the API:

https://github.com/Railgun-Community/waku-relayer-client/blob/1e90006e524ac1b7ab52d388c21588525dd44f60/src/waku/waku-relayer-waku-core.ts#L62

      const bootstrapNodes = getPredefinedBootstrapNodes(fleet, 2);

      const peers = [...new Set([...bootstrapNodes, ...this.directPeers])];
      const waku: Waku = await createRelayNode({
        libp2p: {
          peerDiscovery: [bootstrap({ list: peers })],
        },
      });

danisharora099 commented 1 year ago

Update: TODOs 1-3 have been addressed. TODO 4 is blocked by https://github.com/waku-org/js-waku/issues/1309#issuecomment-1567175094 TODO5 is to be done post TODO4.

fryorcraken commented 1 year ago

Is it now possible to easily pass wakuv2.test + wakuv2.prod entrees + some static multaddrs? IF so, how?

fryorcraken commented 1 year ago

Blocked on https://github.com/waku-org/nwaku/issues/1831

danisharora099 commented 1 year ago

Above has been resolved;

Blocked by https://github.com/libp2p/js-libp2p/issues/1904 and https://github.com/waku-org/js-waku/issues/1415

fryorcraken commented 1 year ago

Is it now possible to easily pass wakuv2.test + wakuv2.prod entrees + some static multaddrs? IF so, how?

@danisharora099 I'd like this to be documented in docs.waku.org to encourage developers to run their own node while still being part of the public network. Can you point to a test or snippet that demonstrates it so we can add it to the docs? cc @LordGhostX

danisharora099 commented 1 year ago

Is it now possible to easily pass wakuv2.test + wakuv2.prod entrees + some static multaddrs? IF so, how?

@danisharora099 I'd like this to be documented in docs.waku.org to encourage developers to run their own node while still being part of the public network. Can you point to a test or snippet that demonstrates it so we can add it to the docs? cc @LordGhostX

Apologies for delay in the response, missed this message.

Yes, it is already possible to do pass multiple enrtrees + static multiaddrs as follows:

waku = await createLightNode({
        libp2p: {
          peerDiscovery: [
            bootstrap({ list: [multiAddrWithId.toString()] }),
            wakuDnsDiscovery(
              [enrTree["PROD"], enrTree["TEST"]],
              NODE_REQUIREMENTS,
            ),
          ],
        },
      });

(https://github.com/waku-org/js-waku/pull/1448)

fryorcraken commented 1 year ago

I would suggest to remove point (5) from here and track it under ~https://github.com/waku-org/js-waku/issues/914~ https://github.com/waku-org/js-waku/issues/1463 instead.

Also note that for (5) see https://github.com/waku-org/js-waku/issues/914#issuecomment-1668882251

danisharora099 commented 1 year ago

I would suggest to remove point (5) from here and track it under ~#914~ #1463 instead.

Also note that for (5) see #914 (comment)

Sounds good, thanks. Marking this issue as complete for now.

Future work to be tracked with the above mentioned issues.

waku-org / js-waku