Eth2 Networking Call 4

[djrtwo]

Call 3

Meeting Date/Time:

Meeting Date/Time: Wednesday 2020/3/25 at 13:00 GMT

Meeting Duration:

1.25 hours

Notes

Notes taken by @benjaminion

Agenda

1. Opening
2. Updates
3. Discussion of challenges and edge-cases in aggregation strategy/discovery/subnets
   • Conforming on this and working through some of the edge-cases is one of the major blockers for mc-testnets. We'll spend some time knowledge sharing and working through problems on this call. Please come prepared to discuss!
4. Network requirements
5. Spec discussion
   • ping
6. Open discussion
7. Closing remarks and plans for next call or other organizational items

[mcdee]

Doubt I'll be able to make it to the call, but a few notes on attestation subnets:

• for a validator to be sure of its attestation being included in an aggregation, it needs to talk to a beacon node that is part of the relevant pubsub channel (technically no more than one peer removed from the same, but the general point stands)
• as such, a beacon node needs to know in advance which subnets it needs to be part of
• this in turn requires the beacon node to know in advance which validators will talk to it for a given epoch
• this places, in my opinion, unreasonable demands on the beacon node
  • For example, in a high availability configuration a beacon node may receive an attestation "cold", without having any prior knowledge of the validator sending the attestation. The only way the beacon node could provide surety in this situation is for it to subscribe to all attestation subnets, which somewhat defeats the purpose

My concern here is that ultimately there will be a class of "preferred" beacon nodes that subscribe to all attestation subnets, which would lead towards some sort of supernode/hub and spoke configuration for the network.

Wondering if it makes more sense for beacon nodes to pick a number of subnets to subscribe to on startup, and to have a "not my subnet" channel (to which every node subscribes) they can use to send along attestations for subnets to which they don't subscribe. Having static subnets would give a bit more stability, users could define the number of subnets to which they want subscribe (even down to 0 for constrained systems), and it wouldn't take a large network or subscriptions/node to have a pretty high chance that attestations would hit a node with the relevant subscription within two hops.

This could, of course, have just as many edge cases as the dynamic subscription system currently slated for 0.11, but I think it's worth considering. Also note that it would be possible to retain the current dynamic subscription model and just add the "not my subnet" channel for all nodes to give a similar effect (outside of the much lower turnover of subscriptions and the ability for beacon nodes to define the subnets to which they wish to subscribe).

[AgeManning]

My current thoughts on the "preferred" beacon nodes, is that a BN that has attached validators, will prioritise peers based on the number of long-lived subscribed subnets. BNs that don't have validators attached, are indifferent to peers with long-lived subnets or not. I was considering adding a CLI param that allows users to randomly subscribe to subnets without validators if they choose.

I don't have metrics yet. But I think the feasibility of having the current v0.11 system work, is the timing to discover peers for a required subnet and ability to subscribe to the topics in time for the validators attestation duty. We have been working on our discovery to speed up and streamline discovery searches for peers on a specific subnet. We also need to be smart about our peer management and timing (prioritise peers with large number of long-lived subnets) and prevent lots of discovery lookups.

[AgeManning]

To help clarify some discussions I've had with various people, I've made a draft/example PR for adding a PING and ENR method to our RPC: https://github.com/ethereum/eth2.0-specs/pull/1673

If we have the time, I think it might be useful to discuss this.

[mcdee]

a BN that has attached validators

Coupling validators to beacon nodes makes advanced architectures more difficult. Round-robin load balancing is the simplest example, but any sort of cluster would be affected here unless they are all connected to each other and online, which somewhat defeats the purpose.

prioritise peers with large number of long-lived subnets

That'll be the supernode thing I was worrying about. If everyone does this, we'll either end up with hub-and-spoke or everyone running all the subnets to obtain more peers. What about the user running the RPi at home that only has enough resources to run a single subnet?

[nkeywal]

Can we add to the agenda a review of the network requirements? A few of us drafted something during ethcc: https://github.com/ethereum/eth2.0-pm/issues/131#issuecomment-594632185

[jrhea]

If we have time, I'd like to discuss the assumptions developers that are working on light clients, tooling, and other non beacon node implementations need to take into account when joining the eth2 network. A few questions i have are:

• should a node be able to tell the difference between a light client and a beacon node. if not, how could that be accomplished?
• will beacon nodes immediately disconnect if a peer doesn't implement all req/resp messages?
• What is the recommendation on how client implementations should handle peers that do not abide by all MUST's in the networking spec? Should this recommendation be added to the spec somewhere?

Note: I realize that most of these questions are related to each other. I just wanted to list them for the sake of clarity and to provide more context to the discussion.

[mkalinin]

Also note that it would be possible to retain the current dynamic subscription model and just add the "not my subnet" channel for all nodes to give a similar effect

Do I understand correctly that "not my subnet" topic's mesh will include all the nodes that lacks subscription to at least one subnet topic? There is a set of peers in the gossipsub protocol called fanout that should be used for this kind of purpose. Otherwise, if such a topic exists it could open a DoS vector as peers are able to send any attestation directly to this topic that will likely include most peers in the network.

[mcdee]

Do I understand correctly that "not my subnet" topic's mesh will include all the nodes that lacks subscription to at least one subnet topic?

The "not my subnet" would be a general channel to which all nodes would subscribe. Fanout only works for a single hop, so if the neither I nor the peers to which I am connected are connected to the subnet the message will be lost.

When a node receives an attestation on the "not my subnet" channel it would:

• send it on the relevant channel if subscribed
• send it to the fanout if the number of peers is suitable (>64? can we know if a peer has a subscription to a given channel so we can trust fanout rather than have a probabilistic approach?)
• gossip it onwards if not which would allow multiple hops.

If we don't have this then minimally-connected nodes aren't going to be able to get attestations out to the network without there being supernodes (i.e. nodes connected to all subnets, so guaranteed to pick up attestations on fanout).

As for the DoS vector: yes, this does open up a potential issue. Would need to consider what additional load this could put on a node, more in the freeloading case than the DoS case.

[mkalinin]

The "not my subnet" would be a general channel to which all nodes would subscribe. Fanout only works for a single hop, so if the neither I nor the peers to which I am connected are connected to the subnet the message will be lost.

I see your point. And looks like I messed up the fanout purpose. Apparently, fanout is used by message publisher to send a message to the topic it isn't subscribed to. It's not meant to be used as a relay mechanism.

[benjaminion]

My notes here: https://hackmd.io/@benjaminion/rkEn7C_88

Please lmk or edit directly if any typos/inaccuracies!