Implement per-validator reward APIs

michaelsproul commented 2 years ago

Description

For the protocol fellowship I've proposed a project to add reward calculation APIs to the beacon-APIs standard (https://github.com/eth-protocol-fellows/cohort-three/pull/51). This issue contemplates the design and implementation on the Lighthouse side.

Present Behaviour

Currently we have a few APIs for doing things with rewards, but none are fully satisfactory:

There's /lighthouse/validator_inclusion/{epoch}/{validator_id}, which includes whether or not the target/head votes matched, but nothing about the source vote, nor the rewards.
There's the undocumented /lighthouse/analysis/attestation_performance/{validator_id} endpoint that we created for beacon.watch (a quasi block explorer, see #3362). It has all relevant attestation information including source matching, but lacks inclusion distance and reward information.
There's the /lighthouse/analysis/block_rewards API for block rewards that is used by blockprint. However it has several limitations: it considers only attestations and sync committee rewards (no slashings) and it is liable to return slightly incorrect results when validators make slashable attestations.

Design Questions

Some main design questions (DQs) that I think we should iterate on are:

DQ1: Bulk or single block/epoch

Should the new reward APIs operate on a single block/epoch or a range? Most of our existing APIs operate on a range because that's substantially more efficient with Lighthouse's database design. However most of the standard beacon-APIs operate only on a single block or epoch at a time, so perhaps we should keep things consistent.

Another point in favour of the single block/single epoch APIs is that we can likely optimise behind the scenes to make them efficient. The ongoing tree-states work will help with this by drastically reducing the size of archive nodes (4TB -> 250GB) and enabling caching of more states in memory (see: https://github.com/sigp/lighthouse/pull/3206).

DQ2: Reward granularity

There's a trade-off between implementation complexity and detail when it comes to the granularity of reward data:

Should attestation rewards include a breakdown by head/target/source (probably yes)
Should block rewards include a breakdown by included attestations, sync aggregate, slashings (maybe?)

We should also be mindful of the fact that rewards for attestations and sync committee messages can be negative. We should specify the rewards as signed integers.

Implementation Considerations

All of the reward APIs should probably use the BlockReplayer, like the existing reward APIs do.

Block Rewards

One way to calculate a coarse block reward is:

Replay the relevant block(s)
Record the balance diff for the proposer index using BlockReplayer hooks
Subtract any attestation or sync committee reward paid to the proposer in the same slot from the balance diff.

Alternatively we can calculate fine-grained rewards:

Replay the relevant block(s)
Compute the fine-grained rewards for each attestation/sync aggregate/slashing processed. The current approach for sync aggregates used by the block_rewards API should suffice. For attestations we could improve on the current block_rewards by diffing the ParticipationFlags to accurately track included attestations (see below). Slashings require more thought.

Attestation Rewards

I think attestation rewards are a little easier than block rewards. We can compute them for a given epoch by diffing the current_epoch_participation which contains a ParticipationFlags bitfield for each validator. For each bit set (or not set) we can then apply the appropriate reward/penalty based on get_base_reward and the inactivity penalty. See:

https://github.com/sigp/lighthouse/blob/fcfd02aeec435203269b03865e3ccc23e5f51e6d/consensus/state_processing/src/per_epoch_processing/altair/rewards_and_penalties.rs#L75-L90

We may not even want a block replayer for this, we might just want to load the state at the start of epoch n + 1 and diff it against the state at epoch n.

Sync Committee Rewards

Require more thought, TODO.

michaelsproul commented 2 years ago

CC @kevinbogner who is interested in working on this. Feel free to post questions/findings on this issue. Or hit me up on Discord.

Some related reading which might be useful:

The spec: https://github.com/ethereum/consensus-specs/blob/dev/specs/altair/beacon-chain.md
Ben Edginton's book: https://eth2book.info/
Pintail's guide to post-Altair rewards: https://pintail.xyz/posts/modelling-the-impact-of-altair/

ensi321 commented 2 years ago

After getting inputs from @michaelsproul and @potuz, @kevinbogner and I are attempting to answer some of the design questions so we can push forward the discussion: DQ1: I think the reward API should operate on a single block/epoch for the sake of consistency with the rest of the Beacon API. Operating on a range may be more efficient on Lighthouse side but it might not be the case for other clients. DQ2.1: For granularity of the data returned from the api, I think a breakdown of source/target/head vote reward is a yes. And I believe the technical difficulty to implement each of the three should be similar. DQ2.2: For block reward data (ie. sync committee and proposer) it is desirable to provide such breakdown. From Beacon API’s perspective, I think including the block reward is ideal for the sake of completeness. However if we believe such data is not too useful that it’s not worth the development effort we can forget about it

michaelsproul commented 1 year ago

Perfect, I agree with those decisions. If you (@naviechan) and @kevinbogner can put together a draft PR to the beacon-APIs repo then I can review it and we can start implementing! Let me know if you need any assistance.

Thanks!

jimmygchen commented 1 year ago

Last piece of remaining work here: https://github.com/sigp/lighthouse/issues/4512

sigp / lighthouse