status-im / nimbus-eth1

Nimbus: an Ethereum Execution Client for Resource-Restricted Devices
https://status-im.github.io/nimbus-eth1/
Apache License 2.0
571 stars 115 forks source link

Adjust contentdb to be able to keep track of headers without a proof for later purging #2207

Open kdeme opened 4 months ago

kdeme commented 4 months ago

Currently Headers are stored with proof before merge and without after. Eventually we will also add proofs after merge, but there will always be the most recent headers that are not yet part of the BeaconState.historical_summaries.

Now the issue arises when these headers need to be updated by headers with their proof. The client needs to be able to recognize when a new offer comes in but for a header which was stored without proof, so that it can accept that new offer and overwrite the header without the proof (there is also the issue that node can not know upfront whether the offered header comes with or without proof).

Currently before accepting data, a Radius check + a p.dbGet is done: https://github.com/status-im/nimbus-eth1/blob/master/fluffy/network/wire/portal_protocol.nim#L442

The dbGet is already a handler that can get customized, as is currently the case for contentDb versus beaconDb. So this would require a custom contentDb for history network, or historyDb (or at least layer above the contentDb).

Some ideas to keep track of this could be:

What could however also work is a specification change:

Related specs issue: https://github.com/ethereum/portal-network-specs/issues/239

kdeme commented 3 weeks ago
  • Define two block header types in the network, one with and one without the proofs. When the one with proof gets offered, check if the one without is stored and purge it after receiving the new one.

Note that this type of solution would still have a problem in the case that certain block headers never get offered with their proof. The headers without the proof would remain in the database with no easy way to find them. In the current scheme, it would be needed to iterate over all content, decode and check the headers that are older than a certain timestamp or perhaps block number. So this does not really solve the issue.