Allow pruning "archive" node

Consensys / teku

Open-source Ethereum consensus client written in Java

https://consensys.io/teku

Apache License 2.0

648 stars 264 forks source link

Allow pruning "archive" node #8331

Open tbenr opened 1 month ago

tbenr commented 1 month ago

In some scenarios, users could be interested in running archive nodes (with finalized states) up to a certain time in the past. This could enable having a low value in --data-storage-archive-frequency, so the node can be very responsive within the interested time interval.

To achieve this we could add a --data-storage-archive-retained-epochs (or similar). Setting value to 1024, teku will prune finalized states older than 1024 epochs compared to the current epoch.

A node running this way won't be a strict archive node, kind of a "time limited archive node".

Note: We should not allow state reconstruction when running in this mode.

zilm13 commented 1 month ago

It would be good if we are not trying state reconstruction too far from archive state. Say, we have states since 9,000,000 slot and user is requesting state from slot 8,999,000. It would be good to give up immediately with 404 or whatever without trying to reconstruct it from genesis. Not sure if it works already like this. Is it?

gfukushima commented 3 weeks ago

Currently working on this with the following approach:

[x] Adding interfaces to the db to allow deletion to the columns where we store the finalized states.
[x] Add a state pruner using similar approach used by the block pruner (limiting pruning done in each run)
[x] Pruner runs exclusively for archive mode when the --data-storage-archive-retained-epochs is set.
[ ] Prevent state reconstruction when state pruning is enabled(?)
[ ] Validate what the current behaviour is for a request that is prior the earliest finalized state present in the db