search

iotaledger / iota-core

Apache License 2.0
60 stars 12 forks source link

Snapshots at epoch boundaries #300

Open jonastheis opened 1 year ago

jonastheis commented 1 year ago

With #288 pruning and storage granularity is aligned to epoch boundaries. However, snapshots are currently still more fine-grained based on slots. This is not necessarily a problem, as it allows more fine-grained export/start of a node without needing to sync from the start of an epoch to the current time.

However, it creates an oddity: If we start a node from a snapshot within an epoch, that epoch is neither fully available on the node nor can it be marked as pruned. For example, let's assume we start a node at slot 12 (epoch=8 slots). Then we don't have slots before 12 available but in the storage we can't say epoch 2 is pruned either.

For engine switching and copying over data from the old to the new engine it is actually beneficial to have slot granularity snapshots available. That way we can simply start from the snapshot at (1 slot before) the forking point, copy over all the history data and delete slots >= forking point from the DB.

### Tasks
- [ ] https://github.com/iotaledger/iota-core/issues/313
- [ ] Decide what to do about snapshots for nodes that are joining new
jonastheis commented 1 year ago

Ideally, snapshots at the same boundary should be exactly the same byte by byte. That way, we can simply compare snapshots` hash and determine whether they are the same.

I started to adjust for this in https://github.com/iotaledger/iota-core/commit/ca91c3ebcafd96abebb51299eedced1a258356a3