feat: implement dynamic pruning strategy

[morph-dev]

What was wrong?

Current pruning logic for IdIndexedV1Store is to delete up to 5% of storage, but no more than 100 elements. However, even 100 elements is too slow on big databases (e.g. 35GB that are currently used). It can take up to a second and sometimes even longer. This wouldn't be such a big problem if this wouldn't block other read/writes.

How was it fixed?

The idea is to have dynamic number of items that we delete. The PruningConfig allows us to set parameters, for this dynamic strategy:

• what is the optimal pruning duration - default 100-300 milliseconds
• by how much to increase/decrease number of items to prune when pruning is faster/slower than optimal - default 20%

Also added logic for handling the zero storage capacity to prune the db (if non-empty) and set radius to zero.

To-Do

• [x] Clean up commit history and use conventional commits.

[njgheorghita]

It seems like this might be relevant... Taking into account #1228 and that (IMO) we will sooner rather than later want to support this, along with a "max db" size (aka a user has 100 gbs of disk space they want to contribute, we'll actually spin up 10 nodes of 10gbs under the hood). Once that's implemented, we'll have a consistent max db size, which means we really don't have to worry about improving the pruning mechanism to support large dbs and can optimize db performance against a fixed target (eg a 10gb storage). Assuming that my description is the direction we're heading, does this pr still make sense?

[pipermerriam]

we'll actually spin up 10 nodes of 10gbs under the hood)

I don't think this is how the architecture will actually work out. I think that under the hood it will be a single 100gb database that is represented in the network across 10 different node-ids.

See: https://github.com/ethereum/portal-network-specs/issues/283#issuecomment-2037869161

[morph-dev]

Together with what Piper said, I would add that there is benefit for smaller dbs as well (as we can purge more there).

In general, I would say it's better to monitor and adjust in any scenario. For example:

• target of 10GB will perform differently on different devices (e.g. SSD vs HD)
• even if you have 1 node of 10GB (i.e. our fixed target), you might also run execution+consensus client and your disk usage is already high

[morph-dev]

@carver I do disagree with the proposed changes about pruning number changes, but not strongly :). Willing to discuss it more before merging.

I'm going to merge this and do a followup PR if needed.