ethereum / trin

An Ethereum portal client: a json-rpc server with nearly instant sync, and low CPU & storage usage
365 stars 113 forks source link

refactor: update pruning strategy #1288

Closed morph-dev closed 4 months ago

morph-dev commented 4 months ago

What was wrong?

The pruning strategy of IdIndexedV1Store (deleting ~5% of db) was taking too long on large databases (example: 40sec on 10GB db).

How was it fixed?

After trying several approaches and measuring performance, I refactored the pruning logic:

  1. Don't use triggers and update usage stats manually
    • Made delete queries return content_size to save few sql queries
  2. Because of 1. no need for estimating how many entries to insert before pruning
    • This also allowed removing pruning_threshold from the config
  3. Prune up to 5%, but no more than 100 items at the time

I run this version on one of Kolby's machines and metrics look ok to me.

For future PR-s:

To-Do