search

rooch-network / rooch

VApp Container with Move Language
https://rooch.network
Apache License 2.0
129 stars 56 forks source link

feat(rooch.statedb): improve import utxo performance by adjusting rocksdb and parallelism #1662

Closed popcnt1 closed 3 weeks ago

popcnt1 commented 3 weeks ago

Summary

  1. utxo importer:
    1. two thread model, one produce update set batch, one apply updates to smt and state db.
    2. print more detailed info
  2. rocksdb config:
    1. 1KiB as default block size: 12% improvement may caused by faster block read (filter and index block cache work well with 1KiB block size)
    2. 512MB memtable for buffering enough writes. Increase WAL size and L1 size according to the L0 size changes caused by larger memtable
    3. in pratice, we should use more than 16GiB block cache size
    4. smaller row cache size, because the hit rate is so low in random access I/O model
    5. enable stats by default, because no significant degradation. maybe 1-2%
    6. disable compression on L0 & L1

it's a good start for rocksdb optimization, because it's the first time to have enough data

vercel[bot] commented 3 weeks ago

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
rooch ✅ Ready (Inspect) Visit Preview 💬 Add feedback May 9, 2024 0:20am
rooch-rooch-portal-v1 ✅ Ready (Inspect) Visit Preview 💬 Add feedback May 9, 2024 0:20am