kaspanet / rusty-kaspa

Kaspa full-node and related libraries in the Rust programming language. This is a stable version at the initial rollout phases.
ISC License
359 stars 109 forks source link

Fine-tune DB parameters for block transaction storage #398

Open coderofstuff opened 4 months ago

coderofstuff commented 4 months ago

Tracking for: https://discord.com/channels/599153230659846165/917551031259377664/1196165330985308170

Problem Statement

Current tn11 nodes perform around 10x write more than actual disk size, and block txs are by far the most significant part of this storage

Look into storing block transactions in rocksdb with special settings that will prevent writing amplification.

Dev Notes

impl ConnBuilder<PathBuf, false, Unspecified, i32> {
    pub fn build(self) -> Result<Arc<DB>, kaspa_utils::fd_budget::Error> {
        let (opts, guard) = default_opts!(self)?;

        // These look like universal compaction settings. We should start with checking out these. If they help
        // to reduce writing amplification but degrade performance, then we should look at applying settings on
        // the column family level. See below 
        rocksdb::UniversalCompactOptions:: ...
        opts.set_universal_compaction_options(uco)

        let db = Arc::new(DB::new(<DBWithThreadMode<MultiThreaded>>::open(&opts, self.db_path.to_str().unwrap()).unwrap(), guard));

        // These look like settings specific to a column family. If the above universal ones are too aggressive, we can check this out. 
        // However note that switching a store such as block txs to a column family might require some infra-level code changes, for instance
        // for figuring out how does one make a batch write combined with a column family store, as well as writing DB upgrade logic etc
        // (upper-level store wrapping old and new store and falling back to old if new is missing the entry)
        rocksdb::CompactOptions:: ...
        db.compact_range_cf_opt(cf, start, end, opts)

        Ok(db)
    }
}