Open kocubinski opened 8 months ago
We talked on the team call about supporting import for preorder and post order to enable usage of iavlv2 in existing networks, but for new networks it would only be an export of preorder to simplify the code in the long run.
This issue tracks IAVL v2's SQLite backed implementation, including briefs of milestones hit (past), in progress work (present), overall strategy and near-term pending items (future).
main dev branch: https://github.com/cosmos/iavl/tree/v2-alpha6
Benchmarks and Tests
Benchmarks demonstrating the performance of IAVL v2 are implemented, the steps below will reproduce them. The dataset used is specified as OsmoLikeMany, which approximates the state of osmosis's application.db in mid Sept 2023.
Similar benchmark tests against IAVL v0 and IAVL v1 (see https://github.com/cosmos/iavl-bench) were run, throughput is ~60x faster than v0 and 12x v1.
TestTree_Hash validates the root hash consistency with iavl v0 in a mode closer to historical mode (see below).
General
SaveVersion
andLoadSnapshot
SaveVersion
if this failed. MemIAVL has shown that this strategy works very well, since the state machine will take much longer to process block n+1 than time spent in commit of block n.Latest Mode
Latest mode is the fastest and most memory intensive operating of mode of IAVL v2. In this mode either the entire tree is stored in a memory, or all branch nodes (leaves excluded).
Historical Mode
Historical mode approximates current behavior in IAVL v0 and v1 where all branch and leaf nodes are saved to disk at every version. Primary use cases are nodes supporting historical proofs or full archive nodes.
Snapshot
Pruning
Discussions in the store v2 working group are split on the efficiency of both approaches below. Orphan tracking is more time efficient and less space efficient, a tree diff is more space efficient but less time efficient. Benchmarks comparing both are needed.
Migration
Osmosis integration
There is no better test than mainnet integration. To support adoption and highlight any issues in IAVL v2 a PoC of integration into an osmosis node is in progress so that mainnet benchmarks may be obtained. This will inform current and future development.
Osmosis forked Cosmos SDK with IAVL v2 patch: https://github.com/cosmos/cosmos-sdk/tree/kocubinski/osmosis-iavl-v2 Osmosis with the above patch: https://github.com/01builders/osmosis/tree/kocubinski/iavl-v2
Status
mainnet is syncing but panics at snapshot (not yet implemented).
TODO
SaveVersion
concurrentlytree.Metrics
and log set/get/delete metrics per tree