Optimize disk space for persistence

gnulib commented 5 years ago

Current disk space usage for 10k transactions is about 225MB (25k / tx). This seems expensive for world scale.

Run experiments to profile disk space usage and find ways to optimize disk space.

gnulib commented 5 years ago

Profiling data

Baseline (no transactions)

 16K    spendr-local//dlt_shard_tips
 12K    spendr-local//dlt_submitter_history
 16K    spendr-local//dlt_transactions
 12K    spendr-local//Shard-Seen-Tx-test-driver-for-double-spending
 12K    spendr-local//Shard-World-State-test-driver-for-double-spending
 16K    spendr-local//dlt_shard_dags
 84K    spendr-local/

1 Transaction

 20K    spendr-local//dlt_shard_tips
 16K    spendr-local//dlt_submitter_history
 20K    spendr-local//dlt_transactions
 16K    spendr-local//Shard-Seen-Tx-test-driver-for-double-spending
 16K    spendr-local//Shard-World-State-test-driver-for-double-spending
 24K    spendr-local//dlt_shard_dags
112K    spendr-local/

100 Transactions

 72K    spendr-local//dlt_shard_tips
492K    spendr-local//dlt_submitter_history
492K    spendr-local//dlt_transactions
488K    spendr-local//Shard-Seen-Tx-test-driver-for-double-spending
460K    spendr-local//Shard-World-State-test-driver-for-double-spending
676K    spendr-local//dlt_shard_dags
2.6M    spendr-local/

1000 Transactions

656K    spendr-local//dlt_shard_tips
4.8M    spendr-local//dlt_submitter_history
4.8M    spendr-local//dlt_transactions
4.8M    spendr-local//Shard-Seen-Tx-test-driver-for-double-spending
4.5M    spendr-local//Shard-World-State-test-driver-for-double-spending
6.7M    spendr-local//dlt_shard_dags
 26M    spendr-local/

10000 Transactions

7.1M    spendr-local//dlt_shard_tips
 49M    spendr-local//dlt_submitter_history
 48M    spendr-local//dlt_transactions
 48M    spendr-local//Shard-Seen-Tx-test-driver-for-double-spending
 46M    spendr-local//Shard-World-State-test-driver-for-double-spending
 59M    spendr-local//dlt_shard_dags
257M    spendr-local/

gnulib commented 5 years ago

Analysis

Comparing with Ethereum

Based on above profiling data, DLT stack takes about 25k / tx (each transaction is 1 block).
In comparison, a full node on ethereum seems to take aprox 45k / block (each block is about 150 tx).

Therefore, disk space usage of DLT stack seems to be same order as Ethereum.

Benefit of sharding

A big chunk of disk space is used up by sharding layer's world state and seen transaction DB. However, in DLT stack architecture a node only processes transactions that are specific to registered shard. Therefore, unlike Ethereum, a node in DLT stack network will not require to process all transactions, thereby significantly reducing the disk space (and processing) for non registered shards in the network.

As an experiment, a "headless" node on the network connected to same "spendr" node above after completing the sync of 10k transactions used up following disk space:

7.1M    countr//dlt_shard_tips
 49M    countr//dlt_submitter_history
 48M    countr//dlt_transactions
 59M    countr//dlt_shard_dags
163M    countr/

Notice that "world state" and "seen tx" namespace are missing, and hence total disk space consumption is only 163M (saving 35%).

gnulib commented 5 years ago

Individual Object Sizes

DagNode

292 + 64 * children count

Transaction

650 + opcode payload length (~ 1024+)

SubmitterHistory

~ 256

WorldState Resource

variable, ~ 1024 + 1024 (key + val)

SeenTx

~ 128

trust-net / dag-lib-go