VM performance meta-issue

[jochem-brouwer]

This meta-issue is to write down (and update) findings in the VM runs so we don't keep doing the same research over and over.

I will write down first my tests on devnet12. This tested: Profiler run - Block 50379 (0x9d3a6d0314055dcec03fa617d388659fdfe0ce2e3fc9df5f6c48db830f6f7c9f with 203 txs. The gas used was 22908541 (22.9M+).

If we use the VM profiler we get this output:

Profiler run - Block 50379 (0x9d3a6d0314055dcec03fa617d388659fdfe0ce2e3fc9df5f6c48db830f6f7c9f with 203 txs
New state root, DAO HF, checkpoints, block validation: 782.939ms
Tx processing [ use per-tx profiler for more details ]: 3.679s
Withdrawals, Rewards, EVM journal commit: 1.121s
Entire block: 5.587s

The findings were that for New state root, DAO HF, checkpoints, block validation, almost all time is spent on ecrecover of the block txs. (About 95%). For Withdrawals, Rewards, EVM journal commit, almost all time is spent on this.evm.journal.commit() https://github.com/ethereumjs/ethereumjs-monorepo/blob/7bcb197f2a4388b7c48ab6776b31733308ac0933/packages/vm/src/runBlock.ts#L185.

In there, all state manager caches are written to disk. This is almost entirely covered by Trie operations, which is called in StateManager.flush. In there, most of the time is spent in lookupNode (in Trie) and Trie._formatNode. The latter consists mainly of serializing (RLP.encode) and hashing (keccak256). Both these methods take up about 100% of the time (split about 1/4 for RLP.encode and 3/4 for hashing).

The time in lookupNode is spent mostly on actual disk reads. (About 55% of the total time). (https://github.com/ethereumjs/ethereumjs-monorepo/blob/7bcb197f2a4388b7c48ab6776b31733308ac0933/packages/trie/src/db/checkpoint.ts#L157)

This gives raise to the following optimization routes which takes upon at least 80% of the trie flush time:

• keccak256
• RLP.encode
• DB disk lookup time

[paulmillr]

Use & benchmark package unrolled-nbl-hashes-sha3 to speed-up keccak256 before doing any big changes. It's fast enough.