[Execution State] Memory usage and performance optimisation

[ramtinms]

This Epic captures some of the improvements to the execution state code (also known as Ledger) in terms of reducing memory and improving the speed of operations and proof verifications.

• [x] https://github.com/onflow/atree/issues/292
• [ ] #1746
• [ ] #1749
• [ ] #1745
• [x] #2559
• [ ] https://github.com/onflow/cadence/issues/1402
• [x] https://github.com/onflow/cadence/issues/1854

[fxamacker]

Atree Inlining & Deduplication (Testnet Results)

Before/After Comparison Using Aug 14, 2024 Devnet State

Mtrie Nodes and Atree Payloads

Atree Payload Sizes (bytes)

#### Total Counts | | Before | After | Change | | :---: | ---: | ---: | ---: | | mtrie nodes | 1,182,195,945 | 343,949,052 | -838,246,893 | | atree payloads | 455,399,623 | 102,283,931 | -353,115,692 | #### Total Sizes | | Before | After | Change | | :---: | ---: | ---: | ---: | | mtrie nodes | 113.49 GB | 33.02 GB | -80.47 GB | | atree payloads | 87.74 GB | 69.28 GB | -18.46 GB | | combined | 201.23 GB | 102.30 GB | -98.93 GB |

| | Before | After | | :---: | ---: | ---: | | count | 455,399,623 | 102,283,931 | | mean | 192.7 | 677.4 | | std | 996.4 | 2,065.7 | | min | 13 | 22 | | 25% | 88 | 199 | | 50% | 139 | 665 | | 75% | 206 | 1,043 | | 99% | 876 | 1,267 | | 99.5% | 1,078 | 1,453 | | max | 5,853,892 | 5,853,892 |

NOTE: Testnet and mainnet results can differ, depending on usage, data, etc. Although testnet memory use was reduced by 1/2 on Aug 14 data, mainnet memory reduction (percentage wise) will likely be more modest.

See atree epic 292 for more details & charts:

• https://github.com/onflow/atree/issues/292
• https://github.com/onflow/atree/releases/tag/v0.8.0-rc.5