Get sign off by system level benchmarks

[dnadales]

• [x] Prepare a branch for the benchmarking team
• [ ] Any regressions shown by the system-level benchmarks should be within the allowed limits, both for the
  • [ ] in-memory backend.
  • [ ] LMDB backend.
• [ ] Get a signoff from product owners and other stakeholders (@abailly-iohk, @disassembler, @kevinhammond)

If the results of the benchmarks are not satisfactory, we might need to work on further optimizations.

[dnadales]

utxo-hd-8.2.1 vs 8.2.1-pre, in-memory, value-only

Results for: utxo-hd-8.2.1 (optimized and unoptimized) vs. 8.2.1-pre -- AWS benchmarks value-only workload

Resource usage

1. Heap size increased by 1072MB or 57%.
2. Allocation rate went up by 47%.
3. Mutator CPU usage increased by 20% and process CPU usage by 24%.
4. GC CPU usage went up by 45%.
5. Minor GCs increased by 50%, whereas Major GCs went down by 46%.
6. CPU 85% span duration improved by 4.9 slots, or 37%.

Forging

For utxo-hd with the in-memory backend, we observe no regression in forger-related metrics.

Peer propagation

For utxo-hd with the in-memory backend, we observe no regression in peer-related metrics.

End-to-end propagation

Block propagation times for utxo-hd with the in-memory backend match those of the 8.2.1-pre baseline for all centiles.

[dnadales]

utxo-hd-8.2.1 vs 8.2.1-pre, in-memory, Plutus workload

Resource usage

1. Heap size increased by 696 MB, or 36%.
2. Allocation rate went up by 19%.
3. Minor GCs increased by 20%, whereas Major GCs decreased by 51%.
4. CPU span duration worsened by 1.9 slots, or 13%.

Forging

For utxo-hd with the in-memory backend, we observe no regression in forger-related metrics.

Peer propagation

For utxo-hd with the in-memory backend, we observe no regression in peer-related metrics.

End-to-end propagation

Block propagation times for utxo-hd with the in-memory backend are on par or single-digit percentages less than the 8.2.1-pre baseline version.

[dnadales]

Due to the increase in resource usage, we will not be able to sign off the current version of the in-memory backend. This issue might be superseded by https://github.com/input-output-hk/ouroboros-consensus/issues/429