Add Block Range Merkle Map cache pool in Cardano Transactions prover

[jpraynaud]

Content

This PR includes the implementation of a cache pool of the Block Range Merkle Maps to improve the Cardano transactions prover performances:

• Implement a resource pool for block range Merkle map:
  • Resources automatically returned to the pool
  • Stale resource detection on resource returned to the pool
  • Non blocking refresh of the resources in the pool
• Warm up the resource pool at aggregator node startup
• Refresh the resource pool of block range Merkle maps when a new Cardano transaction artifact is produced
• Use the resource pool as cache for block range Merkle map in the prover
• Make the size of the prover cache pool configurable
• Implement the configuration of the prover cache pool size in the infrastructure
• Implement the configuration of the prover cache pool size in the CI/CD

Benchmarks

Total Requests	Concurrency	Transactions / Request	Pool (x1): Request/s	Pool (x5): Request/s	Pool (x10): Request/s	Pool (x25): Request/s	Pool (x50): Request/s
1000	100	1	39	156	213	229	237
1000	100	2	28	110	152	169	171
1000	100	3	21	83	116	129	128
1000	100	4	18	70	96	104	105
1000	100	5	15	59	82	87	86
1000	100	6	0	0	0	0	0
1000	100	7	0	0	0	0	0
1000	100	8	0	0	0	0	0
1000	100	9	0	0	0	0	0
1000	100	10	0	0	0	0	0

Running on Linux / 8 cores / 64 GB RAM / 2 TB SSD

Memory impact

~240MB / new pool resource

Pooled Resources	Memory / GB
1	0
5	1
10	3
25	6
50	13
100	25

Conclusions

The performances are considerably improved with the implementation of resource pooling for Block Ranges Merkle maps. We recommend to use a size of the pool with a value between 25 and 50 which gives good trade-off between memory usage and performances gain. This value is subject to change as we suspect that other factors are limiting the throughput which drastically drops with more than 5 transactions to certifiy per request.

Pre-submit checklist

• Branch
  • [x] Tests are provided (if possible)
  • [x] Crates versions are updated (if relevant)
  • [x] CHANGELOG file is updated (if relevant)
  • [x] Commit sequence broadly makes sense
  • [x] Key commits have useful messages
• PR
  • [x] No clippy warnings in the CI
  • [x] Self-reviewed the diff
  • [x] Useful pull request description
  • [x] Reviewer requested

Issue(s)

Closes #1698

[github-actions[bot]]

Test Results

    3 files  ±0     43 suites  ±0   8m 29s :stopwatch: -1s 1 003 tests +9  1 003 :white_check_mark: +9  0 :zzz: ±0  0 :x: ±0  1 101 runs  +9  1 101 :white_check_mark: +9  0 :zzz: ±0  0 :x: ±0 

Results for commit be085ae0. ± Comparison against base commit 37d0390a.

:recycle: This comment has been updated with latest results.