aptos-labs / aptos-core

Aptos is a layer 1 blockchain built to support the widespread use of blockchain through better technology and user experience.
https://aptosfoundation.org
Other
5.86k stars 3.54k forks source link

Add max-tps option to aptos node analyze-validator-performance cli #13290

Open igor-aptos opened 2 weeks ago

igor-aptos commented 2 weeks ago

cargo run -p aptos -- node analyze-validator-performance --analyze-mode max-tps

Description

Type of Change

Which Components or Systems Does This Change Impact?

How Has This Been Tested?

Key Areas to Review

Checklist

trunk-io[bot] commented 2 weeks ago
⏱️ 1h 30m total CI duration on this PR | Job | Cumulative Duration | Recent Runs | |---|---|---| | [rust-targeted-unit-tests](https://github.com/aptos-labs/aptos-core/actions/runs/9137582374/job/25127740852) | 31m | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9099563562/job/25012404986) [🟥](https://github.com/aptos-labs/aptos-core/actions/runs/9131952245/job/25112273527) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9137582374/job/25127740852)  | | [rust-lints](https://github.com/aptos-labs/aptos-core/actions/runs/9137582374/job/25127740758) | 21m | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9099563562/job/25012404441) [🟥](https://github.com/aptos-labs/aptos-core/actions/runs/9131952245/job/25112272944) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9137582374/job/25127740758)  | | [rust-move-tests](https://github.com/aptos-labs/aptos-core/actions/runs/9137582373/job/25127738055) | 14m | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9099563556/job/25012390282) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9131952360/job/25112261607) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9137582373/job/25127738055)  | | [run-tests-main-branch](https://github.com/aptos-labs/aptos-core/actions/runs/9137582221/job/25127742129) | 12m | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9099563539/job/25012412866) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9131952272/job/25112277411) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9137582221/job/25127742129)  | | [general-lints](https://github.com/aptos-labs/aptos-core/actions/runs/9137582374/job/25127740807) | 5m | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9099563562/job/25012403585) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9131952245/job/25112273290) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9137582374/job/25127740807)  | | [check-dynamic-deps](https://github.com/aptos-labs/aptos-core/actions/runs/9137582378/job/25127737739) | 4m | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9099563552/job/25012389145) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9131952257/job/25112258842) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9137582378/job/25127737739)  | | [semgrep/ci](https://github.com/aptos-labs/aptos-core/actions/runs/9137582371/job/25127737723) | 1m | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9099563545/job/25012389120) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9131952248/job/25112259055) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9137582371/job/25127737723)  | | [file_change_determinator](https://github.com/aptos-labs/aptos-core/actions/runs/9137582374/job/25127738005) | 33s | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9099563562/job/25012390090) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9131952245/job/25112260486) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9137582374/job/25127738005)  | | [file_change_determinator](https://github.com/aptos-labs/aptos-core/actions/runs/9137582221/job/25127739975) | 32s | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9099563539/job/25012398324) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9131952272/job/25112267110) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9137582221/job/25127739975)  | | [permission-check](https://github.com/aptos-labs/aptos-core/actions/runs/9137582225/job/25127737557) | 10s | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9099563538/job/25012390072) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9131952271/job/25112260123) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9137582225/job/25127737557)  | | [permission-check](https://github.com/aptos-labs/aptos-core/actions/runs/9137582221/job/25127737585) | 8s | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9099563539/job/25012390194) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9131952272/job/25112260317) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9137582221/job/25127737585)  | | [permission-check](https://github.com/aptos-labs/aptos-core/actions/runs/9137582218/job/25127737443) | 8s | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9099563558/job/25012389407) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9131952267/job/25112258949) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9137582218/job/25127737443)  | | [permission-check](https://github.com/aptos-labs/aptos-core/actions/runs/9137582224/job/25127737450) | 7s | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9099563542/job/25012389253) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9131952332/job/25112260892) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9137582224/job/25127737450)  |

🚨 2 jobs on the last run were significantly faster/slower than expected

Job Duration vs 7d avg Delta
rust-targeted-unit-tests 10m 19m -47%
rust-move-tests 3m 11m -71%

settingsfeedbackdocs ⋅ learn more about trunk.io

igor-aptos commented 1 week ago

ping @sitalkedia , @bchocho for reviews here

github-actions[bot] commented 1 day ago

Forge is running suite compat on a68e71c05caebf01504d4499110f3fba213fb53d ==> 1fb3a233f6f19b1779776d031f9b770d28c9e87b

github-actions[bot] commented 1 day ago

Forge is running suite framework_upgrade on a68e71c05caebf01504d4499110f3fba213fb53d ==> 1fb3a233f6f19b1779776d031f9b770d28c9e87b

github-actions[bot] commented 1 day ago

Forge is running suite realistic_env_max_load on 1fb3a233f6f19b1779776d031f9b770d28c9e87b

github-actions[bot] commented 1 day ago

:white_check_mark: Forge suite compat success on a68e71c05caebf01504d4499110f3fba213fb53d ==> 1fb3a233f6f19b1779776d031f9b770d28c9e87b

Compatibility test results for a68e71c05caebf01504d4499110f3fba213fb53d ==> 1fb3a233f6f19b1779776d031f9b770d28c9e87b (PR)
1. Check liveness of validators at old version: a68e71c05caebf01504d4499110f3fba213fb53d
compatibility::simple-validator-upgrade::liveness-check : committed: 9904.06662973203 txn/s, latency: 3289.705746325025 ms, (p50: 2400 ms, p90: 5700 ms, p99: 18500 ms), latency samples: 344220
2. Upgrading first Validator to new version: 1fb3a233f6f19b1779776d031f9b770d28c9e87b
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 3417.5354836576375 txn/s, latency: 9069.24196454519 ms, (p50: 9600 ms, p90: 13700 ms, p99: 14200 ms), latency samples: 137640
3. Upgrading rest of first batch to new version: 1fb3a233f6f19b1779776d031f9b770d28c9e87b
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 2761.0558698995283 txn/s, latency: 11217.334921494315 ms, (p50: 11500 ms, p90: 16200 ms, p99: 17800 ms), latency samples: 110820
4. upgrading second batch to new version: 1fb3a233f6f19b1779776d031f9b770d28c9e87b
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 6388.715412427143 txn/s, latency: 5122.437016504444 ms, (p50: 4800 ms, p90: 8400 ms, p99: 9500 ms), latency samples: 236300
5. check swarm health
Compatibility test for a68e71c05caebf01504d4499110f3fba213fb53d ==> 1fb3a233f6f19b1779776d031f9b770d28c9e87b passed
Test Ok
github-actions[bot] commented 1 day ago

:white_check_mark: Forge suite realistic_env_max_load success on 1fb3a233f6f19b1779776d031f9b770d28c9e87b

two traffics test: inner traffic : committed: 8329.142411980203 txn/s, latency: 4707.155136879591 ms, (p50: 4500 ms, p90: 5700 ms, p99: 9800 ms), latency samples: 3594400
two traffics test : committed: 99.98891797948832 txn/s, latency: 1804.4066666666668 ms, (p50: 1800 ms, p90: 2000 ms, p99: 2800 ms), latency samples: 1800
Latency breakdown for phase 0: ["QsBatchToPos: max: 0.218, avg: 0.205", "QsPosToProposal: max: 0.228, avg: 0.218", "ConsensusProposalToOrdered: max: 0.442, avg: 0.387", "ConsensusOrderedToCommit: max: 0.378, avg: 0.360", "ConsensusProposalToCommit: max: 0.766, avg: 0.747"]
Max round gap was 1 [limit 4] at version 1767915. Max no progress secs was 5.126108 [limit 15] at version 1767915.
Test Ok
github-actions[bot] commented 1 day ago

:white_check_mark: Forge suite framework_upgrade success on a68e71c05caebf01504d4499110f3fba213fb53d ==> 1fb3a233f6f19b1779776d031f9b770d28c9e87b

Compatibility test results for a68e71c05caebf01504d4499110f3fba213fb53d ==> 1fb3a233f6f19b1779776d031f9b770d28c9e87b (PR)
Upgrade the nodes to version: 1fb3a233f6f19b1779776d031f9b770d28c9e87b
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1202.311516846706 txn/s, submitted: 1204.42412982047 txn/s, failed submission: 2.1126129737642234 txn/s, expired: 2.1126129737642234 txn/s, latency: 2637.5991019133153 ms, (p50: 2100 ms, p90: 4200 ms, p99: 8700 ms), latency samples: 102440
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1103.0870659650632 txn/s, submitted: 1106.3561639590325 txn/s, failed submission: 3.2690979939692815 txn/s, expired: 3.2690979939692815 txn/s, latency: 2859.7959356477563 ms, (p50: 2100 ms, p90: 4900 ms, p99: 9900 ms), latency samples: 94480
5. check swarm health
Compatibility test for a68e71c05caebf01504d4499110f3fba213fb53d ==> 1fb3a233f6f19b1779776d031f9b770d28c9e87b passed
Upgrade the remaining nodes to version: 1fb3a233f6f19b1779776d031f9b770d28c9e87b
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1077.5184939061771 txn/s, submitted: 1080.611637009455 txn/s, failed submission: 3.093143103277933 txn/s, expired: 3.093143103277933 txn/s, latency: 2702.696883329916 ms, (p50: 2100 ms, p90: 4400 ms, p99: 9700 ms), latency samples: 97540
Test Ok