search

aptos-labs / aptos-core

Aptos is a layer 1 blockchain built to support the widespread use of blockchain through better technology and user experience.
https://aptosfoundation.org
Other
5.86k stars 3.54k forks source link

fix flaky rounding tests #13293

Closed zjma closed 2 weeks ago

zjma commented 2 weeks ago

Description

Type of Change

Which Components or Systems Does This Change Impact?

How Has This Been Tested?

Key Areas to Review

Checklist

trunk-io[bot] commented 2 weeks ago
⏱️ 4h 58m total CI duration on this PR | Job | Cumulative Duration | Recent Runs | |---|---|---| | [forge-framework-upgrade-test / forge](https://github.com/aptos-labs/aptos-core/actions/runs/9103739818/job/25026575575) | 1h 12m | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9103739818/job/25026575575)  | | [windows-build](https://github.com/aptos-labs/aptos-core/actions/runs/9103739758/job/25026230235) | 39m | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9103739758/job/25026230235)  | | [rust-smoke-tests](https://github.com/aptos-labs/aptos-core/actions/runs/9103739759/job/25026244232) | 31m | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9103739759/job/25026244232)  | | [rust-targeted-unit-tests](https://github.com/aptos-labs/aptos-core/actions/runs/9103739759/job/25026244379) | 28m | [⬜](https://github.com/aptos-labs/aptos-core/actions/runs/9103691343/job/25026081456) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9103739759/job/25026244379)  | | [execution-performance / single-node-performance](https://github.com/aptos-labs/aptos-core/actions/runs/9103739763/job/25026325490) | 23m | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9103739763/job/25026325490)  | | [rust-move-tests](https://github.com/aptos-labs/aptos-core/actions/runs/9103739762/job/25026237313) | 19m | [⬜](https://github.com/aptos-labs/aptos-core/actions/runs/9103691347/job/25026073599) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9103739762/job/25026237313)  | | [forge-e2e-test / forge](https://github.com/aptos-labs/aptos-core/actions/runs/9103739818/job/25026575469) | 14m | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9103739818/job/25026575469)  | | [forge-compat-test / forge](https://github.com/aptos-labs/aptos-core/actions/runs/9103739818/job/25026575361) | 12m | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9103739818/job/25026575361)  | | [rust-images / rust-all](https://github.com/aptos-labs/aptos-core/actions/runs/9103739818/job/25026238434) | 12m | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9103739818/job/25026238434)  | | [rust-lints](https://github.com/aptos-labs/aptos-core/actions/runs/9103739759/job/25026243578) | 11m | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9103691343/job/25026081050) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9103739759/job/25026243578)  | | [run-tests-main-branch](https://github.com/aptos-labs/aptos-core/actions/runs/9103739765/job/25026239745) | 8m | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9103691364/job/25026085184) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9103739765/job/25026239745)  | | [cli-e2e-tests / run-cli-tests](https://github.com/aptos-labs/aptos-core/actions/runs/9103739818/job/25026575065) | 6m | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9103739818/job/25026575065)  | | [check](https://github.com/aptos-labs/aptos-core/actions/runs/9103739760/job/25026230465) | 4m | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9103739760/job/25026230465)  | | [rust-build-cached-packages](https://github.com/aptos-labs/aptos-core/actions/runs/9103739759/job/25026243745) | 4m | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9103739759/job/25026243745)  | | [general-lints](https://github.com/aptos-labs/aptos-core/actions/runs/9103739759/job/25026243892) | 3m | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9103691343/job/25026081157) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9103739759/job/25026243892)  | | [execution-performance / test-target-determinator](https://github.com/aptos-labs/aptos-core/actions/runs/9103739763/job/25026230396) | 3m | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9103739763/job/25026230396)  | | [test-target-determinator](https://github.com/aptos-labs/aptos-core/actions/runs/9103739818/job/25026234948) | 3m | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9103739818/job/25026234948)  | | [check-dynamic-deps](https://github.com/aptos-labs/aptos-core/actions/runs/9103739766/job/25026230092) | 2m | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9103691346/job/25026072679) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9103739766/job/25026230092)  | | [node-api-compatibility-tests / node-api-compatibility-tests](https://github.com/aptos-labs/aptos-core/actions/runs/9103739818/job/25026574923) | 49s | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9103739818/job/25026574923)  | | [semgrep/ci](https://github.com/aptos-labs/aptos-core/actions/runs/9103739757/job/25026230102) | 41s | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9103691348/job/25026072857) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9103739757/job/25026230102)  | | [file_change_determinator](https://github.com/aptos-labs/aptos-core/actions/runs/9103739765/job/25026234034) | 22s | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9103691364/job/25026079578) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9103739765/job/25026234034)  | | [file_change_determinator](https://github.com/aptos-labs/aptos-core/actions/runs/9103739759/job/25026237727) | 20s | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9103691343/job/25026073621) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9103739759/job/25026237727)  | | [file_change_determinator](https://github.com/aptos-labs/aptos-core/actions/runs/9103739818/job/25026234821) | 8s | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9103739818/job/25026234821)  | | [permission-check](https://github.com/aptos-labs/aptos-core/actions/runs/9103739767/job/25026230218) | 6s | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9103691365/job/25026072805) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9103739767/job/25026230218)  | | [permission-check](https://github.com/aptos-labs/aptos-core/actions/runs/9103739807/job/25026230613) | 5s | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9103691367/job/25026073804) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9103739807/job/25026230613)  | | [permission-check](https://github.com/aptos-labs/aptos-core/actions/runs/9103739818/job/25026230652) | 4s | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9103739818/job/25026230652)  | | [permission-check](https://github.com/aptos-labs/aptos-core/actions/runs/9103739765/job/25026230484) | 4s | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9103691364/job/25026073874) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9103739765/job/25026230484)  | | [permission-check](https://github.com/aptos-labs/aptos-core/actions/runs/9103739768/job/25026230107) | 4s | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9103691362/job/25026072669) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9103739768/job/25026230107)  | | [determine-docker-build-metadata](https://github.com/aptos-labs/aptos-core/actions/runs/9103739818/job/25026234648) | 1s | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9103739818/job/25026234648)  |

🚨 1 job on the last run was significantly faster/slower than expected

Job Duration vs 7d avg Delta
rust-build-cached-packages 4m 5m -24%

settings ⋅ feedback ⋅ docs ⋅ learn more about trunk.io

github-actions[bot] commented 2 weeks ago

Forge is running suite framework_upgrade on 01b24e7e3548382dd25440b39a0438a993387f12 ==> 38a0ba4992c7997fe96c51f15c3d22e318f00baf

github-actions[bot] commented 2 weeks ago

Forge is running suite compat on 01b24e7e3548382dd25440b39a0438a993387f12 ==> 38a0ba4992c7997fe96c51f15c3d22e318f00baf

github-actions[bot] commented 2 weeks ago

Forge is running suite realistic_env_max_load on 38a0ba4992c7997fe96c51f15c3d22e318f00baf

github-actions[bot] commented 2 weeks ago

:white_check_mark: Forge suite compat success on 01b24e7e3548382dd25440b39a0438a993387f12 ==> 38a0ba4992c7997fe96c51f15c3d22e318f00baf

Compatibility test results for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 38a0ba4992c7997fe96c51f15c3d22e318f00baf (PR)
1. Check liveness of validators at old version: 01b24e7e3548382dd25440b39a0438a993387f12
compatibility::simple-validator-upgrade::liveness-check : committed: 6938.430889306946 txn/s, latency: 4763.8175059460345 ms, (p50: 4800 ms, p90: 7500 ms, p99: 8100 ms), latency samples: 243860
2. Upgrading first Validator to new version: 38a0ba4992c7997fe96c51f15c3d22e318f00baf
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 1282.898428102236 txn/s, latency: 21250.239025054467 ms, (p50: 25600 ms, p90: 28100 ms, p99: 28600 ms), latency samples: 73440
3. Upgrading rest of first batch to new version: 38a0ba4992c7997fe96c51f15c3d22e318f00baf
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 1813.1755052878045 txn/s, latency: 15922.22657167718 ms, (p50: 18600 ms, p90: 22300 ms, p99: 22600 ms), latency samples: 91940
4. upgrading second batch to new version: 38a0ba4992c7997fe96c51f15c3d22e318f00baf
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 3002.8021471394204 txn/s, latency: 10368.152585357846 ms, (p50: 11100 ms, p90: 13600 ms, p99: 16600 ms), latency samples: 121840
5. check swarm health
Compatibility test for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 38a0ba4992c7997fe96c51f15c3d22e318f00baf passed
Test Ok
github-actions[bot] commented 2 weeks ago

:white_check_mark: Forge suite realistic_env_max_load success on 38a0ba4992c7997fe96c51f15c3d22e318f00baf

two traffics test: inner traffic : committed: 8482.545700876084 txn/s, latency: 4618.42730268358 ms, (p50: 4500 ms, p90: 5400 ms, p99: 10500 ms), latency samples: 3668980
two traffics test : committed: 100.00209222851676 txn/s, latency: 1966.160465116279 ms, (p50: 1900 ms, p90: 2100 ms, p99: 6200 ms), latency samples: 1720
Latency breakdown for phase 0: ["QsBatchToPos: max: 0.212, avg: 0.206", "QsPosToProposal: max: 0.224, avg: 0.206", "ConsensusProposalToOrdered: max: 0.440, avg: 0.404", "ConsensusOrderedToCommit: max: 0.397, avg: 0.387", "ConsensusProposalToCommit: max: 0.803, avg: 0.790"]
Max round gap was 1 [limit 4] at version 1871722. Max no progress secs was 4.940569 [limit 15] at version 1871722.
Test Ok
github-actions[bot] commented 2 weeks ago

:white_check_mark: Forge suite framework_upgrade success on 01b24e7e3548382dd25440b39a0438a993387f12 ==> 38a0ba4992c7997fe96c51f15c3d22e318f00baf

Compatibility test results for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 38a0ba4992c7997fe96c51f15c3d22e318f00baf (PR)
Upgrade the nodes to version: 38a0ba4992c7997fe96c51f15c3d22e318f00baf
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1333.846038427939 txn/s, submitted: 1336.6321398136893 txn/s, failed submission: 2.7861013857502646 txn/s, expired: 2.7861013857502646 txn/s, latency: 2389.5993298520452 ms, (p50: 2100 ms, p90: 4200 ms, p99: 6000 ms), latency samples: 114900
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1130.0892965232365 txn/s, submitted: 1132.3121314888604 txn/s, failed submission: 2.22283496562399 txn/s, expired: 2.22283496562399 txn/s, latency: 2750.064496459481 ms, (p50: 2400 ms, p90: 4800 ms, p99: 7200 ms), latency samples: 101680
5. check swarm health
Compatibility test for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 38a0ba4992c7997fe96c51f15c3d22e318f00baf passed
Upgrade the remaining nodes to version: 38a0ba4992c7997fe96c51f15c3d22e318f00baf
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1126.2043580862082 txn/s, submitted: 1129.0566192697502 txn/s, failed submission: 2.8522611835419265 txn/s, expired: 2.8522611835419265 txn/s, latency: 2662.7967465419833 ms, (p50: 2300 ms, p90: 4500 ms, p99: 6000 ms), latency samples: 102660
Test Ok