aptos-labs / aptos-core

Aptos is a layer 1 blockchain built to support the widespread use of blockchain through better technology and user experience.
https://aptosfoundation.org
Other
5.85k stars 3.54k forks source link

[storage] turn on feature flag for remove_detailed_error_from_hash #13197

Closed areshand closed 3 days ago

areshand commented 2 weeks ago

Description

for v1.12 testnet release

Type of Change

Which Components or Systems Does This Change Impact?

How Has This Been Tested?

The internal error info was tested previously through unit tests

with devnet

Before the feature is on

transaction https://explorer.aptoslabs.com/txn/1319701?network=devnet Move abort in 0x1::from_bcs: EINVALID_UTF8(0x10001): UTF8 check failed in conversion from bytes to string [with detail error]

Run governance proposal to enable the feature https://explorer.aptoslabs.com/txn/1325739?network=devnet

We still have the error details as expected. Devnet is also running healthy https://explorer.aptoslabs.com/txn/1327821?network=devnet

Key Areas to Review

Checklist

trunk-io[bot] commented 2 weeks ago
⏱️ 15h total CI duration on this PR | Job | Cumulative Duration | Recent Runs | |---|---|---| | [forge-framework-upgrade-test / forge](https://github.com/aptos-labs/aptos-core/actions/runs/9066136022/job/24914037858) | 3h 26m | [⬜](https://github.com/aptos-labs/aptos-core/actions/runs/9035925294/job/24832108127) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9036768970/job/24834993408) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9066136022/job/24914037858)  | | [rust-targeted-unit-tests](https://github.com/aptos-labs/aptos-core/actions/runs/9036768949/job/24834356218) | 1h 59m | [🟥](https://github.com/aptos-labs/aptos-core/actions/runs/9009387174/job/24753483278) [⬜](https://github.com/aptos-labs/aptos-core/actions/runs/9035680147/job/24830964256) [⬜](https://github.com/aptos-labs/aptos-core/actions/runs/9035777106/job/24831253316) [🟥](https://github.com/aptos-labs/aptos-core/actions/runs/9035925211/job/24831704892) [🟥](https://github.com/aptos-labs/aptos-core/actions/runs/9036768949/job/24834356218) (+2 more) | | [rust-move-tests](https://github.com/aptos-labs/aptos-core/actions/runs/9036768965/job/24834347122) | 1h 25m | [🟥](https://github.com/aptos-labs/aptos-core/actions/runs/9009387170/job/24753472927) [⬜](https://github.com/aptos-labs/aptos-core/actions/runs/9035680143/job/24830954577) [⬜](https://github.com/aptos-labs/aptos-core/actions/runs/9035777098/job/24831243048) [🟥](https://github.com/aptos-labs/aptos-core/actions/runs/9035925207/job/24831696032) [🟥](https://github.com/aptos-labs/aptos-core/actions/runs/9036768965/job/24834347122) (+2 more) | | [rust-smoke-tests](https://github.com/aptos-labs/aptos-core/actions/runs/9036768949/job/24834356395) | 1h 11m | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9035925211/job/24831704725) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9036768949/job/24834356395)  | | [rust-lints](https://github.com/aptos-labs/aptos-core/actions/runs/9036768949/job/24834355977) | 55m | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9009387174/job/24753482986) [⬜](https://github.com/aptos-labs/aptos-core/actions/runs/9035680147/job/24830963596) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9035777106/job/24831252862) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9035925211/job/24831704575) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9036768949/job/24834355977) (+2 more) | | [rust-move-unit-coverage](https://github.com/aptos-labs/aptos-core/actions/runs/9036768967/job/24834347398) | 53m | [⬜](https://github.com/aptos-labs/aptos-core/actions/runs/9035680146/job/24830954559) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9035777113/job/24831242146) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9036768967/job/24834347398)  | | [windows-build](https://github.com/aptos-labs/aptos-core/actions/runs/9036768939/job/24834346907) | 53m | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9035925208/job/24831684837) [🟥](https://github.com/aptos-labs/aptos-core/actions/runs/9036768939/job/24834346907)  | | [execution-performance / single-node-performance](https://github.com/aptos-labs/aptos-core/actions/runs/9036768972/job/24834463967) | 46m | [🟥](https://github.com/aptos-labs/aptos-core/actions/runs/9035925215/job/24831798913) [🟥](https://github.com/aptos-labs/aptos-core/actions/runs/9036768972/job/24834463967)  | | [run-tests-main-branch](https://github.com/aptos-labs/aptos-core/actions/runs/9056648543/job/24879499645) | 42m | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9035680053/job/24830969937) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9035776411/job/24831245783) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9035925231/job/24831700124) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9036768927/job/24834362623) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9056648543/job/24879499645) (+5 more) | | [forge-compat-test / forge](https://github.com/aptos-labs/aptos-core/actions/runs/9036768970/job/24834992823) | 33m | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9035925294/job/24832107951) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9036768970/job/24834992823)  | | [rust-images / rust-all](https://github.com/aptos-labs/aptos-core/actions/runs/9036768970/job/24834368610) | 31m | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9035925294/job/24831701076) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9036768970/job/24834368610)  | | [forge-e2e-test / forge](https://github.com/aptos-labs/aptos-core/actions/runs/9036768970/job/24834993013) | 27m | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9035925294/job/24832107459) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9036768970/job/24834993013)  | | [cli-e2e-tests / run-cli-tests](https://github.com/aptos-labs/aptos-core/actions/runs/9036768970/job/24834992304) | 13m | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9035925294/job/24832107818) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9036768970/job/24834992304)  | | [general-lints](https://github.com/aptos-labs/aptos-core/actions/runs/9036768949/job/24834355588) | 13m | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9009387174/job/24753483121) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9035680147/job/24830963370) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9035777106/job/24831253115) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9035925211/job/24831704377) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9036768949/job/24834355588) (+2 more) | | [rust-build-cached-packages](https://github.com/aptos-labs/aptos-core/actions/runs/9036768949/job/24834355765) | 12m | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9035925211/job/24831705085) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9036768949/job/24834355765)  | | [check-dynamic-deps](https://github.com/aptos-labs/aptos-core/actions/runs/9036768962/job/24834346540) | 9m | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9009387177/job/24753472426) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9035680145/job/24830953844) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9035777105/job/24831231473) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9035925205/job/24831684935) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9036768962/job/24834346540) (+2 more) | | [check](https://github.com/aptos-labs/aptos-core/actions/runs/9036768948/job/24834347265) | 8m | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9035925210/job/24831685462) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9036768948/job/24834347265)  | | [execution-performance / test-target-determinator](https://github.com/aptos-labs/aptos-core/actions/runs/9036768972/job/24834347187) | 7m | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9035925215/job/24831685414) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9036768972/job/24834347187)  | | [test-target-determinator](https://github.com/aptos-labs/aptos-core/actions/runs/9036768970/job/24834362188) | 7m | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9035925294/job/24831693156) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9036768970/job/24834362188)  | | [semgrep/ci](https://github.com/aptos-labs/aptos-core/actions/runs/9036768952/job/24834346461) | 3m | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9009387180/job/24753472431) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9035680142/job/24830953775) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9035777117/job/24831231526) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9035925198/job/24831685130) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9036768952/job/24834346461) (+2 more) | | [node-api-compatibility-tests / node-api-compatibility-tests](https://github.com/aptos-labs/aptos-core/actions/runs/9036768970/job/24834993226) | 2m | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9035925294/job/24832107649) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9036768970/job/24834993226)  | | [file_change_determinator](https://github.com/aptos-labs/aptos-core/actions/runs/9036768927/job/24834352426) | 2m | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9009387059/job/24753476988) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9035680053/job/24830960147) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9035776411/job/24831237363) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9035925231/job/24831691546) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9036768927/job/24834352426) (+4 more) | | [file_change_determinator](https://github.com/aptos-labs/aptos-core/actions/runs/9036768949/job/24834347681) | 1m | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9009387174/job/24753472919) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9035680147/job/24830954602) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9035777106/job/24831242870) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9035925211/job/24831696402) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9036768949/job/24834347681) (+2 more) | | [permission-check](https://github.com/aptos-labs/aptos-core/actions/runs/9036768932/job/24834346600) | 30s | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9009387060/job/24753471996) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9035680058/job/24830953659) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9035776412/job/24831230142) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9035925242/job/24831684822) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9036768932/job/24834346600) (+4 more) | | [permission-check](https://github.com/aptos-labs/aptos-core/actions/runs/9036768927/job/24834347167) | 24s | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9009387059/job/24753472157) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9035680053/job/24830954342) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9035776411/job/24831230465) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9035925231/job/24831685739) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9036768927/job/24834347167) (+4 more) | | [permission-check](https://github.com/aptos-labs/aptos-core/actions/runs/9036768924/job/24834346690) | 24s | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9009387069/job/24753471977) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9035680065/job/24830953500) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9035776408/job/24831230093) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9035925239/job/24831684888) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9036768924/job/24834346690) (+4 more) | | [permission-check](https://github.com/aptos-labs/aptos-core/actions/runs/9036768922/job/24834347482) | 24s | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9009387055/job/24753472188) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9035680046/job/24830954027) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9035776406/job/24831230415) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9035925238/job/24831685681) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9036768922/job/24834347482) (+4 more) | | [file_change_determinator](https://github.com/aptos-labs/aptos-core/actions/runs/9036768970/job/24834361982) | 20s | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9035925294/job/24831693016) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9036768970/job/24834361982)  | | [permission-check](https://github.com/aptos-labs/aptos-core/actions/runs/9036768970/job/24834356719) | 5s | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9035925294/job/24831686182) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9036768970/job/24834356719)  | | [determine-docker-build-metadata](https://github.com/aptos-labs/aptos-core/actions/runs/9036768970/job/24834361748) | 4s | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9035925294/job/24831692768) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/9036768970/job/24834361748)  |

🚨 2 jobs on the last run were significantly faster/slower than expected

Job Duration vs 7d avg Delta
rust-images / rust-all 19m 14m +36%
rust-lints 5m 9m -38%

settingsfeedbackdocs ⋅ learn more about trunk.io

msmouse commented 2 weeks ago

enable here for devnet genesis as well, please:

https://github.com/aptos-labs/aptos-core/blob/bbf569abd260d94bc30fe96da297d2aecb193644/types/src/on_chain_config/aptos_features.rs#L134-L137

codecov[bot] commented 1 week ago

Codecov Report

Attention: Patch coverage is 6.06061% with 31 lines in your changes are missing coverage. Please review.

Project coverage is 57.6%. Comparing base (6cdd4c2) to head (bf5010f). Report is 5 commits behind head on main.

Files Patch % Lines
types/src/transaction/mod.rs 0.0% 23 Missing :warning:
types/src/proof/definition.rs 0.0% 8 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #13197 +/- ## ========================================= - Coverage 57.6% 57.6% -0.1% ========================================= Files 834 834 Lines 198397 198430 +33 ========================================= + Hits 114318 114320 +2 - Misses 84079 84110 +31 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

github-actions[bot] commented 1 week ago

Forge is running suite framework_upgrade on 01b24e7e3548382dd25440b39a0438a993387f12 ==> 3f70dece805a5d0db26c8f5073a13d39a2c90d6d

github-actions[bot] commented 1 week ago

Forge is running suite realistic_env_max_load on 3f70dece805a5d0db26c8f5073a13d39a2c90d6d

github-actions[bot] commented 1 week ago

Forge is running suite compat on 01b24e7e3548382dd25440b39a0438a993387f12 ==> 3f70dece805a5d0db26c8f5073a13d39a2c90d6d

github-actions[bot] commented 1 week ago

:white_check_mark: Forge suite realistic_env_max_load success on 3f70dece805a5d0db26c8f5073a13d39a2c90d6d

two traffics test: inner traffic : committed: 8151 txn/s, latency: 4808 ms, (p50: 4500 ms, p90: 5600 ms, p99: 10800 ms), latency samples: 3521660
two traffics test : committed: 100 txn/s, latency: 1841 ms, (p50: 1800 ms, p90: 2200 ms, p99: 2300 ms), latency samples: 1780
Latency breakdown for phase 0: ["QsBatchToPos: max: 0.201, avg: 0.201", "QsPosToProposal: max: 0.233, avg: 0.220", "ConsensusProposalToOrdered: max: 0.472, avg: 0.413", "ConsensusOrderedToCommit: max: 0.380, avg: 0.361", "ConsensusProposalToCommit: max: 0.787, avg: 0.774"]
Max round gap was 1 [limit 4] at version 1753121. Max no progress secs was 4.761114 [limit 15] at version 1753121.
Test Ok
github-actions[bot] commented 1 week ago

:white_check_mark: Forge suite compat success on 01b24e7e3548382dd25440b39a0438a993387f12 ==> 3f70dece805a5d0db26c8f5073a13d39a2c90d6d

Compatibility test results for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 3f70dece805a5d0db26c8f5073a13d39a2c90d6d (PR)
1. Check liveness of validators at old version: 01b24e7e3548382dd25440b39a0438a993387f12
compatibility::simple-validator-upgrade::liveness-check : committed: 6481 txn/s, latency: 5105 ms, (p50: 5000 ms, p90: 8400 ms, p99: 9900 ms), latency samples: 226860
2. Upgrading first Validator to new version: 3f70dece805a5d0db26c8f5073a13d39a2c90d6d
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 1424 txn/s, latency: 19856 ms, (p50: 21800 ms, p90: 27100 ms, p99: 28600 ms), latency samples: 74080
3. Upgrading rest of first batch to new version: 3f70dece805a5d0db26c8f5073a13d39a2c90d6d
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 1582 txn/s, latency: 17621 ms, (p50: 19700 ms, p90: 22700 ms, p99: 23800 ms), latency samples: 83860
4. upgrading second batch to new version: 3f70dece805a5d0db26c8f5073a13d39a2c90d6d
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 3543 txn/s, latency: 8909 ms, (p50: 9600 ms, p90: 12700 ms, p99: 13000 ms), latency samples: 145280
5. check swarm health
Compatibility test for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 3f70dece805a5d0db26c8f5073a13d39a2c90d6d passed
Test Ok
github-actions[bot] commented 1 week ago

Forge is running suite realistic_env_max_load on 3756113e1a77719b93c767858e9b1c7207ebe406

github-actions[bot] commented 1 week ago

Forge is running suite framework_upgrade on 01b24e7e3548382dd25440b39a0438a993387f12 ==> 3756113e1a77719b93c767858e9b1c7207ebe406

github-actions[bot] commented 1 week ago

Forge is running suite compat on 01b24e7e3548382dd25440b39a0438a993387f12 ==> 3756113e1a77719b93c767858e9b1c7207ebe406

github-actions[bot] commented 1 week ago

:white_check_mark: Forge suite realistic_env_max_load success on 3756113e1a77719b93c767858e9b1c7207ebe406

two traffics test: inner traffic : committed: 7863.962558723379 txn/s, latency: 4982.012280335466 ms, (p50: 4800 ms, p90: 5700 ms, p99: 10500 ms), latency samples: 3400640
two traffics test : committed: 100.0251806937126 txn/s, latency: 1909.1366279069769 ms, (p50: 1900 ms, p90: 2200 ms, p99: 2700 ms), latency samples: 1720
Latency breakdown for phase 0: ["QsBatchToPos: max: 0.202, avg: 0.200", "QsPosToProposal: max: 0.246, avg: 0.233", "ConsensusProposalToOrdered: max: 0.439, avg: 0.418", "ConsensusOrderedToCommit: max: 0.376, avg: 0.365", "ConsensusProposalToCommit: max: 0.794, avg: 0.783"]
Max round gap was 1 [limit 4] at version 1752773. Max no progress secs was 4.370554 [limit 15] at version 1752773.
Test Ok
github-actions[bot] commented 1 week ago

:white_check_mark: Forge suite compat success on 01b24e7e3548382dd25440b39a0438a993387f12 ==> 3756113e1a77719b93c767858e9b1c7207ebe406

Compatibility test results for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 3756113e1a77719b93c767858e9b1c7207ebe406 (PR)
1. Check liveness of validators at old version: 01b24e7e3548382dd25440b39a0438a993387f12
compatibility::simple-validator-upgrade::liveness-check : committed: 5118.1062390700745 txn/s, latency: 5774.976366826745 ms, (p50: 5000 ms, p90: 9500 ms, p99: 16000 ms), latency samples: 216560
2. Upgrading first Validator to new version: 3756113e1a77719b93c767858e9b1c7207ebe406
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 1614.2531987762522 txn/s, latency: 16940.446756329115 ms, (p50: 19600 ms, p90: 24000 ms, p99: 24900 ms), latency samples: 88480
3. Upgrading rest of first batch to new version: 3756113e1a77719b93c767858e9b1c7207ebe406
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 1502.514929533177 txn/s, latency: 16006.817987533394 ms, (p50: 19200 ms, p90: 22900 ms, p99: 23600 ms), latency samples: 89840
4. upgrading second batch to new version: 3756113e1a77719b93c767858e9b1c7207ebe406
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 446.1499473505478 txn/s, submitted: 604.9695885969664 txn/s, expired: 158.8196412464186 txn/s, latency: 55784.69185396115 ms, (p50: 57100 ms, p90: 58600 ms, p99: 59000 ms), latency samples: 29499
5. check swarm health
Compatibility test for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 3756113e1a77719b93c767858e9b1c7207ebe406 passed
Test Ok
github-actions[bot] commented 1 week ago

:white_check_mark: Forge suite framework_upgrade success on 01b24e7e3548382dd25440b39a0438a993387f12 ==> 3756113e1a77719b93c767858e9b1c7207ebe406

Compatibility test results for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 3756113e1a77719b93c767858e9b1c7207ebe406 (PR)
Upgrade the nodes to version: 3756113e1a77719b93c767858e9b1c7207ebe406
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1170.2468453669192 txn/s, submitted: 1172.938614993985 txn/s, failed submission: 2.691769627065944 txn/s, expired: 2.691769627065944 txn/s, latency: 2564.226528656316 ms, (p50: 2100 ms, p90: 4500 ms, p99: 7200 ms), latency samples: 104340
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1239.6905978617942 txn/s, submitted: 1242.598044690624 txn/s, failed submission: 2.907446828829754 txn/s, expired: 2.907446828829754 txn/s, latency: 2591.6538066029225 ms, (p50: 2100 ms, p90: 4500 ms, p99: 5700 ms), latency samples: 110860
5. check swarm health
Compatibility test for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 3756113e1a77719b93c767858e9b1c7207ebe406 passed
Upgrade the remaining nodes to version: 3756113e1a77719b93c767858e9b1c7207ebe406
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1096.1574857657035 txn/s, submitted: 1098.3541941540116 txn/s, failed submission: 2.196708388308023 txn/s, expired: 2.196708388308023 txn/s, latency: 2702.7761022044087 ms, (p50: 2100 ms, p90: 4800 ms, p99: 6700 ms), latency samples: 99800
Test Ok
github-actions[bot] commented 1 week ago

Forge is running suite framework_upgrade on 01b24e7e3548382dd25440b39a0438a993387f12 ==> 6f75b9b2e7fce810e764079a7549977dea95d593

github-actions[bot] commented 1 week ago

Forge is running suite realistic_env_max_load on 6f75b9b2e7fce810e764079a7549977dea95d593

github-actions[bot] commented 1 week ago

Forge is running suite compat on 01b24e7e3548382dd25440b39a0438a993387f12 ==> 6f75b9b2e7fce810e764079a7549977dea95d593

github-actions[bot] commented 1 week ago

:white_check_mark: Forge suite realistic_env_max_load success on 6f75b9b2e7fce810e764079a7549977dea95d593

two traffics test: inner traffic : committed: 7353.569551912952 txn/s, latency: 5320.970813129664 ms, (p50: 5100 ms, p90: 6200 ms, p99: 11100 ms), latency samples: 3184240
two traffics test : committed: 99.89548529767143 txn/s, latency: 1960.8627777777779 ms, (p50: 1900 ms, p90: 2200 ms, p99: 4600 ms), latency samples: 1800
Latency breakdown for phase 0: ["QsBatchToPos: max: 0.204, avg: 0.201", "QsPosToProposal: max: 0.369, avg: 0.275", "ConsensusProposalToOrdered: max: 0.487, avg: 0.442", "ConsensusOrderedToCommit: max: 0.401, avg: 0.387", "ConsensusProposalToCommit: max: 0.867, avg: 0.829"]
Max round gap was 1 [limit 4] at version 1348616. Max no progress secs was 4.780177 [limit 15] at version 1348616.
Test Ok
github-actions[bot] commented 1 week ago

:white_check_mark: Forge suite compat success on 01b24e7e3548382dd25440b39a0438a993387f12 ==> 6f75b9b2e7fce810e764079a7549977dea95d593

Compatibility test results for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 6f75b9b2e7fce810e764079a7549977dea95d593 (PR)
1. Check liveness of validators at old version: 01b24e7e3548382dd25440b39a0438a993387f12
compatibility::simple-validator-upgrade::liveness-check : committed: 6148.184318840567 txn/s, latency: 5005.955109024351 ms, (p50: 4800 ms, p90: 5400 ms, p99: 10300 ms), latency samples: 251320
2. Upgrading first Validator to new version: 6f75b9b2e7fce810e764079a7549977dea95d593
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 1674.3497556801105 txn/s, latency: 15860.98635970601 ms, (p50: 18300 ms, p90: 23100 ms, p99: 25000 ms), latency samples: 92520
3. Upgrading rest of first batch to new version: 6f75b9b2e7fce810e764079a7549977dea95d593
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 1667.3702602742937 txn/s, latency: 17078.067461077844 ms, (p50: 19900 ms, p90: 22900 ms, p99: 23700 ms), latency samples: 83500
4. upgrading second batch to new version: 6f75b9b2e7fce810e764079a7549977dea95d593
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 3427.0662906612993 txn/s, latency: 9231.553277242503 ms, (p50: 9700 ms, p90: 12700 ms, p99: 13100 ms), latency samples: 147380
5. check swarm health
Compatibility test for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 6f75b9b2e7fce810e764079a7549977dea95d593 passed
Test Ok
github-actions[bot] commented 1 week ago

:white_check_mark: Forge suite framework_upgrade success on 01b24e7e3548382dd25440b39a0438a993387f12 ==> 6f75b9b2e7fce810e764079a7549977dea95d593

Compatibility test results for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 6f75b9b2e7fce810e764079a7549977dea95d593 (PR)
Upgrade the nodes to version: 6f75b9b2e7fce810e764079a7549977dea95d593
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1182.5035795289864 txn/s, submitted: 1185.6449552620804 txn/s, failed submission: 3.1413757330940815 txn/s, expired: 3.1413757330940815 txn/s, latency: 2567.137125237192 ms, (p50: 2100 ms, p90: 4700 ms, p99: 6500 ms), latency samples: 105400
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1215.3278576839623 txn/s, submitted: 1217.341471586605 txn/s, failed submission: 2.0136139026427946 txn/s, expired: 2.0136139026427946 txn/s, latency: 2551.5316273932253 ms, (p50: 2100 ms, p90: 4200 ms, p99: 7200 ms), latency samples: 108640
5. check swarm health
Compatibility test for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 6f75b9b2e7fce810e764079a7549977dea95d593 passed
Upgrade the remaining nodes to version: 6f75b9b2e7fce810e764079a7549977dea95d593
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1254.276079843485 txn/s, submitted: 1258.6708804035532 txn/s, failed submission: 4.394800560068273 txn/s, expired: 4.394800560068273 txn/s, latency: 2524.438936580238 ms, (p50: 2100 ms, p90: 4500 ms, p99: 6000 ms), latency samples: 114160
Test Ok
github-actions[bot] commented 1 week ago

Forge is running suite realistic_env_max_load on 2ee5dace82d220976ebf44814497933d68c40aba

github-actions[bot] commented 1 week ago

Forge is running suite framework_upgrade on 01b24e7e3548382dd25440b39a0438a993387f12 ==> 2ee5dace82d220976ebf44814497933d68c40aba

github-actions[bot] commented 1 week ago

Forge is running suite compat on 01b24e7e3548382dd25440b39a0438a993387f12 ==> 2ee5dace82d220976ebf44814497933d68c40aba

github-actions[bot] commented 1 week ago

:white_check_mark: Forge suite compat success on 01b24e7e3548382dd25440b39a0438a993387f12 ==> 2ee5dace82d220976ebf44814497933d68c40aba

Compatibility test results for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 2ee5dace82d220976ebf44814497933d68c40aba (PR)
1. Check liveness of validators at old version: 01b24e7e3548382dd25440b39a0438a993387f12
compatibility::simple-validator-upgrade::liveness-check : committed: 6839.24623915166 txn/s, latency: 4840.20404662781 ms, (p50: 5000 ms, p90: 7400 ms, p99: 8700 ms), latency samples: 240200
2. Upgrading first Validator to new version: 2ee5dace82d220976ebf44814497933d68c40aba
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 1767.8923562034436 txn/s, latency: 16419.27508779631 ms, (p50: 18600 ms, p90: 23800 ms, p99: 24100 ms), latency samples: 91120
3. Upgrading rest of first batch to new version: 2ee5dace82d220976ebf44814497933d68c40aba
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 1619.6017562391244 txn/s, latency: 17204.55023046785 ms, (p50: 19300 ms, p90: 23100 ms, p99: 23700 ms), latency samples: 86780
4. upgrading second batch to new version: 2ee5dace82d220976ebf44814497933d68c40aba
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 3030.844206941147 txn/s, latency: 10040.666260789149 ms, (p50: 9900 ms, p90: 12400 ms, p99: 12900 ms), latency samples: 129760
5. check swarm health
Compatibility test for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 2ee5dace82d220976ebf44814497933d68c40aba passed
Test Ok
github-actions[bot] commented 1 week ago

:white_check_mark: Forge suite realistic_env_max_load success on 2ee5dace82d220976ebf44814497933d68c40aba

two traffics test: inner traffic : committed: 8623.265777439374 txn/s, latency: 4550.526464644836 ms, (p50: 4400 ms, p90: 5300 ms, p99: 9900 ms), latency samples: 3721380
two traffics test : committed: 99.88804320073478 txn/s, latency: 1838.3695652173913 ms, (p50: 1800 ms, p90: 2100 ms, p99: 2500 ms), latency samples: 1840
Latency breakdown for phase 0: ["QsBatchToPos: max: 0.205, avg: 0.202", "QsPosToProposal: max: 0.223, avg: 0.205", "ConsensusProposalToOrdered: max: 0.431, avg: 0.402", "ConsensusOrderedToCommit: max: 0.388, avg: 0.370", "ConsensusProposalToCommit: max: 0.786, avg: 0.772"]
Max round gap was 1 [limit 4] at version 1730689. Max no progress secs was 5.071468 [limit 15] at version 1730689.
Test Ok
github-actions[bot] commented 1 week ago

:white_check_mark: Forge suite framework_upgrade success on 01b24e7e3548382dd25440b39a0438a993387f12 ==> 2ee5dace82d220976ebf44814497933d68c40aba

Compatibility test results for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 2ee5dace82d220976ebf44814497933d68c40aba (PR)
Upgrade the nodes to version: 2ee5dace82d220976ebf44814497933d68c40aba
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1281.271104427355 txn/s, submitted: 1283.0624656359219 txn/s, failed submission: 1.791361208566732 txn/s, expired: 1.791361208566732 txn/s, latency: 2380.377874868927 ms, (p50: 2000 ms, p90: 4200 ms, p99: 6500 ms), latency samples: 114440
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1221.4846108696684 txn/s, submitted: 1224.6051919923282 txn/s, failed submission: 3.1205811226597366 txn/s, expired: 3.1205811226597366 txn/s, latency: 2621.4369890510948 ms, (p50: 2100 ms, p90: 4500 ms, p99: 6900 ms), latency samples: 109600
5. check swarm health
Compatibility test for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 2ee5dace82d220976ebf44814497933d68c40aba passed
Upgrade the remaining nodes to version: 2ee5dace82d220976ebf44814497933d68c40aba
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1176.3378272048064 txn/s, submitted: 1178.3761484302286 txn/s, failed submission: 2.0383212254222673 txn/s, expired: 2.0383212254222673 txn/s, latency: 2596.7317289179823 ms, (p50: 2400 ms, p90: 4200 ms, p99: 7200 ms), latency samples: 103880
Test Ok
github-actions[bot] commented 1 week ago

Forge is running suite realistic_env_max_load on d7b981e32157288833946eed423d60f4f73b8aad

github-actions[bot] commented 1 week ago

Forge is running suite compat on 01b24e7e3548382dd25440b39a0438a993387f12 ==> d7b981e32157288833946eed423d60f4f73b8aad

github-actions[bot] commented 1 week ago

Forge is running suite framework_upgrade on 01b24e7e3548382dd25440b39a0438a993387f12 ==> d7b981e32157288833946eed423d60f4f73b8aad

github-actions[bot] commented 1 week ago

:white_check_mark: Forge suite compat success on 01b24e7e3548382dd25440b39a0438a993387f12 ==> d7b981e32157288833946eed423d60f4f73b8aad

Compatibility test results for 01b24e7e3548382dd25440b39a0438a993387f12 ==> d7b981e32157288833946eed423d60f4f73b8aad (PR)
1. Check liveness of validators at old version: 01b24e7e3548382dd25440b39a0438a993387f12
compatibility::simple-validator-upgrade::liveness-check : committed: 5333.965700022138 txn/s, latency: 5544.477768262396 ms, (p50: 4800 ms, p90: 9600 ms, p99: 11300 ms), latency samples: 233540
2. Upgrading first Validator to new version: d7b981e32157288833946eed423d60f4f73b8aad
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 1445.4352782687808 txn/s, latency: 16424.423150870844 ms, (p50: 19200 ms, p90: 23700 ms, p99: 31100 ms), latency samples: 88420
3. Upgrading rest of first batch to new version: d7b981e32157288833946eed423d60f4f73b8aad
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 1826.1884614139701 txn/s, latency: 15697.582202095155 ms, (p50: 18100 ms, p90: 22300 ms, p99: 22600 ms), latency samples: 91640
4. upgrading second batch to new version: d7b981e32157288833946eed423d60f4f73b8aad
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 3303.191592564841 txn/s, latency: 9443.244367294885 ms, (p50: 9700 ms, p90: 12600 ms, p99: 12900 ms), latency samples: 141140
5. check swarm health
Compatibility test for 01b24e7e3548382dd25440b39a0438a993387f12 ==> d7b981e32157288833946eed423d60f4f73b8aad passed
Test Ok
github-actions[bot] commented 1 week ago

:white_check_mark: Forge suite realistic_env_max_load success on d7b981e32157288833946eed423d60f4f73b8aad

two traffics test: inner traffic : committed: 8258.896953468742 txn/s, latency: 4749.4444492131615 ms, (p50: 4500 ms, p90: 5700 ms, p99: 10000 ms), latency samples: 3564900
two traffics test : committed: 99.9659287105897 txn/s, latency: 1854.7340425531916 ms, (p50: 1800 ms, p90: 2100 ms, p99: 3200 ms), latency samples: 1880
Latency breakdown for phase 0: ["QsBatchToPos: max: 0.208, avg: 0.202", "QsPosToProposal: max: 0.222, avg: 0.201", "ConsensusProposalToOrdered: max: 0.472, avg: 0.430", "ConsensusOrderedToCommit: max: 0.381, avg: 0.371", "ConsensusProposalToCommit: max: 0.811, avg: 0.800"]
Max round gap was 1 [limit 4] at version 1829080. Max no progress secs was 5.08068 [limit 15] at version 1829080.
Test Ok
github-actions[bot] commented 1 week ago

:white_check_mark: Forge suite framework_upgrade success on 01b24e7e3548382dd25440b39a0438a993387f12 ==> d7b981e32157288833946eed423d60f4f73b8aad

Compatibility test results for 01b24e7e3548382dd25440b39a0438a993387f12 ==> d7b981e32157288833946eed423d60f4f73b8aad (PR)
Upgrade the nodes to version: d7b981e32157288833946eed423d60f4f73b8aad
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1377.956178464069 txn/s, submitted: 1379.6504860785512 txn/s, failed submission: 1.6943076144824314 txn/s, expired: 1.6943076144824314 txn/s, latency: 2338.272150008783 ms, (p50: 2100 ms, p90: 4200 ms, p99: 7500 ms), latency samples: 113860
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1154.4616779494631 txn/s, submitted: 1157.8678394697704 txn/s, failed submission: 3.406161520307228 txn/s, expired: 3.406161520307228 txn/s, latency: 2647.2393391030682 ms, (p50: 2200 ms, p90: 4500 ms, p99: 6400 ms), latency samples: 101680
5. check swarm health
Compatibility test for 01b24e7e3548382dd25440b39a0438a993387f12 ==> d7b981e32157288833946eed423d60f4f73b8aad passed
Upgrade the remaining nodes to version: d7b981e32157288833946eed423d60f4f73b8aad
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1161.1041139451004 txn/s, submitted: 1164.4489500713105 txn/s, failed submission: 3.344836126210199 txn/s, expired: 3.344836126210199 txn/s, latency: 2579.4754657192243 ms, (p50: 2100 ms, p90: 4200 ms, p99: 6600 ms), latency samples: 104140
Test Ok
github-actions[bot] commented 6 days ago

Forge is running suite framework_upgrade on 01b24e7e3548382dd25440b39a0438a993387f12 ==> 15bed9b46063a57deb6184b4b7692b125815ab4e

github-actions[bot] commented 6 days ago

Forge is running suite compat on 01b24e7e3548382dd25440b39a0438a993387f12 ==> 15bed9b46063a57deb6184b4b7692b125815ab4e

github-actions[bot] commented 6 days ago

Forge is running suite realistic_env_max_load on 15bed9b46063a57deb6184b4b7692b125815ab4e

github-actions[bot] commented 6 days ago

:white_check_mark: Forge suite compat success on 01b24e7e3548382dd25440b39a0438a993387f12 ==> 15bed9b46063a57deb6184b4b7692b125815ab4e

Compatibility test results for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 15bed9b46063a57deb6184b4b7692b125815ab4e (PR)
1. Check liveness of validators at old version: 01b24e7e3548382dd25440b39a0438a993387f12
compatibility::simple-validator-upgrade::liveness-check : committed: 6148.905451093835 txn/s, latency: 5003.810807943648 ms, (p50: 4800 ms, p90: 8700 ms, p99: 10700 ms), latency samples: 235660
2. Upgrading first Validator to new version: 15bed9b46063a57deb6184b4b7692b125815ab4e
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 1647.1452398596957 txn/s, latency: 17240.050158478607 ms, (p50: 19000 ms, p90: 23200 ms, p99: 23500 ms), latency samples: 88340
3. Upgrading rest of first batch to new version: 15bed9b46063a57deb6184b4b7692b125815ab4e
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 1159.9908109941807 txn/s, latency: 22780.571797490516 ms, (p50: 25400 ms, p90: 29300 ms, p99: 30100 ms), latency samples: 68540
4. upgrading second batch to new version: 15bed9b46063a57deb6184b4b7692b125815ab4e
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 3390.489877143342 txn/s, latency: 8934.172181843116 ms, (p50: 9800 ms, p90: 12100 ms, p99: 12900 ms), latency samples: 145840
5. check swarm health
Compatibility test for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 15bed9b46063a57deb6184b4b7692b125815ab4e passed
Test Ok
github-actions[bot] commented 6 days ago

:white_check_mark: Forge suite realistic_env_max_load success on 15bed9b46063a57deb6184b4b7692b125815ab4e

two traffics test: inner traffic : committed: 8346.12812312802 txn/s, latency: 4693.801854549039 ms, (p50: 4500 ms, p90: 5400 ms, p99: 11100 ms), latency samples: 3652640
two traffics test : committed: 100.09446372628818 txn/s, latency: 1939.2602272727272 ms, (p50: 1800 ms, p90: 2200 ms, p99: 7500 ms), latency samples: 1760
Latency breakdown for phase 0: ["QsBatchToPos: max: 0.203, avg: 0.202", "QsPosToProposal: max: 0.228, avg: 0.210", "ConsensusProposalToOrdered: max: 0.443, avg: 0.409", "ConsensusOrderedToCommit: max: 0.380, avg: 0.370", "ConsensusProposalToCommit: max: 0.790, avg: 0.778"]
Max round gap was 1 [limit 4] at version 1087403. Max no progress secs was 5.063201 [limit 15] at version 1087403.
Test Ok
github-actions[bot] commented 6 days ago

:white_check_mark: Forge suite framework_upgrade success on 01b24e7e3548382dd25440b39a0438a993387f12 ==> 15bed9b46063a57deb6184b4b7692b125815ab4e

Compatibility test results for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 15bed9b46063a57deb6184b4b7692b125815ab4e (PR)
Upgrade the nodes to version: 15bed9b46063a57deb6184b4b7692b125815ab4e
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1513.8684536079622 txn/s, submitted: 1515.9670409635096 txn/s, failed submission: 2.0985873555473398 txn/s, expired: 2.0985873555473398 txn/s, latency: 2350.7029630913185 ms, (p50: 1800 ms, p90: 3900 ms, p99: 6600 ms), latency samples: 115420
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1281.0979320837182 txn/s, submitted: 1283.7051619177462 txn/s, failed submission: 2.6072298340279185 txn/s, expired: 2.6072298340279185 txn/s, latency: 2502.2134320074006 ms, (p50: 2100 ms, p90: 4200 ms, p99: 5400 ms), latency samples: 108100
5. check swarm health
Compatibility test for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 15bed9b46063a57deb6184b4b7692b125815ab4e passed
Upgrade the remaining nodes to version: 15bed9b46063a57deb6184b4b7692b125815ab4e
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1156.4039517482106 txn/s, submitted: 1159.4208595091432 txn/s, failed submission: 3.016907760932518 txn/s, expired: 3.016907760932518 txn/s, latency: 2776.539112984146 ms, (p50: 2100 ms, p90: 5100 ms, p99: 7600 ms), latency samples: 99660
Test Ok
github-actions[bot] commented 6 days ago

Forge is running suite realistic_env_max_load on f1a64108396db35682a6cd4a94e2958fa9a020ae

github-actions[bot] commented 6 days ago

Forge is running suite framework_upgrade on 01b24e7e3548382dd25440b39a0438a993387f12 ==> f1a64108396db35682a6cd4a94e2958fa9a020ae

github-actions[bot] commented 6 days ago

Forge is running suite compat on 01b24e7e3548382dd25440b39a0438a993387f12 ==> f1a64108396db35682a6cd4a94e2958fa9a020ae

github-actions[bot] commented 6 days ago

:white_check_mark: Forge suite realistic_env_max_load success on f1a64108396db35682a6cd4a94e2958fa9a020ae

two traffics test: inner traffic : committed: 7946.680319967568 txn/s, latency: 4931.766517885739 ms, (p50: 4800 ms, p90: 5700 ms, p99: 10500 ms), latency samples: 3434580
two traffics test : committed: 100.06885629353441 txn/s, latency: 1904.1333333333334 ms, (p50: 1800 ms, p90: 2100 ms, p99: 5300 ms), latency samples: 1800
Latency breakdown for phase 0: ["QsBatchToPos: max: 0.206, avg: 0.201", "QsPosToProposal: max: 0.255, avg: 0.232", "ConsensusProposalToOrdered: max: 0.436, avg: 0.412", "ConsensusOrderedToCommit: max: 0.373, avg: 0.363", "ConsensusProposalToCommit: max: 0.790, avg: 0.775"]
Max round gap was 1 [limit 4] at version 1549001. Max no progress secs was 4.468983 [limit 15] at version 1549001.
Test Ok
github-actions[bot] commented 6 days ago

:white_check_mark: Forge suite compat success on 01b24e7e3548382dd25440b39a0438a993387f12 ==> f1a64108396db35682a6cd4a94e2958fa9a020ae

Compatibility test results for 01b24e7e3548382dd25440b39a0438a993387f12 ==> f1a64108396db35682a6cd4a94e2958fa9a020ae (PR)
1. Check liveness of validators at old version: 01b24e7e3548382dd25440b39a0438a993387f12
compatibility::simple-validator-upgrade::liveness-check : committed: 5724.764270811091 txn/s, latency: 5337.258283042924 ms, (p50: 4800 ms, p90: 10200 ms, p99: 15000 ms), latency samples: 235300
2. Upgrading first Validator to new version: f1a64108396db35682a6cd4a94e2958fa9a020ae
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 1763.5017499240882 txn/s, latency: 16188.84642289348 ms, (p50: 19300 ms, p90: 23900 ms, p99: 24400 ms), latency samples: 88060
3. Upgrading rest of first batch to new version: f1a64108396db35682a6cd4a94e2958fa9a020ae
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 1822.2934518894695 txn/s, latency: 15779.63602314916 ms, (p50: 18700 ms, p90: 22300 ms, p99: 22600 ms), latency samples: 91580
4. upgrading second batch to new version: f1a64108396db35682a6cd4a94e2958fa9a020ae
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 3186.763109492991 txn/s, latency: 9356.016562089595 ms, (p50: 9900 ms, p90: 11100 ms, p99: 12600 ms), latency samples: 137060
5. check swarm health
Compatibility test for 01b24e7e3548382dd25440b39a0438a993387f12 ==> f1a64108396db35682a6cd4a94e2958fa9a020ae passed
Test Ok
github-actions[bot] commented 6 days ago

Forge is running suite compat on 01b24e7e3548382dd25440b39a0438a993387f12 ==> 51229a000be3554b968d649e985815b9d876becd

github-actions[bot] commented 6 days ago

Forge is running suite framework_upgrade on 01b24e7e3548382dd25440b39a0438a993387f12 ==> 51229a000be3554b968d649e985815b9d876becd

github-actions[bot] commented 6 days ago

Forge is running suite realistic_env_max_load on 51229a000be3554b968d649e985815b9d876becd

github-actions[bot] commented 6 days ago

:white_check_mark: Forge suite compat success on 01b24e7e3548382dd25440b39a0438a993387f12 ==> 51229a000be3554b968d649e985815b9d876becd

Compatibility test results for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 51229a000be3554b968d649e985815b9d876becd (PR)
1. Check liveness of validators at old version: 01b24e7e3548382dd25440b39a0438a993387f12
compatibility::simple-validator-upgrade::liveness-check : committed: 5047.515683702204 txn/s, latency: 5893.993574766355 ms, (p50: 4900 ms, p90: 10200 ms, p99: 15800 ms), latency samples: 222560
2. Upgrading first Validator to new version: 51229a000be3554b968d649e985815b9d876becd
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 1714.1640547023699 txn/s, latency: 16252.012616619657 ms, (p50: 18900 ms, p90: 22800 ms, p99: 24000 ms), latency samples: 92180
3. Upgrading rest of first batch to new version: 51229a000be3554b968d649e985815b9d876becd
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 1808.0714977764037 txn/s, latency: 15822.224323425022 ms, (p50: 19300 ms, p90: 22900 ms, p99: 23400 ms), latency samples: 90160
4. upgrading second batch to new version: 51229a000be3554b968d649e985815b9d876becd
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 3041.0639785054814 txn/s, latency: 9693.981440613026 ms, (p50: 9900 ms, p90: 11400 ms, p99: 12600 ms), latency samples: 130500
5. check swarm health
Compatibility test for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 51229a000be3554b968d649e985815b9d876becd passed
Test Ok
github-actions[bot] commented 6 days ago

:white_check_mark: Forge suite realistic_env_max_load success on 51229a000be3554b968d649e985815b9d876becd

two traffics test: inner traffic : committed: 7882.276046883494 txn/s, latency: 4972.84664318422 ms, (p50: 4800 ms, p90: 5700 ms, p99: 10500 ms), latency samples: 3406800
two traffics test : committed: 100.07251267847379 txn/s, latency: 1869.3916666666667 ms, (p50: 1800 ms, p90: 2100 ms, p99: 4400 ms), latency samples: 1800
Latency breakdown for phase 0: ["QsBatchToPos: max: 0.208, avg: 0.202", "QsPosToProposal: max: 0.273, avg: 0.219", "ConsensusProposalToOrdered: max: 0.455, avg: 0.422", "ConsensusOrderedToCommit: max: 0.369, avg: 0.351", "ConsensusProposalToCommit: max: 0.810, avg: 0.773"]
Max round gap was 1 [limit 4] at version 1685893. Max no progress secs was 4.734646 [limit 15] at version 1685893.
Test Ok
github-actions[bot] commented 5 days ago

:x: Forge suite framework_upgrade failure on 01b24e7e3548382dd25440b39a0438a993387f12 ==> 51229a000be3554b968d649e985815b9d876becd

Compatibility test results for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 51229a000be3554b968d649e985815b9d876becd (PR)
Upgrade the nodes to version: 51229a000be3554b968d649e985815b9d876becd
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1172.2054018075635 txn/s, submitted: 1174.1821394160079 txn/s, failed submission: 1.976737608444458 txn/s, expired: 1.976737608444458 txn/s, latency: 2530.673814877272 ms, (p50: 2100 ms, p90: 4200 ms, p99: 5900 ms), latency samples: 106740
Test Failed: API error: Unknown error error sending request for url (http://aptos-node-3-validator.forge-framework-upgrade-pr-13197.svc:8080/v1/accounts/0000000000000000000000000000000000000000000000000000000000000001/resource/0x1::block::BlockResource): error trying to connect: dns error: failed to lookup address information: Name or service not known

Stack backtrace:
   0: anyhow::error::<impl core::convert::From<E> for anyhow::Error>::from
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/anyhow-1.0.79/src/error.rs:565:25
   1: <core::result::Result<T,F> as core::ops::try_trait::FromResidual<core::result::Result<core::convert::Infallible,E>>>::from_residual
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/result.rs:1963:27
   2: aptos_release_builder::validate::NetworkConfig::increase_lockup::{{closure}}
             at ./aptos-move/aptos-release-builder/src/validate.rs:383:9
   3: aptos_release_builder::validate::execute_release::{{closure}}
             at ./aptos-move/aptos-release-builder/src/validate.rs:404:38
   4: aptos_release_builder::validate::validate_config_and_generate_release::{{closure}}
             at ./aptos-move/aptos-release-builder/src/validate.rs:493:6
   5: aptos_release_builder::validate::validate_config::{{closure}}
             at ./aptos-move/aptos-release-builder/src/validate.rs:479:80
   6: tokio::runtime::park::CachedParkThread::block_on::{{closure}}
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/park.rs:282:63
   7: tokio::runtime::coop::with_budget
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/coop.rs:107:5
   8: tokio::runtime::coop::budget
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/coop.rs:73:5
   9: tokio::runtime::park::CachedParkThread::block_on
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/park.rs:282:31
  10: tokio::runtime::context::blocking::BlockingRegionGuard::block_on
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/context/blocking.rs:66:9
  11: tokio::runtime::scheduler::multi_thread::MultiThread::block_on::{{closure}}
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/scheduler/multi_thread/mod.rs:87:13
  12: tokio::runtime::context::runtime::enter_runtime
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/context/runtime.rs:65:16
  13: tokio::runtime::scheduler::multi_thread::MultiThread::block_on
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/scheduler/multi_thread/mod.rs:86:9
  14: tokio::runtime::runtime::Runtime::block_on
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/runtime.rs:350:50
  15: <aptos_testcases::framework_upgrade::FrameworkUpgrade as aptos_forge::interface::network::NetworkTest>::run
             at ./testsuite/testcases/src/framework_upgrade.rs:112:9
  16: aptos_forge::runner::Forge<F>::run::{{closure}}
             at ./testsuite/forge/src/runner.rs:598:42
  17: aptos_forge::runner::run_test
             at ./testsuite/forge/src/runner.rs:666:11
  18: aptos_forge::runner::Forge<F>::run
             at ./testsuite/forge/src/runner.rs:598:30
  19: forge::run_forge
             at ./testsuite/forge-cli/src/main.rs:427:11
  20: forge::main
             at ./testsuite/forge-cli/src/main.rs:353:21
  21: core::ops::function::FnOnce::call_once
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/ops/function.rs:250:5
  22: std::sys_common::backtrace::__rust_begin_short_backtrace
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/sys_common/backtrace.rs:154:18
  23: std::rt::lang_start::{{closure}}
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:167:18
  24: core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/ops/function.rs:284:13
  25: std::panicking::try::do_call
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:552:40
  26: std::panicking::try
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:516:19
  27: std::panic::catch_unwind
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panic.rs:142:14
  28: std::rt::lang_start_internal::{{closure}}
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:148:48
  29: std::panicking::try::do_call
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:552:40
  30: std::panicking::try
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:516:19
  31: std::panic::catch_unwind
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panic.rs:142:14
  32: std::rt::lang_start_internal
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:148:20
  33: std::rt::lang_start
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:166:17
  34: __libc_start_main
  35: _start
Trailing Log Lines:
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:516:19
  31: std::panic::catch_unwind
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panic.rs:142:14
  32: std::rt::lang_start_internal
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:148:20
  33: std::rt::lang_start
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:166:17
  34: __libc_start_main
  35: _start

Swarm logs can be found here: See fgi output for more information.
thread 'main' panicked at testsuite/forge/src/backend/k8s/swarm.rs:696:18:
called `Result::unwrap()` on an `Err` value: ApiError: namespaces "forge-framework-upgrade-pr-13197" not found: NotFound (ErrorResponse { status: "Failure", message: "namespaces \"forge-framework-upgrade-pr-13197\" not found", reason: "NotFound", code: 404 })

Caused by:
    namespaces "forge-framework-upgrade-pr-13197" not found: NotFound

Stack backtrace:
   0: anyhow::error::<impl core::convert::From<E> for anyhow::Error>::from
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/anyhow-1.0.79/src/error.rs:565:25
   1: <core::result::Result<T,F> as core::ops::try_trait::FromResidual<core::result::Result<core::convert::Infallible,E>>>::from_residual
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/result.rs:1963:27
   2: aptos_forge::backend::k8s::cluster_helper::delete_k8s_cluster::{{closure}}
             at ./testsuite/forge/src/backend/k8s/cluster_helper.rs:289:13
   3: aptos_forge::backend::k8s::cluster_helper::uninstall_testnet_resources::{{closure}}
             at ./testsuite/forge/src/backend/k8s/cluster_helper.rs:399:48
   4: tokio::runtime::park::CachedParkThread::block_on::{{closure}}
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/park.rs:282:63
   5: tokio::runtime::coop::with_budget
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/coop.rs:107:5
   6: tokio::runtime::coop::budget
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/coop.rs:73:5
   7: tokio::runtime::park::CachedParkThread::block_on
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/park.rs:282:31
   8: tokio::runtime::context::blocking::BlockingRegionGuard::block_on
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/context/blocking.rs:66:9
   9: tokio::runtime::scheduler::multi_thread::MultiThread::block_on::{{closure}}
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/scheduler/multi_thread/mod.rs:87:13
  10: tokio::runtime::context::runtime::enter_runtime
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/context/runtime.rs:65:16
  11: tokio::runtime::scheduler::multi_thread::MultiThread::block_on
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/scheduler/multi_thread/mod.rs:86:9
  12: tokio::runtime::runtime::Runtime::block_on
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/runtime.rs:350:50
  13: <aptos_forge::backend::k8s::swarm::K8sSwarm as core::ops::drop::Drop>::drop
             at ./testsuite/forge/src/backend/k8s/swarm.rs:694:13
  14: core::ptr::drop_in_place<aptos_forge::backend::k8s::swarm::K8sSwarm>
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/ptr/mod.rs:498:1
  15: core::ptr::drop_in_place<alloc::boxed::Box<dyn aptos_forge::interface::swarm::Swarm>>
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/ptr/mod.rs:498:1
  16: aptos_forge::runner::Forge<F>::run
             at ./testsuite/forge/src/runner.rs:611:9
  17: forge::run_forge
             at ./testsuite/forge-cli/src/main.rs:427:11
  18: forge::main
             at ./testsuite/forge-cli/src/main.rs:353:21
  19: core::ops::function::FnOnce::call_once
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/ops/function.rs:250:5
  20: std::sys_common::backtrace::__rust_begin_short_backtrace
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/sys_common/backtrace.rs:154:18
  21: std::rt::lang_start::{{closure}}
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:167:18
  22: core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/ops/function.rs:284:13
  23: std::panicking::try::do_call
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:552:40
  24: std::panicking::try
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:516:19
  25: std::panic::catch_unwind
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panic.rs:142:14
  26: std::rt::lang_start_internal::{{closure}}
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:148:48
  27: std::panicking::try::do_call
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:552:40
  28: std::panicking::try
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:516:19
  29: std::panic::catch_unwind
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panic.rs:142:14
  30: std::rt::lang_start_internal
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:148:20
  31: std::rt::lang_start
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:166:17
  32: __libc_start_main
  33: _start
stack backtrace:
   0: rust_begin_unwind
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:645:5
   1: core::panicking::panic_fmt
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/panicking.rs:72:14
   2: core::result::unwrap_failed
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/result.rs:1653:5
   3: core::result::Result<T,E>::unwrap
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/result.rs:1077:23
   4: <aptos_forge::backend::k8s::swarm::K8sSwarm as core::ops::drop::Drop>::drop
             at ./testsuite/forge/src/backend/k8s/swarm.rs:694:13
   5: core::ptr::drop_in_place<aptos_forge::backend::k8s::swarm::K8sSwarm>
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/ptr/mod.rs:498:1
   6: core::ptr::drop_in_place<alloc::boxed::Box<dyn aptos_forge::interface::swarm::Swarm>>
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/ptr/mod.rs:498:1
   7: aptos_forge::runner::Forge<F>::run
             at ./testsuite/forge/src/runner.rs:611:9
   8: forge::run_forge
             at ./testsuite/forge-cli/src/main.rs:427:11
   9: forge::main
             at ./testsuite/forge-cli/src/main.rs:353:21
  10: core::ops::function::FnOnce::call_once
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/ops/function.rs:250:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
Debugging output:
github-actions[bot] commented 5 days ago

Forge is running suite framework_upgrade on 01b24e7e3548382dd25440b39a0438a993387f12 ==> 51229a000be3554b968d649e985815b9d876becd