search

aptos-labs / aptos-core

Aptos is a layer 1 blockchain built to support the widespread use of blockchain through better technology and user experience.
https://aptosfoundation.org
Other
5.85k stars 3.54k forks source link

Run tests only on primary targets. #13184

Closed brmataptos closed 2 weeks ago

brmataptos commented 2 weeks ago

Description

Run tests only on root sources. Fixes #12958. Fixes #12957. Replaces PR #13125.

Distinguish between "target" (to be compiled) and "primary_target" (to be tested/analyzed/etc.) in move-model, so that tests can be run on just the primary source inputs. This may eventually be renamed, but this PR is big enough already without global renames.

Primary targets are distinguished by keeping sources separate on their way to the move model through various interfaces,

The names vary to be clearly distinct from existing names in context, which means they vary across the whole code base. In summary:

In move-model internals (model.rs), the is_dep property is negated as is_target (to be compiled), while the root packages (to be tested or whatever) are identified as is_prime_target.

In the future these might become is_compilation_target and is_target or something but that can easily be done in a followup PR once we settle on the names.

How Has This Been Tested?

Tested by running all move-compiler* tests, and then all other third_party tests as well as aptos-move-examples and aptos-transactional-test-harness both with and without MOVE_COMPILER_V2=true. For the latter case, the test frameworks were fixed to put v2 outputs into .v2_exp.

Currently, all .v2_exp test outputs seem compatible except for

Key Areas to Review

You will want to filter tests out initially and look at .rs files. The key change is the new Vec parameter to pass source dependencies around (distinguishing them from either primary sources or bytecode dependencies), are processed in various interface functions in move-package/src/compilation/compiled_package.rs and move-model/src/lib.rs and then are passed to GlobalEnv::add_source in model.rs tagged as is_target and/or is_primary_target.

(Tests show that occasionally sources wind up included multiple times, perhaps through dependency chains, so the code for GlobalEnv::add_source is a little more complicated now to handle this case conservatively.)

Getting all tests to run led to some harness changes:

transactional-test-runner/src/vm_test_harness.rs and aptos-transactional-test-harness/src/aptos_test_harness.rs were modified to generate output as EXP_EXT_V2=".v2_exp" if MOVE_COMPILER_V2 is set in the environment. This led to a few .v2_exp test outputs that included multiple temp filenames. My previous hack to handle a single temp filename was insufficient, so I generalized that in transactional-test-runner/src/framework.rs.

One disassembler test also had issues due to script name mangling in V2, so move-cli/src/base/disassemble.rs was fixed to use a new move_model::ast::ModuleName::pseudo-script_name_builder() function which abstracts the details of that name mangling.

Type of Change

Which Components or Systems Does This Change Impact?

Checklist

Commit Listing

trunk-io[bot] commented 2 weeks ago
⏱️ 32h 45m total CI duration on this PR | Job | Cumulative Duration | Recent Runs | |---|---|---| | [forge-framework-upgrade-test / forge](https://github.com/aptos-labs/aptos-core/actions/runs/8946480216/job/24577429495) | 5h 5m | [⬜](https://github.com/aptos-labs/aptos-core/actions/runs/8944928457/job/24572987683) [⬜](https://github.com/aptos-labs/aptos-core/actions/runs/8945008278/job/24573487122) [🟥](https://github.com/aptos-labs/aptos-core/actions/runs/8945008278/job/24575024227) [⬜](https://github.com/aptos-labs/aptos-core/actions/runs/8945008278/job/24577159439) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8946480216/job/24577429495) (+3 more) | | [windows-build](https://github.com/aptos-labs/aptos-core/actions/runs/8946480166/job/24577178734) | 4h 11m | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8943834340/job/24569397374) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8944346798/job/24571071979) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8944928471/job/24572882794) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8945008430/job/24573122500) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8946480166/job/24577178734) (+2 more) | | [rust-targeted-unit-tests](https://github.com/aptos-labs/aptos-core/actions/runs/8946480165/job/24577188081) | 3h 38m | [🟥](https://github.com/aptos-labs/aptos-core/actions/runs/8943834344/job/24569418921) [🟥](https://github.com/aptos-labs/aptos-core/actions/runs/8944346785/job/24571085547) [⬜](https://github.com/aptos-labs/aptos-core/actions/runs/8944928482/job/24572894725) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8945008436/job/24573143445) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8946480165/job/24577188081) (+6 more) | | [rust-move-unit-coverage](https://github.com/aptos-labs/aptos-core/actions/runs/8946480156/job/24577179215) | 2h 57m | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8943834343/job/24569408210) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8944346800/job/24571072639) [⬜](https://github.com/aptos-labs/aptos-core/actions/runs/8944928470/job/24572883676) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8945008414/job/24573131564) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8946480156/job/24577179215) (+5 more) | | [rust-smoke-tests](https://github.com/aptos-labs/aptos-core/actions/runs/8946480165/job/24577187878) | 2h 54m | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8943834344/job/24569419108) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8944346785/job/24571085278) [⬜](https://github.com/aptos-labs/aptos-core/actions/runs/8944928482/job/24572894251) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8945008436/job/24573143161) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8946480165/job/24577187878) (+2 more) | | [execution-performance / single-node-performance](https://github.com/aptos-labs/aptos-core/actions/runs/8946480187/job/24577188525) | 2h 22m | [🟥](https://github.com/aptos-labs/aptos-core/actions/runs/8944346823/job/24571082891) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8944346823/job/24571826695) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8944928494/job/24572892941) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8945008437/job/24573132214) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8946480187/job/24577188525) (+3 more) | | [rust-move-tests](https://github.com/aptos-labs/aptos-core/actions/runs/8946480161/job/24577179324) | 1h 49m | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8943834348/job/24569408133) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8944346796/job/24571072623) [⬜](https://github.com/aptos-labs/aptos-core/actions/runs/8944928474/job/24572883963) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8945008418/job/24573131649) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8946480161/job/24577179324) (+6 more) | | [forge-e2e-test / forge](https://github.com/aptos-labs/aptos-core/actions/runs/8946480216/job/24577429134) | 1h 32m | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8944346818/job/24571509855) [⬜](https://github.com/aptos-labs/aptos-core/actions/runs/8944928457/job/24572963813) [🟥](https://github.com/aptos-labs/aptos-core/actions/runs/8945008278/job/24573487279) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8945008278/job/24575024059) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8946480216/job/24577429134) (+2 more) | | [rust-unit-tests](https://github.com/aptos-labs/aptos-core/actions/runs/8943834344/job/24569419520) | 1h 19m | [🟥](https://github.com/aptos-labs/aptos-core/actions/runs/8943063915/job/24567001239) [⬜](https://github.com/aptos-labs/aptos-core/actions/runs/8943783421/job/24569256189) [🟥](https://github.com/aptos-labs/aptos-core/actions/runs/8943834344/job/24569419520)  | | [forge-compat-test / forge](https://github.com/aptos-labs/aptos-core/actions/runs/8946480216/job/24577429029) | 1h 11m | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8943834269/job/24569817449) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8944346818/job/24571510036) [⬜](https://github.com/aptos-labs/aptos-core/actions/runs/8944928457/job/24572964061) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8945008278/job/24573486934) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8946480216/job/24577429029) (+1 more) | | [rust-images / rust-all](https://github.com/aptos-labs/aptos-core/actions/runs/8946480216/job/24577192656) | 1h 8m | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8943834269/job/24569425112) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8944346818/job/24571098448) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8944928457/job/24572895220) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8945008278/job/24573143613) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8946480216/job/24577192656) (+2 more) | | [rust-lints](https://github.com/aptos-labs/aptos-core/actions/runs/8946480165/job/24577187477) | 58m | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8943834344/job/24569418349) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8944346785/job/24571084571) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8944928482/job/24572893863) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8945008436/job/24573142995) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8946480165/job/24577187477) (+6 more) | | [cli-e2e-tests / run-cli-tests](https://github.com/aptos-labs/aptos-core/actions/runs/8946480216/job/24577428912) | 43m | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8943834269/job/24569816830) [🟥](https://github.com/aptos-labs/aptos-core/actions/runs/8944346818/job/24571509367) [⬜](https://github.com/aptos-labs/aptos-core/actions/runs/8944928457/job/24572963585) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8945008278/job/24573486330) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8946480216/job/24577428912) (+1 more) | | [run-tests-main-branch](https://github.com/aptos-labs/aptos-core/actions/runs/8946480152/job/24577189868) | 42m | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8943834248/job/24569414026) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8944346786/job/24571087543) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8944928438/job/24572900092) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8945008249/job/24573136630) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8946480152/job/24577189868) (+6 more) | | [rust-build-cached-packages](https://github.com/aptos-labs/aptos-core/actions/runs/8946480165/job/24577188224) | 29m | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8943834344/job/24569419321) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8944346785/job/24571085067) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8944928482/job/24572894881) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8945008436/job/24573143296) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8946480165/job/24577188224) (+2 more) | | [check](https://github.com/aptos-labs/aptos-core/actions/runs/8946480162/job/24577179125) | 29m | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8943834345/job/24569397826) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8944346792/job/24571072783) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8944928477/job/24572883857) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8945008427/job/24573122847) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8946480162/job/24577179125) (+2 more) | | [framework-upgrade-determinator](https://github.com/aptos-labs/aptos-core/actions/runs/8946480216/job/24577189515) | 23m | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8943834269/job/24569418265) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8944346818/job/24571093010) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8944928457/job/24572889839) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8945008278/job/24573138266) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8946480216/job/24577189515) (+2 more) | | [general-lints](https://github.com/aptos-labs/aptos-core/actions/runs/8946480165/job/24577187745) | 22m | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8943834344/job/24569418551) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8944346785/job/24571084871) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8944928482/job/24572894064) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8945008436/job/24573142823) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8946480165/job/24577187745) (+6 more) | | [check-dynamic-deps](https://github.com/aptos-labs/aptos-core/actions/runs/8946480159/job/24577178831) | 14m | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8943834339/job/24569397212) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8944346797/job/24571072045) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8944928465/job/24572882785) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8945008420/job/24573122268) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8946480159/job/24577178831) (+6 more) | | [node-api-compatibility-tests / node-api-compatibility-tests](https://github.com/aptos-labs/aptos-core/actions/runs/8946480216/job/24577429397) | 5m | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8943834269/job/24569817601) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8944346818/job/24571509539) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8944928457/job/24572964282) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8945008278/job/24573486533) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8946480216/job/24577429397) (+1 more) | | [semgrep/ci](https://github.com/aptos-labs/aptos-core/actions/runs/8946480148/job/24577178794) | 5m | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8943834350/job/24569397428) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8944346807/job/24571071991) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8944928468/job/24572883003) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8945008410/job/24573122287) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8946480148/job/24577178794) (+6 more) | | [file_change_determinator](https://github.com/aptos-labs/aptos-core/actions/runs/8946480165/job/24577179317) | 2m | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8943834344/job/24569408452) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8944346785/job/24571072862) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8944928482/job/24572883353) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8945008436/job/24573131864) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8946480165/job/24577179317) (+6 more) | | [file_change_determinator](https://github.com/aptos-labs/aptos-core/actions/runs/8946480152/job/24577182078) | 2m | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8943834248/job/24569405254) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8944346786/job/24571077643) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8944928438/job/24572891034) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8945008249/job/24573128887) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8946480152/job/24577182078) (+6 more) | | [file_change_determinator](https://github.com/aptos-labs/aptos-core/actions/runs/8946480216/job/24577189366) | 1m | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8943834269/job/24569418046) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8944346818/job/24571093205) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8944928457/job/24572890026) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8945008278/job/24573138689) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8946480216/job/24577189366) (+2 more) | | [execution-performance / file_change_determinator](https://github.com/aptos-labs/aptos-core/actions/runs/8946480187/job/24577179152) | 1m | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8943834370/job/24569397880) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8944346823/job/24571072380) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8944928494/job/24572883561) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8945008437/job/24573122806) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8946480187/job/24577179152) (+2 more) | | [permission-check](https://github.com/aptos-labs/aptos-core/actions/runs/8946480152/job/24577179033) | 44s | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8943834248/job/24569397235) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8944346786/job/24571072796) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8944928438/job/24572884129) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8945008249/job/24573121988) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8946480152/job/24577179033) (+6 more) | | [permission-check](https://github.com/aptos-labs/aptos-core/actions/runs/8946480151/job/24577178892) | 39s | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8943834254/job/24569396880) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8944346790/job/24571071941) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8944928432/job/24572882792) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8945008251/job/24573121710) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8946480151/job/24577178892) (+6 more) | | [permission-check](https://github.com/aptos-labs/aptos-core/actions/runs/8946480157/job/24577179264) | 38s | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8943834246/job/24569397108) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8944346791/job/24571072757) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8944928433/job/24572883300) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8945008256/job/24573122091) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8946480157/job/24577179264) (+6 more) | | [permission-check](https://github.com/aptos-labs/aptos-core/actions/runs/8946480147/job/24577178886) | 37s | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8943834262/job/24569396863) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8944346789/job/24571072189) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8944928436/job/24572882800) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8945008263/job/24573121726) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8946480147/job/24577178886) (+6 more) | | [permission-check](https://github.com/aptos-labs/aptos-core/actions/runs/8946480216/job/24577182673) | 28s | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8943834269/job/24569410267) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8944346818/job/24571086590) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8944928457/job/24572884229) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8945008278/job/24573132845) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8946480216/job/24577182673) (+2 more) | | [determine-docker-build-metadata](https://github.com/aptos-labs/aptos-core/actions/runs/8946480216/job/24577189186) | 21s | [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8943834269/job/24569417838) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8944346818/job/24571092791) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8944928457/job/24572890195) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8945008278/job/24573138042) [🟩](https://github.com/aptos-labs/aptos-core/actions/runs/8946480216/job/24577189186) (+2 more) |

🚨 3 jobs on the last run were significantly faster/slower than expected

Job Duration vs 7d avg Delta
rust-targeted-unit-tests 22m 14m +58%
rust-move-tests 12m 8m +42%
rust-unit-tests 38m 29m +29%

settings ⋅ feedback ⋅ docs ⋅ learn more about trunk.io

codecov[bot] commented 2 weeks ago

Codecov Report

Attention: Patch coverage is 79.87013% with 31 lines in your changes are missing coverage. Please review.

Project coverage is 57.6%. Comparing base (8aa625d) to head (6884168). Report is 5 commits behind head on main.

Files Patch % Lines
...s/move-package/src/compilation/compiled_package.rs 25.0% 9 Missing :warning:
third_party/move/move-model/src/model.rs 84.3% 5 Missing :warning:
...a/transactional-test-runner/src/vm_test_harness.rs 60.0% 4 Missing :warning:
..._party/move/tools/move-cli/src/base/disassemble.rs 42.8% 4 Missing :warning:
third_party/move/move-model/src/lib.rs 91.1% 3 Missing :warning:
third_party/move/evm/move-to-yul/src/lib.rs 0.0% 2 Missing :warning:
...g-infra/transactional-test-runner/src/framework.rs 94.2% 2 Missing :warning:
...rd_party/move/move-compiler-v2/src/plan_builder.rs 0.0% 1 Missing :warning:
...e/move-prover/boogie-backend/src/boogie_wrapper.rs 0.0% 1 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #13184 +/- ## ======================================== Coverage 57.6% 57.6% ======================================== Files 834 834 Lines 198264 198369 +105 ======================================== + Hits 114212 114306 +94 - Misses 84052 84063 +11 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

github-actions[bot] commented 2 weeks ago

Forge is running suite framework_upgrade on 01b24e7e3548382dd25440b39a0438a993387f12 ==> b4eb7a0bc64f73527c56d85aa165e85a25ea8234

github-actions[bot] commented 2 weeks ago

Forge is running suite realistic_env_max_load on b4eb7a0bc64f73527c56d85aa165e85a25ea8234

github-actions[bot] commented 2 weeks ago

Forge is running suite compat on 01b24e7e3548382dd25440b39a0438a993387f12 ==> b4eb7a0bc64f73527c56d85aa165e85a25ea8234

github-actions[bot] commented 2 weeks ago

:white_check_mark: Forge suite realistic_env_max_load success on b4eb7a0bc64f73527c56d85aa165e85a25ea8234

two traffics test: inner traffic : committed: 8020 txn/s, latency: 4900 ms, (p50: 4800 ms, p90: 5700 ms, p99: 10200 ms), latency samples: 3456740
two traffics test : committed: 100 txn/s, latency: 1981 ms, (p50: 1800 ms, p90: 2200 ms, p99: 7500 ms), latency samples: 1800
Latency breakdown for phase 0: ["QsBatchToPos: max: 0.207, avg: 0.202", "QsPosToProposal: max: 0.253, avg: 0.233", "ConsensusProposalToOrdered: max: 0.452, avg: 0.417", "ConsensusOrderedToCommit: max: 0.374, avg: 0.360", "ConsensusProposalToCommit: max: 0.791, avg: 0.777"]
Max round gap was 1 [limit 4] at version 1729915. Max no progress secs was 4.573969 [limit 15] at version 1729915.
Test Ok
github-actions[bot] commented 2 weeks ago

:white_check_mark: Forge suite compat success on 01b24e7e3548382dd25440b39a0438a993387f12 ==> b4eb7a0bc64f73527c56d85aa165e85a25ea8234

Compatibility test results for 01b24e7e3548382dd25440b39a0438a993387f12 ==> b4eb7a0bc64f73527c56d85aa165e85a25ea8234 (PR)
1. Check liveness of validators at old version: 01b24e7e3548382dd25440b39a0438a993387f12
compatibility::simple-validator-upgrade::liveness-check : committed: 6036 txn/s, latency: 5147 ms, (p50: 4800 ms, p90: 9600 ms, p99: 13200 ms), latency samples: 235420
2. Upgrading first Validator to new version: b4eb7a0bc64f73527c56d85aa165e85a25ea8234
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 1720 txn/s, latency: 15941 ms, (p50: 19000 ms, p90: 22700 ms, p99: 23800 ms), latency samples: 91180
3. Upgrading rest of first batch to new version: b4eb7a0bc64f73527c56d85aa165e85a25ea8234
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 1611 txn/s, latency: 15983 ms, (p50: 18700 ms, p90: 22100 ms, p99: 27800 ms), latency samples: 91840
4. upgrading second batch to new version: b4eb7a0bc64f73527c56d85aa165e85a25ea8234
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 3536 txn/s, latency: 8902 ms, (p50: 9600 ms, p90: 12600 ms, p99: 13000 ms), latency samples: 145000
5. check swarm health
Compatibility test for 01b24e7e3548382dd25440b39a0438a993387f12 ==> b4eb7a0bc64f73527c56d85aa165e85a25ea8234 passed
Test Ok
github-actions[bot] commented 2 weeks ago

Forge is running suite compat on 01b24e7e3548382dd25440b39a0438a993387f12 ==> d088616fadb904bbdbc0d7dcec826166a24ba31a

github-actions[bot] commented 2 weeks ago

Forge is running suite realistic_env_max_load on d088616fadb904bbdbc0d7dcec826166a24ba31a

github-actions[bot] commented 2 weeks ago

Forge is running suite framework_upgrade on 01b24e7e3548382dd25440b39a0438a993387f12 ==> d088616fadb904bbdbc0d7dcec826166a24ba31a

github-actions[bot] commented 2 weeks ago

:white_check_mark: Forge suite compat success on 01b24e7e3548382dd25440b39a0438a993387f12 ==> d088616fadb904bbdbc0d7dcec826166a24ba31a

Compatibility test results for 01b24e7e3548382dd25440b39a0438a993387f12 ==> d088616fadb904bbdbc0d7dcec826166a24ba31a (PR)
1. Check liveness of validators at old version: 01b24e7e3548382dd25440b39a0438a993387f12
compatibility::simple-validator-upgrade::liveness-check : committed: 6698 txn/s, latency: 4883 ms, (p50: 4800 ms, p90: 8100 ms, p99: 10200 ms), latency samples: 241160
2. Upgrading first Validator to new version: d088616fadb904bbdbc0d7dcec826166a24ba31a
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 1761 txn/s, latency: 16178 ms, (p50: 19100 ms, p90: 22300 ms, p99: 22800 ms), latency samples: 91620
3. Upgrading rest of first batch to new version: d088616fadb904bbdbc0d7dcec826166a24ba31a
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 1727 txn/s, latency: 16796 ms, (p50: 20200 ms, p90: 22200 ms, p99: 22600 ms), latency samples: 86380
4. upgrading second batch to new version: d088616fadb904bbdbc0d7dcec826166a24ba31a
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 3470 txn/s, latency: 9230 ms, (p50: 9700 ms, p90: 12600 ms, p99: 12900 ms), latency samples: 145740
5. check swarm health
Compatibility test for 01b24e7e3548382dd25440b39a0438a993387f12 ==> d088616fadb904bbdbc0d7dcec826166a24ba31a passed
Test Ok
github-actions[bot] commented 2 weeks ago

:white_check_mark: Forge suite realistic_env_max_load success on d088616fadb904bbdbc0d7dcec826166a24ba31a

two traffics test: inner traffic : committed: 7843 txn/s, latency: 4992 ms, (p50: 4800 ms, p90: 6000 ms, p99: 10800 ms), latency samples: 3396320
two traffics test : committed: 100 txn/s, latency: 1908 ms, (p50: 1900 ms, p90: 2100 ms, p99: 2500 ms), latency samples: 1760
Latency breakdown for phase 0: ["QsBatchToPos: max: 0.202, avg: 0.201", "QsPosToProposal: max: 0.246, avg: 0.232", "ConsensusProposalToOrdered: max: 0.433, avg: 0.423", "ConsensusOrderedToCommit: max: 0.379, avg: 0.360", "ConsensusProposalToCommit: max: 0.793, avg: 0.783"]
Max round gap was 1 [limit 4] at version 1703636. Max no progress secs was 4.659287 [limit 15] at version 1703636.
Test Ok
github-actions[bot] commented 2 weeks ago

Forge is running suite framework_upgrade on 01b24e7e3548382dd25440b39a0438a993387f12 ==> 65d82549c6cfe342b10c889443e5fd708a49fea1

github-actions[bot] commented 2 weeks ago

Forge is running suite compat on 01b24e7e3548382dd25440b39a0438a993387f12 ==> 65d82549c6cfe342b10c889443e5fd708a49fea1

github-actions[bot] commented 2 weeks ago

Forge is running suite realistic_env_max_load on 65d82549c6cfe342b10c889443e5fd708a49fea1

github-actions[bot] commented 2 weeks ago

:white_check_mark: Forge suite compat success on 01b24e7e3548382dd25440b39a0438a993387f12 ==> 65d82549c6cfe342b10c889443e5fd708a49fea1

Compatibility test results for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 65d82549c6cfe342b10c889443e5fd708a49fea1 (PR)
1. Check liveness of validators at old version: 01b24e7e3548382dd25440b39a0438a993387f12
compatibility::simple-validator-upgrade::liveness-check : committed: 5722 txn/s, latency: 5282 ms, (p50: 4800 ms, p90: 9800 ms, p99: 16000 ms), latency samples: 234640
2. Upgrading first Validator to new version: 65d82549c6cfe342b10c889443e5fd708a49fea1
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 1773 txn/s, latency: 16461 ms, (p50: 17900 ms, p90: 24100 ms, p99: 24400 ms), latency samples: 88660
3. Upgrading rest of first batch to new version: 65d82549c6cfe342b10c889443e5fd708a49fea1
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 1470 txn/s, latency: 19695 ms, (p50: 17100 ms, p90: 30800 ms, p99: 33500 ms), latency samples: 89680
4. upgrading second batch to new version: 65d82549c6cfe342b10c889443e5fd708a49fea1
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 3675 txn/s, latency: 8698 ms, (p50: 9800 ms, p90: 12400 ms, p99: 12900 ms), latency samples: 143340
5. check swarm health
Compatibility test for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 65d82549c6cfe342b10c889443e5fd708a49fea1 passed
Test Ok
github-actions[bot] commented 2 weeks ago

:white_check_mark: Forge suite realistic_env_max_load success on 65d82549c6cfe342b10c889443e5fd708a49fea1

two traffics test: inner traffic : committed: 7959 txn/s, latency: 4928 ms, (p50: 4800 ms, p90: 5700 ms, p99: 11700 ms), latency samples: 3438580
two traffics test : committed: 100 txn/s, latency: 1871 ms, (p50: 1800 ms, p90: 2100 ms, p99: 5200 ms), latency samples: 1740
Latency breakdown for phase 0: ["QsBatchToPos: max: 0.208, avg: 0.201", "QsPosToProposal: max: 0.249, avg: 0.234", "ConsensusProposalToOrdered: max: 0.431, avg: 0.415", "ConsensusOrderedToCommit: max: 0.380, avg: 0.362", "ConsensusProposalToCommit: max: 0.800, avg: 0.777"]
Max round gap was 1 [limit 4] at version 1753882. Max no progress secs was 4.95678 [limit 15] at version 1753882.
Test Ok
github-actions[bot] commented 2 weeks ago

:x: Forge suite framework_upgrade failure on 01b24e7e3548382dd25440b39a0438a993387f12 ==> 65d82549c6cfe342b10c889443e5fd708a49fea1

Compatibility test results for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 65d82549c6cfe342b10c889443e5fd708a49fea1 (PR)
Upgrade the nodes to version: 65d82549c6cfe342b10c889443e5fd708a49fea1
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1292 txn/s, submitted: 1293 txn/s, failed submission: 1 txn/s, expired: 1 txn/s, latency: 2431 ms, (p50: 2100 ms, p90: 4100 ms, p99: 6900 ms), latency samples: 115020
Test Failed: API error: Unknown error error sending request for url (http://aptos-node-3-validator.forge-framework-upgrade-pr-13184.svc:8080/v1/accounts/0000000000000000000000000000000000000000000000000000000000000001/resource/0x1::block::BlockResource): error trying to connect: dns error: failed to lookup address information: Name or service not known

Stack backtrace:
   0: anyhow::error::<impl core::convert::From<E> for anyhow::Error>::from
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/anyhow-1.0.79/src/error.rs:565:25
   1: <core::result::Result<T,F> as core::ops::try_trait::FromResidual<core::result::Result<core::convert::Infallible,E>>>::from_residual
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/result.rs:1963:27
   2: aptos_release_builder::validate::NetworkConfig::increase_lockup::{{closure}}
             at ./aptos-move/aptos-release-builder/src/validate.rs:383:9
   3: aptos_release_builder::validate::execute_release::{{closure}}
             at ./aptos-move/aptos-release-builder/src/validate.rs:404:38
   4: aptos_release_builder::validate::validate_config_and_generate_release::{{closure}}
             at ./aptos-move/aptos-release-builder/src/validate.rs:493:6
   5: aptos_release_builder::validate::validate_config::{{closure}}
             at ./aptos-move/aptos-release-builder/src/validate.rs:479:80
   6: tokio::runtime::park::CachedParkThread::block_on::{{closure}}
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/park.rs:282:63
   7: tokio::runtime::coop::with_budget
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/coop.rs:107:5
   8: tokio::runtime::coop::budget
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/coop.rs:73:5
   9: tokio::runtime::park::CachedParkThread::block_on
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/park.rs:282:31
  10: tokio::runtime::context::blocking::BlockingRegionGuard::block_on
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/context/blocking.rs:66:9
  11: tokio::runtime::scheduler::multi_thread::MultiThread::block_on::{{closure}}
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/scheduler/multi_thread/mod.rs:87:13
  12: tokio::runtime::context::runtime::enter_runtime
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/context/runtime.rs:65:16
  13: tokio::runtime::scheduler::multi_thread::MultiThread::block_on
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/scheduler/multi_thread/mod.rs:86:9
  14: tokio::runtime::runtime::Runtime::block_on
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/runtime.rs:350:50
  15: <aptos_testcases::framework_upgrade::FrameworkUpgrade as aptos_forge::interface::network::NetworkTest>::run
             at ./testsuite/testcases/src/framework_upgrade.rs:112:9
  16: aptos_forge::runner::Forge<F>::run::{{closure}}
             at ./testsuite/forge/src/runner.rs:598:42
  17: aptos_forge::runner::run_test
             at ./testsuite/forge/src/runner.rs:666:11
  18: aptos_forge::runner::Forge<F>::run
             at ./testsuite/forge/src/runner.rs:598:30
  19: forge::run_forge
             at ./testsuite/forge-cli/src/main.rs:427:11
  20: forge::main
             at ./testsuite/forge-cli/src/main.rs:353:21
  21: core::ops::function::FnOnce::call_once
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/ops/function.rs:250:5
  22: std::sys_common::backtrace::__rust_begin_short_backtrace
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/sys_common/backtrace.rs:154:18
  23: std::rt::lang_start::{{closure}}
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:167:18
  24: core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/ops/function.rs:284:13
  25: std::panicking::try::do_call
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:552:40
  26: std::panicking::try
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:516:19
  27: std::panic::catch_unwind
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panic.rs:142:14
  28: std::rt::lang_start_internal::{{closure}}
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:148:48
  29: std::panicking::try::do_call
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:552:40
  30: std::panicking::try
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:516:19
  31: std::panic::catch_unwind
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panic.rs:142:14
  32: std::rt::lang_start_internal
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:148:20
  33: std::rt::lang_start
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:166:17
  34: __libc_start_main
  35: _start
Trailing Log Lines:
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:516:19
  31: std::panic::catch_unwind
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panic.rs:142:14
  32: std::rt::lang_start_internal
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:148:20
  33: std::rt::lang_start
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:166:17
  34: __libc_start_main
  35: _start

Swarm logs can be found here: See fgi output for more information.
thread 'main' panicked at testsuite/forge/src/backend/k8s/swarm.rs:696:18:
called `Result::unwrap()` on an `Err` value: ApiError: namespaces "forge-framework-upgrade-pr-13184" not found: NotFound (ErrorResponse { status: "Failure", message: "namespaces \"forge-framework-upgrade-pr-13184\" not found", reason: "NotFound", code: 404 })

Caused by:
    namespaces "forge-framework-upgrade-pr-13184" not found: NotFound

Stack backtrace:
   0: anyhow::error::<impl core::convert::From<E> for anyhow::Error>::from
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/anyhow-1.0.79/src/error.rs:565:25
   1: <core::result::Result<T,F> as core::ops::try_trait::FromResidual<core::result::Result<core::convert::Infallible,E>>>::from_residual
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/result.rs:1963:27
   2: aptos_forge::backend::k8s::cluster_helper::delete_k8s_cluster::{{closure}}
             at ./testsuite/forge/src/backend/k8s/cluster_helper.rs:289:13
   3: aptos_forge::backend::k8s::cluster_helper::uninstall_testnet_resources::{{closure}}
             at ./testsuite/forge/src/backend/k8s/cluster_helper.rs:399:48
   4: tokio::runtime::park::CachedParkThread::block_on::{{closure}}
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/park.rs:282:63
   5: tokio::runtime::coop::with_budget
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/coop.rs:107:5
   6: tokio::runtime::coop::budget
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/coop.rs:73:5
   7: tokio::runtime::park::CachedParkThread::block_on
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/park.rs:282:31
   8: tokio::runtime::context::blocking::BlockingRegionGuard::block_on
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/context/blocking.rs:66:9
   9: tokio::runtime::scheduler::multi_thread::MultiThread::block_on::{{closure}}
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/scheduler/multi_thread/mod.rs:87:13
  10: tokio::runtime::context::runtime::enter_runtime
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/context/runtime.rs:65:16
  11: tokio::runtime::scheduler::multi_thread::MultiThread::block_on
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/scheduler/multi_thread/mod.rs:86:9
  12: tokio::runtime::runtime::Runtime::block_on
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/runtime.rs:350:50
  13: <aptos_forge::backend::k8s::swarm::K8sSwarm as core::ops::drop::Drop>::drop
             at ./testsuite/forge/src/backend/k8s/swarm.rs:694:13
  14: core::ptr::drop_in_place<aptos_forge::backend::k8s::swarm::K8sSwarm>
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/ptr/mod.rs:498:1
  15: core::ptr::drop_in_place<alloc::boxed::Box<dyn aptos_forge::interface::swarm::Swarm>>
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/ptr/mod.rs:498:1
  16: aptos_forge::runner::Forge<F>::run
             at ./testsuite/forge/src/runner.rs:611:9
  17: forge::run_forge
             at ./testsuite/forge-cli/src/main.rs:427:11
  18: forge::main
             at ./testsuite/forge-cli/src/main.rs:353:21
  19: core::ops::function::FnOnce::call_once
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/ops/function.rs:250:5
  20: std::sys_common::backtrace::__rust_begin_short_backtrace
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/sys_common/backtrace.rs:154:18
  21: std::rt::lang_start::{{closure}}
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:167:18
  22: core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/ops/function.rs:284:13
  23: std::panicking::try::do_call
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:552:40
  24: std::panicking::try
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:516:19
  25: std::panic::catch_unwind
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panic.rs:142:14
  26: std::rt::lang_start_internal::{{closure}}
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:148:48
  27: std::panicking::try::do_call
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:552:40
  28: std::panicking::try
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:516:19
  29: std::panic::catch_unwind
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panic.rs:142:14
  30: std::rt::lang_start_internal
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:148:20
  31: std::rt::lang_start
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:166:17
  32: __libc_start_main
  33: _start
stack backtrace:
   0: rust_begin_unwind
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:645:5
   1: core::panicking::panic_fmt
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/panicking.rs:72:14
   2: core::result::unwrap_failed
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/result.rs:1653:5
   3: core::result::Result<T,E>::unwrap
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/result.rs:1077:23
   4: <aptos_forge::backend::k8s::swarm::K8sSwarm as core::ops::drop::Drop>::drop
             at ./testsuite/forge/src/backend/k8s/swarm.rs:694:13
   5: core::ptr::drop_in_place<aptos_forge::backend::k8s::swarm::K8sSwarm>
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/ptr/mod.rs:498:1
   6: core::ptr::drop_in_place<alloc::boxed::Box<dyn aptos_forge::interface::swarm::Swarm>>
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/ptr/mod.rs:498:1
   7: aptos_forge::runner::Forge<F>::run
             at ./testsuite/forge/src/runner.rs:611:9
   8: forge::run_forge
             at ./testsuite/forge-cli/src/main.rs:427:11
   9: forge::main
             at ./testsuite/forge-cli/src/main.rs:353:21
  10: core::ops::function::FnOnce::call_once
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/ops/function.rs:250:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
Debugging output:
github-actions[bot] commented 2 weeks ago

Forge is running suite compat on 01b24e7e3548382dd25440b39a0438a993387f12 ==> 043afef146ea5af28c17a4215dabcef8e255e21d

github-actions[bot] commented 2 weeks ago

Forge is running suite realistic_env_max_load on 043afef146ea5af28c17a4215dabcef8e255e21d

github-actions[bot] commented 2 weeks ago

Forge is running suite framework_upgrade on 01b24e7e3548382dd25440b39a0438a993387f12 ==> 043afef146ea5af28c17a4215dabcef8e255e21d

github-actions[bot] commented 2 weeks ago

Forge is running suite realistic_env_max_load on 1c769ce5c52f8c3a265b6cefdce9a4831192e924

github-actions[bot] commented 2 weeks ago

Forge is running suite compat on 01b24e7e3548382dd25440b39a0438a993387f12 ==> 1c769ce5c52f8c3a265b6cefdce9a4831192e924

github-actions[bot] commented 2 weeks ago

Forge is running suite framework_upgrade on 01b24e7e3548382dd25440b39a0438a993387f12 ==> 1c769ce5c52f8c3a265b6cefdce9a4831192e924

github-actions[bot] commented 2 weeks ago

:white_check_mark: Forge suite compat success on 01b24e7e3548382dd25440b39a0438a993387f12 ==> 1c769ce5c52f8c3a265b6cefdce9a4831192e924

Compatibility test results for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 1c769ce5c52f8c3a265b6cefdce9a4831192e924 (PR)
1. Check liveness of validators at old version: 01b24e7e3548382dd25440b39a0438a993387f12
compatibility::simple-validator-upgrade::liveness-check : committed: 6258 txn/s, latency: 5207 ms, (p50: 4800 ms, p90: 9300 ms, p99: 12200 ms), latency samples: 225320
2. Upgrading first Validator to new version: 1c769ce5c52f8c3a265b6cefdce9a4831192e924
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 1283 txn/s, latency: 19583 ms, (p50: 25000 ms, p90: 29800 ms, p99: 30600 ms), latency samples: 77020
3. Upgrading rest of first batch to new version: 1c769ce5c52f8c3a265b6cefdce9a4831192e924
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 1782 txn/s, latency: 16304 ms, (p50: 19000 ms, p90: 22500 ms, p99: 22900 ms), latency samples: 92680
4. upgrading second batch to new version: 1c769ce5c52f8c3a265b6cefdce9a4831192e924
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 3458 txn/s, latency: 9056 ms, (p50: 9900 ms, p90: 12000 ms, p99: 12700 ms), latency samples: 138320
5. check swarm health
Compatibility test for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 1c769ce5c52f8c3a265b6cefdce9a4831192e924 passed
Test Ok
github-actions[bot] commented 2 weeks ago

:x: Forge suite realistic_env_max_load failure on 1c769ce5c52f8c3a265b6cefdce9a4831192e924

two traffics test: inner traffic : committed: 7416 txn/s, latency: 5266 ms, (p50: 4700 ms, p90: 6000 ms, p99: 17700 ms), latency samples: 3225960
two traffics test : committed: 100 txn/s, latency: 2221 ms, (p50: 1900 ms, p90: 2300 ms, p99: 10500 ms), latency samples: 1880
Latency breakdown for phase 0: ["QsBatchToPos: max: 0.216, avg: 0.204", "QsPosToProposal: max: 0.799, avg: 0.265", "ConsensusProposalToOrdered: max: 0.564, avg: 0.435", "ConsensusOrderedToCommit: max: 0.398, avg: 0.372", "ConsensusProposalToCommit: max: 0.962, avg: 0.807"]
Test Failed: check for success

Caused by:
    "QsPosToProposal" metric violated threshold of 0.6, max_breach_pct: 5, breach_pct: 7 

Stack backtrace:
   0: anyhow::error::<impl anyhow::Error>::msg
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/anyhow-1.0.79/src/error.rs:83:36
   1: aptos_forge::success_criteria::MetricsThreshold::ensure_metrics_threshold
             at ./testsuite/forge/src/success_criteria.rs:68:13
   2: aptos_forge::success_criteria::LatencyBreakdownThreshold::ensure_threshold
             at ./testsuite/forge/src/success_criteria.rs:132:13
   3: aptos_forge::success_criteria::SuccessCriteriaChecker::check_for_success::{{closure}}
             at ./testsuite/forge/src/success_criteria.rs:286:13
   4: tokio::runtime::park::CachedParkThread::block_on::{{closure}}
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/park.rs:282:63
   5: tokio::runtime::coop::with_budget
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/coop.rs:107:5
   6: tokio::runtime::coop::budget
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/coop.rs:73:5
   7: tokio::runtime::park::CachedParkThread::block_on
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/park.rs:282:31
   8: tokio::runtime::context::blocking::BlockingRegionGuard::block_on
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/context/blocking.rs:66:9
   9: tokio::runtime::scheduler::multi_thread::MultiThread::block_on::{{closure}}
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/scheduler/multi_thread/mod.rs:87:13
  10: tokio::runtime::context::runtime::enter_runtime
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/context/runtime.rs:65:16
  11: tokio::runtime::scheduler::multi_thread::MultiThread::block_on
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/scheduler/multi_thread/mod.rs:86:9
  12: tokio::runtime::runtime::Runtime::block_on
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/runtime.rs:350:50
  13: aptos_forge::interface::network::NetworkContext::check_for_success
             at ./testsuite/forge/src/interface/network.rs:71:9
  14: <dyn aptos_testcases::NetworkLoadTest as aptos_forge::interface::network::NetworkTest>::run
             at ./testsuite/testcases/src/lib.rs:229:13
  15: <aptos_testcases::CompositeNetworkTest as aptos_forge::interface::network::NetworkTest>::run
             at ./testsuite/testcases/src/lib.rs:499:9
  16: aptos_forge::runner::Forge<F>::run::{{closure}}
             at ./testsuite/forge/src/runner.rs:598:42
  17: aptos_forge::runner::run_test
             at ./testsuite/forge/src/runner.rs:666:11
  18: aptos_forge::runner::Forge<F>::run
             at ./testsuite/forge/src/runner.rs:598:30
  19: forge::run_forge
             at ./testsuite/forge-cli/src/main.rs:427:11
  20: forge::main
             at ./testsuite/forge-cli/src/main.rs:353:21
  21: core::ops::function::FnOnce::call_once
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/ops/function.rs:250:5
  22: std::sys_common::backtrace::__rust_begin_short_backtrace
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/sys_common/backtrace.rs:154:18
  23: std::rt::lang_start::{{closure}}
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:167:18
  24: core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/ops/function.rs:284:13
  25: std::panicking::try::do_call
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:552:40
  26: std::panicking::try
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:516:19
  27: std::panic::catch_unwind
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panic.rs:142:14
  28: std::rt::lang_start_internal::{{closure}}
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:148:48
  29: std::panicking::try::do_call
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:552:40
  30: std::panicking::try
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:516:19
  31: std::panic::catch_unwind
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panic.rs:142:14
  32: std::rt::lang_start_internal
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:148:20
  33: std::rt::lang_start
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:166:17
  34: __libc_start_main
  35: _start
Trailing Log Lines:
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:516:19
  31: std::panic::catch_unwind
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panic.rs:142:14
  32: std::rt::lang_start_internal
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:148:20
  33: std::rt::lang_start
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:166:17
  34: __libc_start_main
  35: _start

Swarm logs can be found here: See fgi output for more information.
{"level":"INFO","source":{"package":"aptos_forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:292"},"thread_name":"main","hostname":"forge-e2e-pr-13184-1714771915-1c769ce5c52f8c3a265b6cefdce9a4831","timestamp":"2024-05-03T21:45:43.233381Z","message":"Deleting namespace forge-e2e-pr-13184: Some(NamespaceStatus { conditions: None, phase: Some(\"Terminating\") })"}
{"level":"INFO","source":{"package":"aptos_forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:400"},"thread_name":"main","hostname":"forge-e2e-pr-13184-1714771915-1c769ce5c52f8c3a265b6cefdce9a4831","timestamp":"2024-05-03T21:45:43.233408Z","message":"aptos-node resources for Forge removed in namespace: forge-e2e-pr-13184"}
Failed to run tests:
Tests Failed

failures:
    CompositeNetworkTest

test result: FAILED. 0 passed; 1 failed; 0 filtered out

Error: Tests Failed

Stack backtrace:
   0: anyhow::error::<impl anyhow::Error>::msg
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/anyhow-1.0.79/src/error.rs:83:36
   1: aptos_forge::runner::Forge<F>::run
             at ./testsuite/forge/src/runner.rs:618:13
   2: forge::run_forge
             at ./testsuite/forge-cli/src/main.rs:427:11
   3: forge::main
             at ./testsuite/forge-cli/src/main.rs:353:21
   4: core::ops::function::FnOnce::call_once
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/ops/function.rs:250:5
   5: std::sys_common::backtrace::__rust_begin_short_backtrace
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/sys_common/backtrace.rs:154:18
   6: std::rt::lang_start::{{closure}}
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:167:18
   7: core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/ops/function.rs:284:13
   8: std::panicking::try::do_call
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:552:40
   9: std::panicking::try
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:516:19
  10: std::panic::catch_unwind
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panic.rs:142:14
  11: std::rt::lang_start_internal::{{closure}}
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:148:48
  12: std::panicking::try::do_call
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:552:40
  13: std::panicking::try
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:516:19
  14: std::panic::catch_unwind
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panic.rs:142:14
  15: std::rt::lang_start_internal
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:148:20
  16: std::rt::lang_start
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:166:17
  17: __libc_start_main
  18: _start
Debugging output:
NAME                                    READY   STATUS      RESTARTS   AGE
aptos-node-0-fullnode-eforge219-0       1/1     Running     0          12m
aptos-node-0-validator-0                1/1     Running     0          12m
aptos-node-1-fullnode-eforge219-0       1/1     Running     0          12m
aptos-node-1-validator-0                1/1     Running     0          12m
aptos-node-2-fullnode-eforge219-0       1/1     Running     0          12m
aptos-node-2-validator-0                1/1     Running     0          12m
aptos-node-3-fullnode-eforge219-0       1/1     Running     0          12m
aptos-node-3-validator-0                0/1     Init:0/1    0          104s
aptos-node-4-fullnode-eforge219-0       1/1     Running     0          12m
aptos-node-4-validator-0                1/1     Running     0          12m
aptos-node-5-validator-0                1/1     Running     0          12m
aptos-node-6-validator-0                1/1     Running     0          12m
genesis-aptos-genesis-eforge219-gkw2m   0/1     Completed   0          13m
github-actions[bot] commented 2 weeks ago

Forge is running suite realistic_env_max_load on 1c769ce5c52f8c3a265b6cefdce9a4831192e924

github-actions[bot] commented 2 weeks ago

Forge is running suite framework_upgrade on 01b24e7e3548382dd25440b39a0438a993387f12 ==> 1c769ce5c52f8c3a265b6cefdce9a4831192e924

github-actions[bot] commented 2 weeks ago

:white_check_mark: Forge suite realistic_env_max_load success on 1c769ce5c52f8c3a265b6cefdce9a4831192e924

two traffics test: inner traffic : committed: 7632 txn/s, latency: 5144 ms, (p50: 4800 ms, p90: 6300 ms, p99: 10700 ms), latency samples: 3289780
two traffics test : committed: 100 txn/s, latency: 2218 ms, (p50: 1900 ms, p90: 2400 ms, p99: 14000 ms), latency samples: 1700
Latency breakdown for phase 0: ["QsBatchToPos: max: 0.206, avg: 0.202", "QsPosToProposal: max: 0.398, avg: 0.246", "ConsensusProposalToOrdered: max: 0.486, avg: 0.444", "ConsensusOrderedToCommit: max: 0.375, avg: 0.357", "ConsensusProposalToCommit: max: 0.820, avg: 0.800"]
Max round gap was 1 [limit 4] at version 1635744. Max no progress secs was 4.150161 [limit 15] at version 1635744.
Test Ok
github-actions[bot] commented 2 weeks ago

:x: Forge suite framework_upgrade failure on 01b24e7e3548382dd25440b39a0438a993387f12 ==> 1c769ce5c52f8c3a265b6cefdce9a4831192e924

Compatibility test results for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 1c769ce5c52f8c3a265b6cefdce9a4831192e924 (PR)
Upgrade the nodes to version: 1c769ce5c52f8c3a265b6cefdce9a4831192e924
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1239 txn/s, submitted: 1243 txn/s, failed submission: 3 txn/s, expired: 3 txn/s, latency: 2404 ms, (p50: 2100 ms, p90: 3900 ms, p99: 5300 ms), latency samples: 111580
Test Failed: API error: Unknown error error sending request for url (http://aptos-node-3-validator.forge-framework-upgrade-pr-13184.svc:8080/v1/accounts/0000000000000000000000000000000000000000000000000000000000000001/resource/0x1::block::BlockResource): error trying to connect: dns error: failed to lookup address information: Name or service not known

Stack backtrace:
   0: anyhow::error::<impl core::convert::From<E> for anyhow::Error>::from
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/anyhow-1.0.79/src/error.rs:565:25
   1: <core::result::Result<T,F> as core::ops::try_trait::FromResidual<core::result::Result<core::convert::Infallible,E>>>::from_residual
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/result.rs:1963:27
   2: aptos_release_builder::validate::NetworkConfig::increase_lockup::{{closure}}
             at ./aptos-move/aptos-release-builder/src/validate.rs:383:9
   3: aptos_release_builder::validate::execute_release::{{closure}}
             at ./aptos-move/aptos-release-builder/src/validate.rs:404:38
   4: aptos_release_builder::validate::validate_config_and_generate_release::{{closure}}
             at ./aptos-move/aptos-release-builder/src/validate.rs:493:6
   5: aptos_release_builder::validate::validate_config::{{closure}}
             at ./aptos-move/aptos-release-builder/src/validate.rs:479:80
   6: tokio::runtime::park::CachedParkThread::block_on::{{closure}}
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/park.rs:282:63
   7: tokio::runtime::coop::with_budget
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/coop.rs:107:5
   8: tokio::runtime::coop::budget
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/coop.rs:73:5
   9: tokio::runtime::park::CachedParkThread::block_on
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/park.rs:282:31
  10: tokio::runtime::context::blocking::BlockingRegionGuard::block_on
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/context/blocking.rs:66:9
  11: tokio::runtime::scheduler::multi_thread::MultiThread::block_on::{{closure}}
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/scheduler/multi_thread/mod.rs:87:13
  12: tokio::runtime::context::runtime::enter_runtime
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/context/runtime.rs:65:16
  13: tokio::runtime::scheduler::multi_thread::MultiThread::block_on
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/scheduler/multi_thread/mod.rs:86:9
  14: tokio::runtime::runtime::Runtime::block_on
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/runtime.rs:350:50
  15: <aptos_testcases::framework_upgrade::FrameworkUpgrade as aptos_forge::interface::network::NetworkTest>::run
             at ./testsuite/testcases/src/framework_upgrade.rs:112:9
  16: aptos_forge::runner::Forge<F>::run::{{closure}}
             at ./testsuite/forge/src/runner.rs:598:42
  17: aptos_forge::runner::run_test
             at ./testsuite/forge/src/runner.rs:666:11
  18: aptos_forge::runner::Forge<F>::run
             at ./testsuite/forge/src/runner.rs:598:30
  19: forge::run_forge
             at ./testsuite/forge-cli/src/main.rs:427:11
  20: forge::main
             at ./testsuite/forge-cli/src/main.rs:353:21
  21: core::ops::function::FnOnce::call_once
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/ops/function.rs:250:5
  22: std::sys_common::backtrace::__rust_begin_short_backtrace
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/sys_common/backtrace.rs:154:18
  23: std::rt::lang_start::{{closure}}
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:167:18
  24: core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/ops/function.rs:284:13
  25: std::panicking::try::do_call
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:552:40
  26: std::panicking::try
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:516:19
  27: std::panic::catch_unwind
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panic.rs:142:14
  28: std::rt::lang_start_internal::{{closure}}
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:148:48
  29: std::panicking::try::do_call
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:552:40
  30: std::panicking::try
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:516:19
  31: std::panic::catch_unwind
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panic.rs:142:14
  32: std::rt::lang_start_internal
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:148:20
  33: std::rt::lang_start
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:166:17
  34: __libc_start_main
  35: _start
Trailing Log Lines:
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:516:19
  31: std::panic::catch_unwind
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panic.rs:142:14
  32: std::rt::lang_start_internal
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:148:20
  33: std::rt::lang_start
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:166:17
  34: __libc_start_main
  35: _start

Swarm logs can be found here: See fgi output for more information.
thread 'main' panicked at testsuite/forge/src/backend/k8s/swarm.rs:696:18:
called `Result::unwrap()` on an `Err` value: ApiError: namespaces "forge-framework-upgrade-pr-13184" not found: NotFound (ErrorResponse { status: "Failure", message: "namespaces \"forge-framework-upgrade-pr-13184\" not found", reason: "NotFound", code: 404 })

Caused by:
    namespaces "forge-framework-upgrade-pr-13184" not found: NotFound

Stack backtrace:
   0: anyhow::error::<impl core::convert::From<E> for anyhow::Error>::from
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/anyhow-1.0.79/src/error.rs:565:25
   1: <core::result::Result<T,F> as core::ops::try_trait::FromResidual<core::result::Result<core::convert::Infallible,E>>>::from_residual
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/result.rs:1963:27
   2: aptos_forge::backend::k8s::cluster_helper::delete_k8s_cluster::{{closure}}
             at ./testsuite/forge/src/backend/k8s/cluster_helper.rs:289:13
   3: aptos_forge::backend::k8s::cluster_helper::uninstall_testnet_resources::{{closure}}
             at ./testsuite/forge/src/backend/k8s/cluster_helper.rs:399:48
   4: tokio::runtime::park::CachedParkThread::block_on::{{closure}}
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/park.rs:282:63
   5: tokio::runtime::coop::with_budget
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/coop.rs:107:5
   6: tokio::runtime::coop::budget
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/coop.rs:73:5
   7: tokio::runtime::park::CachedParkThread::block_on
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/park.rs:282:31
   8: tokio::runtime::context::blocking::BlockingRegionGuard::block_on
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/context/blocking.rs:66:9
   9: tokio::runtime::scheduler::multi_thread::MultiThread::block_on::{{closure}}
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/scheduler/multi_thread/mod.rs:87:13
  10: tokio::runtime::context::runtime::enter_runtime
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/context/runtime.rs:65:16
  11: tokio::runtime::scheduler::multi_thread::MultiThread::block_on
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/scheduler/multi_thread/mod.rs:86:9
  12: tokio::runtime::runtime::Runtime::block_on
             at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/runtime.rs:350:50
  13: <aptos_forge::backend::k8s::swarm::K8sSwarm as core::ops::drop::Drop>::drop
             at ./testsuite/forge/src/backend/k8s/swarm.rs:694:13
  14: core::ptr::drop_in_place<aptos_forge::backend::k8s::swarm::K8sSwarm>
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/ptr/mod.rs:498:1
  15: core::ptr::drop_in_place<alloc::boxed::Box<dyn aptos_forge::interface::swarm::Swarm>>
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/ptr/mod.rs:498:1
  16: aptos_forge::runner::Forge<F>::run
             at ./testsuite/forge/src/runner.rs:611:9
  17: forge::run_forge
             at ./testsuite/forge-cli/src/main.rs:427:11
  18: forge::main
             at ./testsuite/forge-cli/src/main.rs:353:21
  19: core::ops::function::FnOnce::call_once
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/ops/function.rs:250:5
  20: std::sys_common::backtrace::__rust_begin_short_backtrace
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/sys_common/backtrace.rs:154:18
  21: std::rt::lang_start::{{closure}}
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:167:18
  22: core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/ops/function.rs:284:13
  23: std::panicking::try::do_call
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:552:40
  24: std::panicking::try
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:516:19
  25: std::panic::catch_unwind
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panic.rs:142:14
  26: std::rt::lang_start_internal::{{closure}}
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:148:48
  27: std::panicking::try::do_call
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:552:40
  28: std::panicking::try
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:516:19
  29: std::panic::catch_unwind
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panic.rs:142:14
  30: std::rt::lang_start_internal
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:148:20
  31: std::rt::lang_start
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:166:17
  32: __libc_start_main
  33: _start
stack backtrace:
   0: rust_begin_unwind
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:645:5
   1: core::panicking::panic_fmt
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/panicking.rs:72:14
   2: core::result::unwrap_failed
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/result.rs:1653:5
   3: core::result::Result<T,E>::unwrap
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/result.rs:1077:23
   4: <aptos_forge::backend::k8s::swarm::K8sSwarm as core::ops::drop::Drop>::drop
             at ./testsuite/forge/src/backend/k8s/swarm.rs:694:13
   5: core::ptr::drop_in_place<aptos_forge::backend::k8s::swarm::K8sSwarm>
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/ptr/mod.rs:498:1
   6: core::ptr::drop_in_place<alloc::boxed::Box<dyn aptos_forge::interface::swarm::Swarm>>
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/ptr/mod.rs:498:1
   7: aptos_forge::runner::Forge<F>::run
             at ./testsuite/forge/src/runner.rs:611:9
   8: forge::run_forge
             at ./testsuite/forge-cli/src/main.rs:427:11
   9: forge::main
             at ./testsuite/forge-cli/src/main.rs:353:21
  10: core::ops::function::FnOnce::call_once
             at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/ops/function.rs:250:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
Debugging output:
github-actions[bot] commented 2 weeks ago

Forge is running suite framework_upgrade on 01b24e7e3548382dd25440b39a0438a993387f12 ==> 68841680eca1473fa5604b460d395d05274c783c

github-actions[bot] commented 2 weeks ago

Forge is running suite compat on 01b24e7e3548382dd25440b39a0438a993387f12 ==> 68841680eca1473fa5604b460d395d05274c783c

github-actions[bot] commented 2 weeks ago

Forge is running suite realistic_env_max_load on 68841680eca1473fa5604b460d395d05274c783c

github-actions[bot] commented 2 weeks ago

:white_check_mark: Forge suite compat success on 01b24e7e3548382dd25440b39a0438a993387f12 ==> 68841680eca1473fa5604b460d395d05274c783c

Compatibility test results for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 68841680eca1473fa5604b460d395d05274c783c (PR)
1. Check liveness of validators at old version: 01b24e7e3548382dd25440b39a0438a993387f12
compatibility::simple-validator-upgrade::liveness-check : committed: 6402 txn/s, latency: 5002 ms, (p50: 4800 ms, p90: 9000 ms, p99: 10200 ms), latency samples: 236900
2. Upgrading first Validator to new version: 68841680eca1473fa5604b460d395d05274c783c
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 1671 txn/s, latency: 16949 ms, (p50: 19300 ms, p90: 23400 ms, p99: 23800 ms), latency samples: 88580
3. Upgrading rest of first batch to new version: 68841680eca1473fa5604b460d395d05274c783c
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 1502 txn/s, latency: 17770 ms, (p50: 19200 ms, p90: 22900 ms, p99: 25500 ms), latency samples: 82640
4. upgrading second batch to new version: 68841680eca1473fa5604b460d395d05274c783c
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 3622 txn/s, latency: 8606 ms, (p50: 9900 ms, p90: 11900 ms, p99: 12300 ms), latency samples: 144900
5. check swarm health
Compatibility test for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 68841680eca1473fa5604b460d395d05274c783c passed
Test Ok
github-actions[bot] commented 2 weeks ago

:white_check_mark: Forge suite realistic_env_max_load success on 68841680eca1473fa5604b460d395d05274c783c

two traffics test: inner traffic : committed: 7702 txn/s, latency: 5098 ms, (p50: 5000 ms, p90: 6000 ms, p99: 10800 ms), latency samples: 3327280
two traffics test : committed: 100 txn/s, latency: 1977 ms, (p50: 1900 ms, p90: 2200 ms, p99: 7500 ms), latency samples: 1800
Latency breakdown for phase 0: ["QsBatchToPos: max: 0.203, avg: 0.201", "QsPosToProposal: max: 0.273, avg: 0.239", "ConsensusProposalToOrdered: max: 0.459, avg: 0.433", "ConsensusOrderedToCommit: max: 0.373, avg: 0.360", "ConsensusProposalToCommit: max: 0.807, avg: 0.793"]
Max round gap was 1 [limit 4] at version 1644690. Max no progress secs was 4.763345 [limit 15] at version 1644690.
Test Ok
github-actions[bot] commented 2 weeks ago

:white_check_mark: Forge suite framework_upgrade success on 01b24e7e3548382dd25440b39a0438a993387f12 ==> 68841680eca1473fa5604b460d395d05274c783c

Compatibility test results for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 68841680eca1473fa5604b460d395d05274c783c (PR)
Upgrade the nodes to version: 68841680eca1473fa5604b460d395d05274c783c
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1246 txn/s, submitted: 1248 txn/s, failed submission: 1 txn/s, expired: 1 txn/s, latency: 2405 ms, (p50: 2100 ms, p90: 3900 ms, p99: 6300 ms), latency samples: 113440
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1211 txn/s, submitted: 1214 txn/s, failed submission: 2 txn/s, expired: 2 txn/s, latency: 2478 ms, (p50: 2100 ms, p90: 4500 ms, p99: 6200 ms), latency samples: 109040
5. check swarm health
Compatibility test for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 68841680eca1473fa5604b460d395d05274c783c passed
Upgrade the remaining nodes to version: 68841680eca1473fa5604b460d395d05274c783c
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1160 txn/s, submitted: 1163 txn/s, failed submission: 2 txn/s, expired: 2 txn/s, latency: 2712 ms, (p50: 2400 ms, p90: 4500 ms, p99: 5700 ms), latency samples: 102160
Test Ok