Closed igor-aptos closed 2 weeks ago
🚨 3 jobs on the last run were significantly faster/slower than expected
Job | Duration | vs 7d avg | Delta |
---|---|---|---|
cli-e2e-tests / run-cli-tests | 9m | 7m | |
rust-targeted-unit-tests | 24m | 20m | |
windows-build | 31m | 39m |
compat
on 01b24e7e3548382dd25440b39a0438a993387f12
==> 6bd7ca1f5101211b4189eb10392715ac4bf88470
realistic_env_max_load
on 6bd7ca1f5101211b4189eb10392715ac4bf88470
framework_upgrade
on 01b24e7e3548382dd25440b39a0438a993387f12
==> 6bd7ca1f5101211b4189eb10392715ac4bf88470
All modified and coverable lines are covered by tests :white_check_mark:
Project coverage is 57.5%. Comparing base (
8c6963d
) to head (7bcbf54
).
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
compat
success on 01b24e7e3548382dd25440b39a0438a993387f12
==> 6bd7ca1f5101211b4189eb10392715ac4bf88470
Compatibility test results for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 6bd7ca1f5101211b4189eb10392715ac4bf88470 (PR)
1. Check liveness of validators at old version: 01b24e7e3548382dd25440b39a0438a993387f12
compatibility::simple-validator-upgrade::liveness-check : committed: 6523.801616657681 txn/s, latency: 5035.165442240572 ms, (p50: 4800 ms, p90: 8700 ms, p99: 10200 ms), latency samples: 234940
2. Upgrading first Validator to new version: 6bd7ca1f5101211b4189eb10392715ac4bf88470
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 1707.0878962429867 txn/s, latency: 17043.215553097347 ms, (p50: 19000 ms, p90: 22600 ms, p99: 22900 ms), latency samples: 90400
3. Upgrading rest of first batch to new version: 6bd7ca1f5101211b4189eb10392715ac4bf88470
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 1389.9912597565424 txn/s, latency: 20172.14916284094 ms, (p50: 22300 ms, p90: 24600 ms, p99: 26400 ms), latency samples: 74060
4. upgrading second batch to new version: 6bd7ca1f5101211b4189eb10392715ac4bf88470
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 3532.097734708499 txn/s, latency: 8884.596503109882 ms, (p50: 9600 ms, p90: 12700 ms, p99: 12900 ms), latency samples: 144700
5. check swarm health
Compatibility test for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 6bd7ca1f5101211b4189eb10392715ac4bf88470 passed
Test Ok
realistic_env_max_load
success on 6bd7ca1f5101211b4189eb10392715ac4bf88470
two traffics test: inner traffic : committed: 7781.1816250482425 txn/s, latency: 5041.269490683747 ms, (p50: 4800 ms, p90: 5900 ms, p99: 11700 ms), latency samples: 3362940
two traffics test : committed: 99.97048721194317 txn/s, latency: 1863.911797752809 ms, (p50: 1800 ms, p90: 2100 ms, p99: 3600 ms), latency samples: 1780
Latency breakdown for phase 0: ["QsBatchToPos: max: 0.203, avg: 0.201", "QsPosToProposal: max: 0.283, avg: 0.243", "ConsensusProposalToOrdered: max: 0.463, avg: 0.422", "ConsensusOrderedToCommit: max: 0.391, avg: 0.372", "ConsensusProposalToCommit: max: 0.817, avg: 0.795"]
Max round gap was 1 [limit 4] at version 1739482. Max no progress secs was 4.621513 [limit 15] at version 1739482.
Test Ok
framework_upgrade
success on 01b24e7e3548382dd25440b39a0438a993387f12
==> 6bd7ca1f5101211b4189eb10392715ac4bf88470
Compatibility test results for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 6bd7ca1f5101211b4189eb10392715ac4bf88470 (PR)
Upgrade the nodes to version: 6bd7ca1f5101211b4189eb10392715ac4bf88470
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1002.1025199021367 txn/s, submitted: 1004.5172247693707 txn/s, failed submission: 2.4147048672340645 txn/s, expired: 2.4147048672340645 txn/s, latency: 2941.418170865279 ms, (p50: 2200 ms, p90: 5500 ms, p99: 12000 ms), latency samples: 91300
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1068.7818903885739 txn/s, submitted: 1071.4132778905564 txn/s, failed submission: 2.6313875019825375 txn/s, expired: 2.6313875019825375 txn/s, latency: 2762.0120332375873 ms, (p50: 2400 ms, p90: 5000 ms, p99: 7500 ms), latency samples: 97480
5. check swarm health
Compatibility test for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 6bd7ca1f5101211b4189eb10392715ac4bf88470 passed
Upgrade the remaining nodes to version: 6bd7ca1f5101211b4189eb10392715ac4bf88470
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1301.5106448811648 txn/s, submitted: 1303.4235761982784 txn/s, failed submission: 1.9129313171135989 txn/s, expired: 1.9129313171135989 txn/s, latency: 2491.8814256843652 ms, (p50: 2000 ms, p90: 4300 ms, p99: 6600 ms), latency samples: 108860
Test Ok
compat
on 01b24e7e3548382dd25440b39a0438a993387f12
==> dcbe7abac010666fd13f85d23fe471bd604ecb5a
framework_upgrade
on 01b24e7e3548382dd25440b39a0438a993387f12
==> dcbe7abac010666fd13f85d23fe471bd604ecb5a
realistic_env_max_load
on dcbe7abac010666fd13f85d23fe471bd604ecb5a
compat
success on 01b24e7e3548382dd25440b39a0438a993387f12
==> dcbe7abac010666fd13f85d23fe471bd604ecb5a
Compatibility test results for 01b24e7e3548382dd25440b39a0438a993387f12 ==> dcbe7abac010666fd13f85d23fe471bd604ecb5a (PR)
1. Check liveness of validators at old version: 01b24e7e3548382dd25440b39a0438a993387f12
compatibility::simple-validator-upgrade::liveness-check : committed: 5453.57069932496 txn/s, latency: 6027.205207788129 ms, (p50: 5100 ms, p90: 9000 ms, p99: 13900 ms), latency samples: 191060
2. Upgrading first Validator to new version: dcbe7abac010666fd13f85d23fe471bd604ecb5a
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 1833.430973849232 txn/s, latency: 15574.100740095777 ms, (p50: 19300 ms, p90: 22200 ms, p99: 22600 ms), latency samples: 91880
3. Upgrading rest of first batch to new version: dcbe7abac010666fd13f85d23fe471bd604ecb5a
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 1615.5753706122327 txn/s, latency: 16878.982298578198 ms, (p50: 19500 ms, p90: 22100 ms, p99: 24400 ms), latency samples: 84400
4. upgrading second batch to new version: dcbe7abac010666fd13f85d23fe471bd604ecb5a
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 3625.8875405256367 txn/s, latency: 8600.140023457983 ms, (p50: 9900 ms, p90: 11800 ms, p99: 12300 ms), latency samples: 144940
5. check swarm health
Compatibility test for 01b24e7e3548382dd25440b39a0438a993387f12 ==> dcbe7abac010666fd13f85d23fe471bd604ecb5a passed
Test Ok
realistic_env_max_load
success on dcbe7abac010666fd13f85d23fe471bd604ecb5a
two traffics test: inner traffic : committed: 8236.588010744483 txn/s, latency: 4763.629876583507 ms, (p50: 4500 ms, p90: 5600 ms, p99: 10800 ms), latency samples: 3553820
two traffics test : committed: 99.89075199107367 txn/s, latency: 1918.965294117647 ms, (p50: 1800 ms, p90: 2000 ms, p99: 6300 ms), latency samples: 1700
Latency breakdown for phase 0: ["QsBatchToPos: max: 0.212, avg: 0.203", "QsPosToProposal: max: 0.242, avg: 0.225", "ConsensusProposalToOrdered: max: 0.438, avg: 0.391", "ConsensusOrderedToCommit: max: 0.383, avg: 0.369", "ConsensusProposalToCommit: max: 0.774, avg: 0.759"]
Max round gap was 1 [limit 4] at version 1690943. Max no progress secs was 4.898467 [limit 15] at version 1690943.
Test Ok
framework_upgrade
success on 01b24e7e3548382dd25440b39a0438a993387f12
==> dcbe7abac010666fd13f85d23fe471bd604ecb5a
Compatibility test results for 01b24e7e3548382dd25440b39a0438a993387f12 ==> dcbe7abac010666fd13f85d23fe471bd604ecb5a (PR)
Upgrade the nodes to version: dcbe7abac010666fd13f85d23fe471bd604ecb5a
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 986.654604959693 txn/s, submitted: 988.8403705887574 txn/s, failed submission: 2.1857656290644507 txn/s, expired: 2.1857656290644507 txn/s, latency: 2982.7842822330526 ms, (p50: 2400 ms, p90: 4800 ms, p99: 7200 ms), latency samples: 90280
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1100.247742765164 txn/s, submitted: 1102.2673694600217 txn/s, failed submission: 2.0196266948575317 txn/s, expired: 2.0196266948575317 txn/s, latency: 2903.0115745461962 ms, (p50: 2200 ms, p90: 5200 ms, p99: 9000 ms), latency samples: 98060
5. check swarm health
Compatibility test for 01b24e7e3548382dd25440b39a0438a993387f12 ==> dcbe7abac010666fd13f85d23fe471bd604ecb5a passed
Upgrade the remaining nodes to version: dcbe7abac010666fd13f85d23fe471bd604ecb5a
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1299.140941804143 txn/s, submitted: 1299.947027186855 txn/s, failed submission: 0.8060853827119812 txn/s, expired: 0.8060853827119812 txn/s, latency: 2825.1608376421923 ms, (p50: 2100 ms, p90: 5200 ms, p99: 7600 ms), latency samples: 96700
Test Ok
framework_upgrade
on 01b24e7e3548382dd25440b39a0438a993387f12
==> 53ec8c3561e0fb1efda847bc0936b4b5a66cb4e0
compat
on 01b24e7e3548382dd25440b39a0438a993387f12
==> 53ec8c3561e0fb1efda847bc0936b4b5a66cb4e0
realistic_env_max_load
on 53ec8c3561e0fb1efda847bc0936b4b5a66cb4e0
compat
success on 01b24e7e3548382dd25440b39a0438a993387f12
==> 53ec8c3561e0fb1efda847bc0936b4b5a66cb4e0
Compatibility test results for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 53ec8c3561e0fb1efda847bc0936b4b5a66cb4e0 (PR)
1. Check liveness of validators at old version: 01b24e7e3548382dd25440b39a0438a993387f12
compatibility::simple-validator-upgrade::liveness-check : committed: 6091.346138529413 txn/s, latency: 5208.940217676371 ms, (p50: 4800 ms, p90: 9600 ms, p99: 12900 ms), latency samples: 228780
2. Upgrading first Validator to new version: 53ec8c3561e0fb1efda847bc0936b4b5a66cb4e0
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 1694.8764114968835 txn/s, latency: 16046.736673258625 ms, (p50: 19200 ms, p90: 22600 ms, p99: 23800 ms), latency samples: 91020
3. Upgrading rest of first batch to new version: 53ec8c3561e0fb1efda847bc0936b4b5a66cb4e0
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 1654.6527012529157 txn/s, latency: 17273.002021791766 ms, (p50: 17600 ms, p90: 24100 ms, p99: 24400 ms), latency samples: 82600
4. upgrading second batch to new version: 53ec8c3561e0fb1efda847bc0936b4b5a66cb4e0
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 829.8101887145685 txn/s, latency: 38471.2294 ms, (p50: 39600 ms, p90: 42100 ms, p99: 42500 ms), latency samples: 40000
5. check swarm health
Compatibility test for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 53ec8c3561e0fb1efda847bc0936b4b5a66cb4e0 passed
Test Ok
realistic_env_max_load
success on 53ec8c3561e0fb1efda847bc0936b4b5a66cb4e0
two traffics test: inner traffic : committed: 8117.814039084987 txn/s, latency: 4832.3627390825395 ms, (p50: 4500 ms, p90: 5700 ms, p99: 13800 ms), latency samples: 3504020
two traffics test : committed: 99.93326062092846 txn/s, latency: 1815.7603260869564 ms, (p50: 1800 ms, p90: 2100 ms, p99: 2300 ms), latency samples: 1840
Latency breakdown for phase 0: ["QsBatchToPos: max: 0.211, avg: 0.202", "QsPosToProposal: max: 0.355, avg: 0.230", "ConsensusProposalToOrdered: max: 0.450, avg: 0.420", "ConsensusOrderedToCommit: max: 0.375, avg: 0.363", "ConsensusProposalToCommit: max: 0.808, avg: 0.783"]
Max round gap was 1 [limit 4] at version 1748763. Max no progress secs was 4.750388 [limit 15] at version 1748763.
Test Ok
realistic_env_max_load
on 7bcbf54df1a32d83fec7aaeb198e28bac9b94c37
compat
on 01b24e7e3548382dd25440b39a0438a993387f12
==> 7bcbf54df1a32d83fec7aaeb198e28bac9b94c37
framework_upgrade
on 01b24e7e3548382dd25440b39a0438a993387f12
==> 7bcbf54df1a32d83fec7aaeb198e28bac9b94c37
compat
success on 01b24e7e3548382dd25440b39a0438a993387f12
==> 7bcbf54df1a32d83fec7aaeb198e28bac9b94c37
Compatibility test results for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 7bcbf54df1a32d83fec7aaeb198e28bac9b94c37 (PR)
1. Check liveness of validators at old version: 01b24e7e3548382dd25440b39a0438a993387f12
compatibility::simple-validator-upgrade::liveness-check : committed: 4699.299832630329 txn/s, latency: 5599.393269589552 ms, (p50: 5000 ms, p90: 9500 ms, p99: 13100 ms), latency samples: 214400
2. Upgrading first Validator to new version: 7bcbf54df1a32d83fec7aaeb198e28bac9b94c37
compatibility::simple-validator-upgrade::single-validator-upgrade : committed: 1448.7478615784955 txn/s, latency: 18760.17094375338 ms, (p50: 21600 ms, p90: 28300 ms, p99: 29500 ms), latency samples: 73960
3. Upgrading rest of first batch to new version: 7bcbf54df1a32d83fec7aaeb198e28bac9b94c37
compatibility::simple-validator-upgrade::half-validator-upgrade : committed: 1631.397070951372 txn/s, latency: 16273.322786959474 ms, (p50: 19200 ms, p90: 22500 ms, p99: 23700 ms), latency samples: 88340
4. upgrading second batch to new version: 7bcbf54df1a32d83fec7aaeb198e28bac9b94c37
compatibility::simple-validator-upgrade::rest-validator-upgrade : committed: 3433.343534023123 txn/s, latency: 9134.96384997081 ms, (p50: 9900 ms, p90: 11700 ms, p99: 12200 ms), latency samples: 137040
5. check swarm health
Compatibility test for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 7bcbf54df1a32d83fec7aaeb198e28bac9b94c37 passed
Test Ok
realistic_env_max_load
success on 7bcbf54df1a32d83fec7aaeb198e28bac9b94c37
two traffics test: inner traffic : committed: 7714.352967387171 txn/s, latency: 5079.997429224895 ms, (p50: 5000 ms, p90: 5800 ms, p99: 10500 ms), latency samples: 3331680
two traffics test : committed: 99.92157870693346 txn/s, latency: 1927.3692307692309 ms, (p50: 1800 ms, p90: 2200 ms, p99: 6300 ms), latency samples: 1820
Latency breakdown for phase 0: ["QsBatchToPos: max: 0.203, avg: 0.202", "QsPosToProposal: max: 0.254, avg: 0.233", "ConsensusProposalToOrdered: max: 0.458, avg: 0.421", "ConsensusOrderedToCommit: max: 0.382, avg: 0.370", "ConsensusProposalToCommit: max: 0.807, avg: 0.791"]
Max round gap was 1 [limit 4] at version 1680174. Max no progress secs was 4.431851 [limit 15] at version 1680174.
Test Ok
framework_upgrade
failure on 01b24e7e3548382dd25440b39a0438a993387f12
==> 7bcbf54df1a32d83fec7aaeb198e28bac9b94c37
Compatibility test results for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 7bcbf54df1a32d83fec7aaeb198e28bac9b94c37 (PR)
Upgrade the nodes to version: 7bcbf54df1a32d83fec7aaeb198e28bac9b94c37
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1409.86972553618 txn/s, submitted: 1410.8895228060073 txn/s, failed submission: 1.019797269827255 txn/s, expired: 1.019797269827255 txn/s, latency: 2497.496518987342 ms, (p50: 1800 ms, p90: 4200 ms, p99: 9000 ms), latency samples: 110600
Test Failed: API error: Unknown error error sending request for url (http://aptos-node-3-validator.forge-framework-upgrade-pr-13296.svc:8080/v1/estimate_gas_price): error trying to connect: dns error: failed to lookup address information: Name or service not known
Stack backtrace:
0: anyhow::error::<impl core::convert::From<E> for anyhow::Error>::from
at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/anyhow-1.0.79/src/error.rs:565:25
1: <core::result::Result<T,F> as core::ops::try_trait::FromResidual<core::result::Result<core::convert::Infallible,E>>>::from_residual
at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/result.rs:1963:27
2: aptos_release_builder::validate::NetworkConfig::execute_proposal::{{closure}}
at ./aptos-move/aptos-release-builder/src/validate.rs:364:9
3: aptos_release_builder::validate::NetworkConfig::submit_and_execute_multi_step_proposal::{{closure}}
at ./aptos-move/aptos-release-builder/src/validate.rs:124:64
4: aptos_release_builder::validate::execute_release::{{closure}}
at ./aptos-move/aptos-release-builder/src/validate.rs:432:22
5: aptos_release_builder::validate::validate_config_and_generate_release::{{closure}}
at ./aptos-move/aptos-release-builder/src/validate.rs:493:6
6: aptos_release_builder::validate::validate_config::{{closure}}
at ./aptos-move/aptos-release-builder/src/validate.rs:479:80
7: tokio::runtime::park::CachedParkThread::block_on::{{closure}}
at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/park.rs:282:63
8: tokio::runtime::coop::with_budget
at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/coop.rs:107:5
9: tokio::runtime::coop::budget
at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/coop.rs:73:5
10: tokio::runtime::park::CachedParkThread::block_on
at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/park.rs:282:31
11: tokio::runtime::context::blocking::BlockingRegionGuard::block_on
at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/context/blocking.rs:66:9
12: tokio::runtime::scheduler::multi_thread::MultiThread::block_on::{{closure}}
at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/scheduler/multi_thread/mod.rs:87:13
13: tokio::runtime::context::runtime::enter_runtime
at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/context/runtime.rs:65:16
14: tokio::runtime::scheduler::multi_thread::MultiThread::block_on
at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/scheduler/multi_thread/mod.rs:86:9
15: tokio::runtime::runtime::Runtime::block_on
at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/runtime.rs:350:50
16: <aptos_testcases::framework_upgrade::FrameworkUpgrade as aptos_forge::interface::network::NetworkTest>::run
at ./testsuite/testcases/src/framework_upgrade.rs:112:9
17: aptos_forge::runner::Forge<F>::run::{{closure}}
at ./testsuite/forge/src/runner.rs:598:42
18: aptos_forge::runner::run_test
at ./testsuite/forge/src/runner.rs:666:11
19: aptos_forge::runner::Forge<F>::run
at ./testsuite/forge/src/runner.rs:598:30
20: forge::run_forge
at ./testsuite/forge-cli/src/main.rs:427:11
21: forge::main
at ./testsuite/forge-cli/src/main.rs:353:21
22: core::ops::function::FnOnce::call_once
at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/ops/function.rs:250:5
23: std::sys_common::backtrace::__rust_begin_short_backtrace
at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/sys_common/backtrace.rs:154:18
24: std::rt::lang_start::{{closure}}
at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:167:18
25: core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once
at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/ops/function.rs:284:13
26: std::panicking::try::do_call
at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:552:40
27: std::panicking::try
at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:516:19
28: std::panic::catch_unwind
at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panic.rs:142:14
29: std::rt::lang_start_internal::{{closure}}
at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:148:48
30: std::panicking::try::do_call
at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:552:40
31: std::panicking::try
at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:516:19
32: std::panic::catch_unwind
at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panic.rs:142:14
33: std::rt::lang_start_internal
at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:148:20
34: std::rt::lang_start
at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:166:17
35: __libc_start_main
36: _start
Trailing Log Lines:
at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:516:19
32: std::panic::catch_unwind
at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panic.rs:142:14
33: std::rt::lang_start_internal
at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:148:20
34: std::rt::lang_start
at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:166:17
35: __libc_start_main
36: _start
Swarm logs can be found here: See fgi output for more information.
thread 'main' panicked at testsuite/forge/src/backend/k8s/swarm.rs:696:18:
called `Result::unwrap()` on an `Err` value: ApiError: namespaces "forge-framework-upgrade-pr-13296" not found: NotFound (ErrorResponse { status: "Failure", message: "namespaces \"forge-framework-upgrade-pr-13296\" not found", reason: "NotFound", code: 404 })
Caused by:
namespaces "forge-framework-upgrade-pr-13296" not found: NotFound
Stack backtrace:
0: anyhow::error::<impl core::convert::From<E> for anyhow::Error>::from
at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/anyhow-1.0.79/src/error.rs:565:25
1: <core::result::Result<T,F> as core::ops::try_trait::FromResidual<core::result::Result<core::convert::Infallible,E>>>::from_residual
at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/result.rs:1963:27
2: aptos_forge::backend::k8s::cluster_helper::delete_k8s_cluster::{{closure}}
at ./testsuite/forge/src/backend/k8s/cluster_helper.rs:289:13
3: aptos_forge::backend::k8s::cluster_helper::uninstall_testnet_resources::{{closure}}
at ./testsuite/forge/src/backend/k8s/cluster_helper.rs:399:48
4: tokio::runtime::park::CachedParkThread::block_on::{{closure}}
at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/park.rs:282:63
5: tokio::runtime::coop::with_budget
at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/coop.rs:107:5
6: tokio::runtime::coop::budget
at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/coop.rs:73:5
7: tokio::runtime::park::CachedParkThread::block_on
at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/park.rs:282:31
8: tokio::runtime::context::blocking::BlockingRegionGuard::block_on
at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/context/blocking.rs:66:9
9: tokio::runtime::scheduler::multi_thread::MultiThread::block_on::{{closure}}
at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/scheduler/multi_thread/mod.rs:87:13
10: tokio::runtime::context::runtime::enter_runtime
at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/context/runtime.rs:65:16
11: tokio::runtime::scheduler::multi_thread::MultiThread::block_on
at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/scheduler/multi_thread/mod.rs:86:9
12: tokio::runtime::runtime::Runtime::block_on
at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.1/src/runtime/runtime.rs:350:50
13: <aptos_forge::backend::k8s::swarm::K8sSwarm as core::ops::drop::Drop>::drop
at ./testsuite/forge/src/backend/k8s/swarm.rs:694:13
14: core::ptr::drop_in_place<aptos_forge::backend::k8s::swarm::K8sSwarm>
at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/ptr/mod.rs:498:1
15: core::ptr::drop_in_place<alloc::boxed::Box<dyn aptos_forge::interface::swarm::Swarm>>
at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/ptr/mod.rs:498:1
16: aptos_forge::runner::Forge<F>::run
at ./testsuite/forge/src/runner.rs:611:9
17: forge::run_forge
at ./testsuite/forge-cli/src/main.rs:427:11
18: forge::main
at ./testsuite/forge-cli/src/main.rs:353:21
19: core::ops::function::FnOnce::call_once
at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/ops/function.rs:250:5
20: std::sys_common::backtrace::__rust_begin_short_backtrace
at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/sys_common/backtrace.rs:154:18
21: std::rt::lang_start::{{closure}}
at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:167:18
22: core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once
at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/ops/function.rs:284:13
23: std::panicking::try::do_call
at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:552:40
24: std::panicking::try
at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:516:19
25: std::panic::catch_unwind
at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panic.rs:142:14
26: std::rt::lang_start_internal::{{closure}}
at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:148:48
27: std::panicking::try::do_call
at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:552:40
28: std::panicking::try
at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:516:19
29: std::panic::catch_unwind
at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panic.rs:142:14
30: std::rt::lang_start_internal
at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:148:20
31: std::rt::lang_start
at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/rt.rs:166:17
32: __libc_start_main
33: _start
stack backtrace:
0: rust_begin_unwind
at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:645:5
1: core::panicking::panic_fmt
at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/panicking.rs:72:14
2: core::result::unwrap_failed
at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/result.rs:1653:5
3: core::result::Result<T,E>::unwrap
at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/result.rs:1077:23
4: <aptos_forge::backend::k8s::swarm::K8sSwarm as core::ops::drop::Drop>::drop
at ./testsuite/forge/src/backend/k8s/swarm.rs:694:13
5: core::ptr::drop_in_place<aptos_forge::backend::k8s::swarm::K8sSwarm>
at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/ptr/mod.rs:498:1
6: core::ptr::drop_in_place<alloc::boxed::Box<dyn aptos_forge::interface::swarm::Swarm>>
at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/ptr/mod.rs:498:1
7: aptos_forge::runner::Forge<F>::run
at ./testsuite/forge/src/runner.rs:611:9
8: forge::run_forge
at ./testsuite/forge-cli/src/main.rs:427:11
9: forge::main
at ./testsuite/forge-cli/src/main.rs:353:21
10: core::ops::function::FnOnce::call_once
at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/ops/function.rs:250:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
Debugging output:
framework_upgrade
on 01b24e7e3548382dd25440b39a0438a993387f12
==> 7bcbf54df1a32d83fec7aaeb198e28bac9b94c37
framework_upgrade
success on 01b24e7e3548382dd25440b39a0438a993387f12
==> 7bcbf54df1a32d83fec7aaeb198e28bac9b94c37
Compatibility test results for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 7bcbf54df1a32d83fec7aaeb198e28bac9b94c37 (PR)
Upgrade the nodes to version: 7bcbf54df1a32d83fec7aaeb198e28bac9b94c37
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1232.9239435630768 txn/s, submitted: 1237.012047140329 txn/s, failed submission: 4.088103577252254 txn/s, expired: 4.088103577252254 txn/s, latency: 2610.6480592939342 ms, (p50: 2100 ms, p90: 4500 ms, p99: 8700 ms), latency samples: 102540
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1115.8592642874905 txn/s, submitted: 1117.7048036950364 txn/s, failed submission: 1.845539407545984 txn/s, expired: 1.845539407545984 txn/s, latency: 2823.669319826339 ms, (p50: 2100 ms, p90: 4800 ms, p99: 8100 ms), latency samples: 96740
5. check swarm health
Compatibility test for 01b24e7e3548382dd25440b39a0438a993387f12 ==> 7bcbf54df1a32d83fec7aaeb198e28bac9b94c37 passed
Upgrade the remaining nodes to version: 7bcbf54df1a32d83fec7aaeb198e28bac9b94c37
framework_upgrade::framework-upgrade::full-framework-upgrade : committed: 1094.7008770519503 txn/s, submitted: 1096.0954004622074 txn/s, failed submission: 1.3945234102572615 txn/s, expired: 1.3945234102572615 txn/s, latency: 2859.565212314225 ms, (p50: 2700 ms, p90: 4500 ms, p99: 6300 ms), latency samples: 94200
Test Ok
Description
devnet should have it enabled by default.
extracting from not-yet-reviewed https://github.com/aptos-labs/aptos-core/pull/13194
Type of Change
Which Components or Systems Does This Change Impact?
How Has This Been Tested?
Key Areas to Review
Checklist