search

aptos-labs / aptos-core

Aptos is a layer 1 blockchain built to support the widespread use of blockchain through better technology and user experience.
https://aptosfoundation.org
Other
5.86k stars 3.54k forks source link

[framework] Create release config for 1.4 #7696

Closed runtian-zhou closed 1 year ago

runtian-zhou commented 1 year ago

Description

Updated the yaml file to track release for 1.4. A few modifications were made in the PR:

  1. The validate-proposal command no longer mint to the validator. Tests now need to manually mint to validators before executing the proposals. This allows the tooling to better mimic what the release process will look like in real life.
  2. Added the full 1.4 release configuration. Tentatively we will have five different goverance proposals to vote. We are still waiting on finalizing the fifth one to enable transaction reshuffling cc @zekun000
  3. Change the release tooling to be able to handle multiple proposals in one release conifg.
  4. Add the ability to include a customize script for the release builder

Test Plan

Framework upgrade test

github-actions[bot] commented 1 year ago

Forge is running suite framework_upgrade on aptos-node-v1.3.0_3fc3d42b6cfe27460004f9a0326451bcda840a60 ==> 784933dae2522e45a67bfc4e0d0ab5d6c687a2dc

github-actions[bot] commented 1 year ago

Forge is running suite compat on testnet_2d8b1b57553d869190f61df1aaf7f31a8fc19a7b ==> 784933dae2522e45a67bfc4e0d0ab5d6c687a2dc

github-actions[bot] commented 1 year ago

Forge is running suite land_blocking on 784933dae2522e45a67bfc4e0d0ab5d6c687a2dc

github-actions[bot] commented 1 year ago

:white_check_mark: Forge suite framework_upgrade success on aptos-node-v1.3.0_3fc3d42b6cfe27460004f9a0326451bcda840a60 ==> 784933dae2522e45a67bfc4e0d0ab5d6c687a2dc

Compatibility test results for aptos-node-v1.3.0_3fc3d42b6cfe27460004f9a0326451bcda840a60 ==> 784933dae2522e45a67bfc4e0d0ab5d6c687a2dc (PR)
Upgrade the nodes to version: 784933dae2522e45a67bfc4e0d0ab5d6c687a2dc
framework_upgrade::framework-upgrade::full-framework-upgrade : 6578 TPS, 5864 ms latency, 9500 ms p99 latency,no expired txns
5. check swarm health
Compatibility test for aptos-node-v1.3.0_3fc3d42b6cfe27460004f9a0326451bcda840a60 ==> 784933dae2522e45a67bfc4e0d0ab5d6c687a2dc passed
Test Ok
github-actions[bot] commented 1 year ago

:white_check_mark: Forge suite compat success on testnet_2d8b1b57553d869190f61df1aaf7f31a8fc19a7b ==> 784933dae2522e45a67bfc4e0d0ab5d6c687a2dc

Compatibility test results for testnet_2d8b1b57553d869190f61df1aaf7f31a8fc19a7b ==> 784933dae2522e45a67bfc4e0d0ab5d6c687a2dc (PR)
1. Check liveness of validators at old version: testnet_2d8b1b57553d869190f61df1aaf7f31a8fc19a7b
compatibility::simple-validator-upgrade::liveness-check : 8099 TPS, 4717 ms latency, 6800 ms p99 latency,no expired txns
2. Upgrading first Validator to new version: 784933dae2522e45a67bfc4e0d0ab5d6c687a2dc
compatibility::simple-validator-upgrade::single-validator-upgrade : 4404 TPS, 9114 ms latency, 11800 ms p99 latency,no expired txns
3. Upgrading rest of first batch to new version: 784933dae2522e45a67bfc4e0d0ab5d6c687a2dc
compatibility::simple-validator-upgrade::half-validator-upgrade : 4804 TPS, 8022 ms latency, 10800 ms p99 latency,no expired txns
4. upgrading second batch to new version: 784933dae2522e45a67bfc4e0d0ab5d6c687a2dc
compatibility::simple-validator-upgrade::rest-validator-upgrade : 6722 TPS, 5689 ms latency, 11400 ms p99 latency,no expired txns
5. check swarm health
Compatibility test for testnet_2d8b1b57553d869190f61df1aaf7f31a8fc19a7b ==> 784933dae2522e45a67bfc4e0d0ab5d6c687a2dc passed
Test Ok
github-actions[bot] commented 1 year ago

:white_check_mark: Forge suite land_blocking success on 784933dae2522e45a67bfc4e0d0ab5d6c687a2dc

performance benchmark with full nodes : 5683 TPS, 6981 ms latency, 9900 ms p99 latency,(!) expired 100 out of 2426800 txns
Test Ok
movekevin commented 1 year ago

is our plan moving forward to start checking this into the code? why not keep release.yaml?

The problem with this current approach is we need to remember to move new release steps into a new file (e.g. v1.5) after a branch cut. The release tooling generates all steps from the yaml file regardless of if those steps have already been executed. @runtian-zhou and @perryjrandall are still thinking through how to make this the least manual/error prone possible so there might be more changes coming later for 1.5.

github-actions[bot] commented 1 year ago

Forge is running suite land_blocking on 1fb687d4a5b20915752f5e5a1836789532896dac

github-actions[bot] commented 1 year ago

Forge is running suite compat on testnet_2d8b1b57553d869190f61df1aaf7f31a8fc19a7b ==> 1fb687d4a5b20915752f5e5a1836789532896dac

github-actions[bot] commented 1 year ago

Forge is running suite framework_upgrade on aptos-node-v1.3.0_3fc3d42b6cfe27460004f9a0326451bcda840a60 ==> 1fb687d4a5b20915752f5e5a1836789532896dac

github-actions[bot] commented 1 year ago

:white_check_mark: Forge suite compat success on testnet_2d8b1b57553d869190f61df1aaf7f31a8fc19a7b ==> 1fb687d4a5b20915752f5e5a1836789532896dac

Compatibility test results for testnet_2d8b1b57553d869190f61df1aaf7f31a8fc19a7b ==> 1fb687d4a5b20915752f5e5a1836789532896dac (PR)
1. Check liveness of validators at old version: testnet_2d8b1b57553d869190f61df1aaf7f31a8fc19a7b
compatibility::simple-validator-upgrade::liveness-check : 7800 TPS, 4892 ms latency, 7000 ms p99 latency,no expired txns
2. Upgrading first Validator to new version: 1fb687d4a5b20915752f5e5a1836789532896dac
compatibility::simple-validator-upgrade::single-validator-upgrade : 4998 TPS, 8142 ms latency, 11100 ms p99 latency,no expired txns
3. Upgrading rest of first batch to new version: 1fb687d4a5b20915752f5e5a1836789532896dac
compatibility::simple-validator-upgrade::half-validator-upgrade : 4401 TPS, 8986 ms latency, 12500 ms p99 latency,no expired txns
4. upgrading second batch to new version: 1fb687d4a5b20915752f5e5a1836789532896dac
compatibility::simple-validator-upgrade::rest-validator-upgrade : 6468 TPS, 5853 ms latency, 10200 ms p99 latency,no expired txns
5. check swarm health
Compatibility test for testnet_2d8b1b57553d869190f61df1aaf7f31a8fc19a7b ==> 1fb687d4a5b20915752f5e5a1836789532896dac passed
Test Ok
github-actions[bot] commented 1 year ago

:white_check_mark: Forge suite land_blocking success on 1fb687d4a5b20915752f5e5a1836789532896dac

performance benchmark with full nodes : 5867 TPS, 6751 ms latency, 9000 ms p99 latency,(!) expired 160 out of 2505680 txns
Test Ok
github-actions[bot] commented 1 year ago

:white_check_mark: Forge suite framework_upgrade success on aptos-node-v1.3.0_3fc3d42b6cfe27460004f9a0326451bcda840a60 ==> 1fb687d4a5b20915752f5e5a1836789532896dac

Compatibility test results for aptos-node-v1.3.0_3fc3d42b6cfe27460004f9a0326451bcda840a60 ==> 1fb687d4a5b20915752f5e5a1836789532896dac (PR)
Upgrade the nodes to version: 1fb687d4a5b20915752f5e5a1836789532896dac
framework_upgrade::framework-upgrade::full-framework-upgrade : 6610 TPS, 5847 ms latency, 8700 ms p99 latency,no expired txns
5. check swarm health
Compatibility test for aptos-node-v1.3.0_3fc3d42b6cfe27460004f9a0326451bcda840a60 ==> 1fb687d4a5b20915752f5e5a1836789532896dac passed
Test Ok
github-actions[bot] commented 1 year ago

Forge is running suite land_blocking on 810d812e62a2e98190d08081352f0ac2ea4175ac

github-actions[bot] commented 1 year ago

Forge is running suite framework_upgrade on aptos-node-v1.3.0_3fc3d42b6cfe27460004f9a0326451bcda840a60 ==> 810d812e62a2e98190d08081352f0ac2ea4175ac

github-actions[bot] commented 1 year ago

Forge is running suite compat on testnet_2d8b1b57553d869190f61df1aaf7f31a8fc19a7b ==> 810d812e62a2e98190d08081352f0ac2ea4175ac

github-actions[bot] commented 1 year ago

:x: Forge suite framework_upgrade failure on aptos-node-v1.3.0_3fc3d42b6cfe27460004f9a0326451bcda840a60 ==> 810d812e62a2e98190d08081352f0ac2ea4175ac

Compatibility test results for aptos-node-v1.3.0_3fc3d42b6cfe27460004f9a0326451bcda840a60 ==> 810d812e62a2e98190d08081352f0ac2ea4175ac (PR)
Upgrade the nodes to version: 810d812e62a2e98190d08081352f0ac2ea4175ac
Test Failed: Simulation failed with status: Transaction Executed and Committed with Error MAX_GAS_UNITS_BELOW_MIN_TRANSACTION_GAS_UNITS

Stack backtrace:
   0: <core::result::Result<T,F> as core::ops::try_trait::FromResidual<core::result::Result<core::convert::Infallible,E>>>::from_residual
             at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/core/src/result.rs:2108:27
      aptos_release_builder::validate::NetworkConfig::create_governance_proposal::{{closure}}
             at ./aptos-move/aptos-release-builder/src/validate.rs:206:13
      <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
             at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/core/src/future/mod.rs:91:19
   1: aptos_release_builder::validate::NetworkConfig::submit_and_execute_multi_step_proposal::{{closure}}
             at ./aptos-move/aptos-release-builder/src/validate.rs:106:13
      <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
             at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/core/src/future/mod.rs:91:19
      aptos_release_builder::validate::execute_release::{{closure}}
             at ./aptos-move/aptos-release-builder/src/validate.rs:374:13
      <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
             at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/core/src/future/mod.rs:91:19
      aptos_release_builder::validate::validate_config::{{closure}}
             at ./aptos-move/aptos-release-builder/src/validate.rs:422:68
      <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
             at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/core/src/future/mod.rs:91:19
   2: tokio::park::thread::CachedParkThread::block_on::{{closure}}
             at /usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.21.2/src/park/thread.rs:267:54
      tokio::coop::with_budget::{{closure}}
             at /usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.21.2/src/coop.rs:102:9
      std::thread::local::LocalKey<T>::try_with
             at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/std/src/thread/local.rs:446:16
      std::thread::local::LocalKey<T>::with
             at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/std/src/thread/local.rs:422:9
   3: tokio::coop::with_budget
             at /usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.21.2/src/coop.rs:95:5
      tokio::coop::budget
             at /usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.21.2/src/coop.rs:72:5
      tokio::park::thread::CachedParkThread::block_on
             at /usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.21.2/src/park/thread.rs:267:31
   4: tokio::runtime::enter::Enter::block_on
             at /usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.21.2/src/runtime/enter.rs:152:13
      tokio::runtime::scheduler::multi_thread::MultiThread::block_on
             at /usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.21.2/src/runtime/scheduler/multi_thread/mod.rs:79:9
   5: tokio::runtime::Runtime::block_on
             at /usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.21.2/src/runtime/mod.rs:492:44
   6: <aptos_testcases::framework_upgrade::FrameworkUpgrade as aptos_forge::interface::network::NetworkTest>::run
             at ./testsuite/testcases/src/framework_upgrade.rs:95:9
   7: aptos_forge::runner::Forge<F>::run::{{closure}}
             at ./testsuite/forge/src/runner.rs:397:42
      aptos_forge::runner::run_test
             at ./testsuite/forge/src/runner.rs:464:11
      aptos_forge::runner::Forge<F>::run
             at ./testsuite/forge/src/runner.rs:397:30
   8: forge::run_forge
             at ./testsuite/forge-cli/src/main.rs:334:11
      forge::main
             at ./testsuite/forge-cli/src/main.rs:261:21
   9: core::ops::function::FnOnce::call_once
             at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/core/src/ops/function.rs:251:5
      std::sys_common::backtrace::__rust_begin_short_backtrace
             at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/std/src/sys_common/backtrace.rs:121:18
  10: std::rt::lang_start::{{closure}}
             at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/std/src/rt.rs:166:18
  11: core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once
             at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/core/src/ops/function.rs:286:13
      std::panicking::try::do_call
             at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/std/src/panicking.rs:483:40
      std::panicking::try
             at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/std/src/panicking.rs:447:19
      std::panic::catch_unwind
             at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/std/src/panic.rs:137:14
      std::rt::lang_start_internal::{{closure}}
             at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/std/src/rt.rs:148:48
      std::panicking::try::do_call
             at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/std/src/panicking.rs:483:40
      std::panicking::try
             at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/std/src/panicking.rs:447:19
      std::panic::catch_unwind
             at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/std/src/panic.rs:137:14
      std::rt::lang_start_internal
             at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/std/src/rt.rs:148:20
  12: main
  13: __libc_start_main
  14: _start
Trailing Log Lines:
      std::panicking::try
             at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/std/src/panicking.rs:447:19
      std::panic::catch_unwind
             at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/std/src/panic.rs:137:14
      std::rt::lang_start_internal
             at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/std/src/rt.rs:148:20
  12: main
  13: __libc_start_main
  14: _start

Swarm logs can be found here: See fgi output for more information.
{"level":"INFO","source":{"package":"aptos_forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:282"},"thread_name":"main","hostname":"forge-framework-upgrade-pr-7696-1681705700-aptos-node-v1-3-0-3f","timestamp":"2023-04-17T04:45:13.152326Z","message":"Deleting namespace forge-framework-upgrade-pr-7696: Some(NamespaceStatus { phase: Some(\"Terminating\") })"}
{"level":"INFO","source":{"package":"aptos_forge","file":"testsuite/forge/src/backend/k8s/cluster_helper.rs:390"},"thread_name":"main","hostname":"forge-framework-upgrade-pr-7696-1681705700-aptos-node-v1-3-0-3f","timestamp":"2023-04-17T04:45:13.152349Z","message":"aptos-node resources for Forge removed in namespace: forge-framework-upgrade-pr-7696"}

failures:
    framework_upgrade::framework-upgrade

test result: FAILED. 0 passed; 1 failed; 0 filtered out

Failed to run tests:
Tests Failed
Error: Tests Failed

Stack backtrace:
   0: aptos_forge::runner::Forge<F>::run
             at ./testsuite/forge/src/runner.rs:417:17
   1: forge::run_forge
             at ./testsuite/forge-cli/src/main.rs:334:11
      forge::main
             at ./testsuite/forge-cli/src/main.rs:261:21
   2: core::ops::function::FnOnce::call_once
             at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/core/src/ops/function.rs:251:5
      std::sys_common::backtrace::__rust_begin_short_backtrace
             at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/std/src/sys_common/backtrace.rs:121:18
   3: std::rt::lang_start::{{closure}}
             at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/std/src/rt.rs:166:18
   4: core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once
             at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/core/src/ops/function.rs:286:13
      std::panicking::try::do_call
             at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/std/src/panicking.rs:483:40
      std::panicking::try
             at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/std/src/panicking.rs:447:19
      std::panic::catch_unwind
             at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/std/src/panic.rs:137:14
      std::rt::lang_start_internal::{{closure}}
             at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/std/src/rt.rs:148:48
      std::panicking::try::do_call
             at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/std/src/panicking.rs:483:40
      std::panicking::try
             at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/std/src/panicking.rs:447:19
      std::panic::catch_unwind
             at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/std/src/panic.rs:137:14
      std::rt::lang_start_internal
             at /rustc/90743e7298aca107ddaa0c202a4d3604e29bfeb6/library/std/src/rt.rs:148:20
   5: main
   6: __libc_start_main
   7: _start
Debugging output:
NAME                                    READY   STATUS      RESTARTS   AGE
genesis-aptos-genesis-eforge209-qssjf   0/1     Completed   0          14m
github-actions[bot] commented 1 year ago

:white_check_mark: Forge suite land_blocking success on 810d812e62a2e98190d08081352f0ac2ea4175ac

performance benchmark with full nodes : 5874 TPS, 6742 ms latency, 9600 ms p99 latency,(!) expired 320 out of 2508760 txns
Test Ok
github-actions[bot] commented 1 year ago

:white_check_mark: Forge suite compat success on testnet_2d8b1b57553d869190f61df1aaf7f31a8fc19a7b ==> 810d812e62a2e98190d08081352f0ac2ea4175ac

Compatibility test results for testnet_2d8b1b57553d869190f61df1aaf7f31a8fc19a7b ==> 810d812e62a2e98190d08081352f0ac2ea4175ac (PR)
1. Check liveness of validators at old version: testnet_2d8b1b57553d869190f61df1aaf7f31a8fc19a7b
compatibility::simple-validator-upgrade::liveness-check : 8126 TPS, 4694 ms latency, 6800 ms p99 latency,no expired txns
2. Upgrading first Validator to new version: 810d812e62a2e98190d08081352f0ac2ea4175ac
compatibility::simple-validator-upgrade::single-validator-upgrade : 4824 TPS, 8394 ms latency, 12100 ms p99 latency,no expired txns
3. Upgrading rest of first batch to new version: 810d812e62a2e98190d08081352f0ac2ea4175ac
compatibility::simple-validator-upgrade::half-validator-upgrade : 4405 TPS, 9011 ms latency, 11300 ms p99 latency,no expired txns
4. upgrading second batch to new version: 810d812e62a2e98190d08081352f0ac2ea4175ac
compatibility::simple-validator-upgrade::rest-validator-upgrade : 6708 TPS, 5641 ms latency, 9600 ms p99 latency,no expired txns
5. check swarm health
Compatibility test for testnet_2d8b1b57553d869190f61df1aaf7f31a8fc19a7b ==> 810d812e62a2e98190d08081352f0ac2ea4175ac passed
Test Ok
github-actions[bot] commented 1 year ago

Forge is running suite land_blocking on 112ccbfbcff733cdb16beab9e713a6623f6846e5

github-actions[bot] commented 1 year ago

Forge is running suite framework_upgrade on aptos-node-v1.3.0_3fc3d42b6cfe27460004f9a0326451bcda840a60 ==> 112ccbfbcff733cdb16beab9e713a6623f6846e5

github-actions[bot] commented 1 year ago

Forge is running suite compat on testnet_2d8b1b57553d869190f61df1aaf7f31a8fc19a7b ==> 112ccbfbcff733cdb16beab9e713a6623f6846e5

github-actions[bot] commented 1 year ago

:white_check_mark: Forge suite compat success on testnet_2d8b1b57553d869190f61df1aaf7f31a8fc19a7b ==> 112ccbfbcff733cdb16beab9e713a6623f6846e5

Compatibility test results for testnet_2d8b1b57553d869190f61df1aaf7f31a8fc19a7b ==> 112ccbfbcff733cdb16beab9e713a6623f6846e5 (PR)
1. Check liveness of validators at old version: testnet_2d8b1b57553d869190f61df1aaf7f31a8fc19a7b
compatibility::simple-validator-upgrade::liveness-check : 8145 TPS, 4697 ms latency, 6600 ms p99 latency,no expired txns
2. Upgrading first Validator to new version: 112ccbfbcff733cdb16beab9e713a6623f6846e5
compatibility::simple-validator-upgrade::single-validator-upgrade : 4354 TPS, 9334 ms latency, 12400 ms p99 latency,no expired txns
3. Upgrading rest of first batch to new version: 112ccbfbcff733cdb16beab9e713a6623f6846e5
compatibility::simple-validator-upgrade::half-validator-upgrade : 5092 TPS, 7943 ms latency, 10900 ms p99 latency,no expired txns
4. upgrading second batch to new version: 112ccbfbcff733cdb16beab9e713a6623f6846e5
compatibility::simple-validator-upgrade::rest-validator-upgrade : 6792 TPS, 5677 ms latency, 9400 ms p99 latency,no expired txns
5. check swarm health
Compatibility test for testnet_2d8b1b57553d869190f61df1aaf7f31a8fc19a7b ==> 112ccbfbcff733cdb16beab9e713a6623f6846e5 passed
Test Ok
github-actions[bot] commented 1 year ago

:white_check_mark: Forge suite land_blocking success on 112ccbfbcff733cdb16beab9e713a6623f6846e5

performance benchmark with full nodes : 5847 TPS, 6776 ms latency, 10200 ms p99 latency,no expired txns
Test Ok
github-actions[bot] commented 1 year ago

:white_check_mark: Forge suite framework_upgrade success on aptos-node-v1.3.0_3fc3d42b6cfe27460004f9a0326451bcda840a60 ==> 112ccbfbcff733cdb16beab9e713a6623f6846e5

Compatibility test results for aptos-node-v1.3.0_3fc3d42b6cfe27460004f9a0326451bcda840a60 ==> 112ccbfbcff733cdb16beab9e713a6623f6846e5 (PR)
Upgrade the nodes to version: 112ccbfbcff733cdb16beab9e713a6623f6846e5
framework_upgrade::framework-upgrade::full-framework-upgrade : 6290 TPS, 6150 ms latency, 9000 ms p99 latency,no expired txns
5. check swarm health
Compatibility test for aptos-node-v1.3.0_3fc3d42b6cfe27460004f9a0326451bcda840a60 ==> 112ccbfbcff733cdb16beab9e713a6623f6846e5 passed
Test Ok
sherry-x commented 1 year ago

The problem with this current approach is we need to remember to move new release steps into a new file (e.g. v1.5) after a branch cut. The release tooling generates all steps from the yaml file regardless of if those steps have already been executed. @runtian-zhou and @perryjrandall are still thinking through how to make this the least manual/error prone possible so there might be more changes coming later for 1.5.

In this case, can we create the old 1.x.yaml retrospectively for now? so that we keep track on what's been enabled on which release.

github-actions[bot] commented 1 year ago

Forge is running suite framework_upgrade on aptos-node-v1.3.0_3fc3d42b6cfe27460004f9a0326451bcda840a60 ==> 112ccbfbcff733cdb16beab9e713a6623f6846e5

github-actions[bot] commented 1 year ago

Forge is running suite compat on testnet_2d8b1b57553d869190f61df1aaf7f31a8fc19a7b ==> 112ccbfbcff733cdb16beab9e713a6623f6846e5

github-actions[bot] commented 1 year ago

:white_check_mark: Forge suite compat success on testnet_2d8b1b57553d869190f61df1aaf7f31a8fc19a7b ==> 112ccbfbcff733cdb16beab9e713a6623f6846e5

Compatibility test results for testnet_2d8b1b57553d869190f61df1aaf7f31a8fc19a7b ==> 112ccbfbcff733cdb16beab9e713a6623f6846e5 (PR)
1. Check liveness of validators at old version: testnet_2d8b1b57553d869190f61df1aaf7f31a8fc19a7b
compatibility::simple-validator-upgrade::liveness-check : 7700 TPS, 4952 ms latency, 7700 ms p99 latency,no expired txns
2. Upgrading first Validator to new version: 112ccbfbcff733cdb16beab9e713a6623f6846e5
compatibility::simple-validator-upgrade::single-validator-upgrade : 4672 TPS, 8843 ms latency, 12000 ms p99 latency,no expired txns
3. Upgrading rest of first batch to new version: 112ccbfbcff733cdb16beab9e713a6623f6846e5
compatibility::simple-validator-upgrade::half-validator-upgrade : 4935 TPS, 8640 ms latency, 12200 ms p99 latency,no expired txns
4. upgrading second batch to new version: 112ccbfbcff733cdb16beab9e713a6623f6846e5
compatibility::simple-validator-upgrade::rest-validator-upgrade : 6730 TPS, 5658 ms latency, 10000 ms p99 latency,no expired txns
5. check swarm health
Compatibility test for testnet_2d8b1b57553d869190f61df1aaf7f31a8fc19a7b ==> 112ccbfbcff733cdb16beab9e713a6623f6846e5 passed
Test Ok
github-actions[bot] commented 1 year ago

:white_check_mark: Forge suite framework_upgrade success on aptos-node-v1.3.0_3fc3d42b6cfe27460004f9a0326451bcda840a60 ==> 112ccbfbcff733cdb16beab9e713a6623f6846e5

Compatibility test results for aptos-node-v1.3.0_3fc3d42b6cfe27460004f9a0326451bcda840a60 ==> 112ccbfbcff733cdb16beab9e713a6623f6846e5 (PR)
Upgrade the nodes to version: 112ccbfbcff733cdb16beab9e713a6623f6846e5
framework_upgrade::framework-upgrade::full-framework-upgrade : 6509 TPS, 5874 ms latency, 8400 ms p99 latency,no expired txns
5. check swarm health
Compatibility test for aptos-node-v1.3.0_3fc3d42b6cfe27460004f9a0326451bcda840a60 ==> 112ccbfbcff733cdb16beab9e713a6623f6846e5 passed
Test Ok
github-actions[bot] commented 1 year ago

Forge is running suite land_blocking on 70cdb43d72114330068dfcbeb765c6487037339a

github-actions[bot] commented 1 year ago

Forge is running suite compat on testnet_2d8b1b57553d869190f61df1aaf7f31a8fc19a7b ==> 70cdb43d72114330068dfcbeb765c6487037339a

github-actions[bot] commented 1 year ago

Forge is running suite framework_upgrade on aptos-node-v1.3.0_3fc3d42b6cfe27460004f9a0326451bcda840a60 ==> 70cdb43d72114330068dfcbeb765c6487037339a

github-actions[bot] commented 1 year ago

:white_check_mark: Forge suite compat success on testnet_2d8b1b57553d869190f61df1aaf7f31a8fc19a7b ==> 70cdb43d72114330068dfcbeb765c6487037339a

Compatibility test results for testnet_2d8b1b57553d869190f61df1aaf7f31a8fc19a7b ==> 70cdb43d72114330068dfcbeb765c6487037339a (PR)
1. Check liveness of validators at old version: testnet_2d8b1b57553d869190f61df1aaf7f31a8fc19a7b
compatibility::simple-validator-upgrade::liveness-check : 7787 TPS, 4858 ms latency, 7400 ms p99 latency,no expired txns
2. Upgrading first Validator to new version: 70cdb43d72114330068dfcbeb765c6487037339a
compatibility::simple-validator-upgrade::single-validator-upgrade : 4725 TPS, 8417 ms latency, 11600 ms p99 latency,no expired txns
3. Upgrading rest of first batch to new version: 70cdb43d72114330068dfcbeb765c6487037339a
compatibility::simple-validator-upgrade::half-validator-upgrade : 4350 TPS, 9132 ms latency, 12100 ms p99 latency,no expired txns
4. upgrading second batch to new version: 70cdb43d72114330068dfcbeb765c6487037339a
compatibility::simple-validator-upgrade::rest-validator-upgrade : 6548 TPS, 5831 ms latency, 9500 ms p99 latency,no expired txns
5. check swarm health
Compatibility test for testnet_2d8b1b57553d869190f61df1aaf7f31a8fc19a7b ==> 70cdb43d72114330068dfcbeb765c6487037339a passed
Test Ok
github-actions[bot] commented 1 year ago

:white_check_mark: Forge suite land_blocking success on 70cdb43d72114330068dfcbeb765c6487037339a

performance benchmark with full nodes : 5832 TPS, 6773 ms latency, 14100 ms p99 latency,(!) expired 480 out of 2491100 txns
Test Ok
github-actions[bot] commented 1 year ago

:white_check_mark: Forge suite framework_upgrade success on aptos-node-v1.3.0_3fc3d42b6cfe27460004f9a0326451bcda840a60 ==> 70cdb43d72114330068dfcbeb765c6487037339a

Compatibility test results for aptos-node-v1.3.0_3fc3d42b6cfe27460004f9a0326451bcda840a60 ==> 70cdb43d72114330068dfcbeb765c6487037339a (PR)
Upgrade the nodes to version: 70cdb43d72114330068dfcbeb765c6487037339a
framework_upgrade::framework-upgrade::full-framework-upgrade : 6246 TPS, 6449 ms latency, 10500 ms p99 latency,no expired txns
5. check swarm health
Compatibility test for aptos-node-v1.3.0_3fc3d42b6cfe27460004f9a0326451bcda840a60 ==> 70cdb43d72114330068dfcbeb765c6487037339a passed
Test Ok
github-actions[bot] commented 1 year ago

Forge is running suite framework_upgrade on aptos-node-v1.3.0_3fc3d42b6cfe27460004f9a0326451bcda840a60 ==> 418e0444573e400cbfe17303789ea0cbb945be0a

github-actions[bot] commented 1 year ago

Forge is running suite land_blocking on 418e0444573e400cbfe17303789ea0cbb945be0a

github-actions[bot] commented 1 year ago

Forge is running suite compat on testnet_2d8b1b57553d869190f61df1aaf7f31a8fc19a7b ==> 418e0444573e400cbfe17303789ea0cbb945be0a

github-actions[bot] commented 1 year ago

:white_check_mark: Forge suite compat success on testnet_2d8b1b57553d869190f61df1aaf7f31a8fc19a7b ==> 418e0444573e400cbfe17303789ea0cbb945be0a

Compatibility test results for testnet_2d8b1b57553d869190f61df1aaf7f31a8fc19a7b ==> 418e0444573e400cbfe17303789ea0cbb945be0a (PR)
1. Check liveness of validators at old version: testnet_2d8b1b57553d869190f61df1aaf7f31a8fc19a7b
compatibility::simple-validator-upgrade::liveness-check : 8143 TPS, 4698 ms latency, 6800 ms p99 latency,no expired txns
2. Upgrading first Validator to new version: 418e0444573e400cbfe17303789ea0cbb945be0a
compatibility::simple-validator-upgrade::single-validator-upgrade : 4734 TPS, 8474 ms latency, 11500 ms p99 latency,no expired txns
3. Upgrading rest of first batch to new version: 418e0444573e400cbfe17303789ea0cbb945be0a
compatibility::simple-validator-upgrade::half-validator-upgrade : 4722 TPS, 8609 ms latency, 11100 ms p99 latency,no expired txns
4. upgrading second batch to new version: 418e0444573e400cbfe17303789ea0cbb945be0a
compatibility::simple-validator-upgrade::rest-validator-upgrade : 6877 TPS, 5633 ms latency, 10600 ms p99 latency,no expired txns
5. check swarm health
Compatibility test for testnet_2d8b1b57553d869190f61df1aaf7f31a8fc19a7b ==> 418e0444573e400cbfe17303789ea0cbb945be0a passed
Test Ok
github-actions[bot] commented 1 year ago

:white_check_mark: Forge suite land_blocking success on 418e0444573e400cbfe17303789ea0cbb945be0a

performance benchmark with full nodes : 5864 TPS, 6755 ms latency, 9200 ms p99 latency,no expired txns
Test Ok
github-actions[bot] commented 1 year ago

:white_check_mark: Forge suite framework_upgrade success on aptos-node-v1.3.0_3fc3d42b6cfe27460004f9a0326451bcda840a60 ==> 418e0444573e400cbfe17303789ea0cbb945be0a

Compatibility test results for aptos-node-v1.3.0_3fc3d42b6cfe27460004f9a0326451bcda840a60 ==> 418e0444573e400cbfe17303789ea0cbb945be0a (PR)
Upgrade the nodes to version: 418e0444573e400cbfe17303789ea0cbb945be0a
framework_upgrade::framework-upgrade::full-framework-upgrade : 6416 TPS, 6338 ms latency, 10600 ms p99 latency,(!) expired 450 out of 192940 txns
5. check swarm health
Compatibility test for aptos-node-v1.3.0_3fc3d42b6cfe27460004f9a0326451bcda840a60 ==> 418e0444573e400cbfe17303789ea0cbb945be0a passed
Test Ok
github-actions[bot] commented 1 year ago

Forge is running suite compat on testnet_2d8b1b57553d869190f61df1aaf7f31a8fc19a7b ==> 68176847973c8e7033fd479dbd785aa260194e78

github-actions[bot] commented 1 year ago

Forge is running suite framework_upgrade on aptos-node-v1.3.0_3fc3d42b6cfe27460004f9a0326451bcda840a60 ==> 68176847973c8e7033fd479dbd785aa260194e78

github-actions[bot] commented 1 year ago

:white_check_mark: Forge suite compat success on testnet_2d8b1b57553d869190f61df1aaf7f31a8fc19a7b ==> 68176847973c8e7033fd479dbd785aa260194e78

Compatibility test results for testnet_2d8b1b57553d869190f61df1aaf7f31a8fc19a7b ==> 68176847973c8e7033fd479dbd785aa260194e78 (PR)
1. Check liveness of validators at old version: testnet_2d8b1b57553d869190f61df1aaf7f31a8fc19a7b
compatibility::simple-validator-upgrade::liveness-check : 7826 TPS, 4979 ms latency, 6900 ms p99 latency,no expired txns
2. Upgrading first Validator to new version: 68176847973c8e7033fd479dbd785aa260194e78
compatibility::simple-validator-upgrade::single-validator-upgrade : 4508 TPS, 8762 ms latency, 11900 ms p99 latency,no expired txns
3. Upgrading rest of first batch to new version: 68176847973c8e7033fd479dbd785aa260194e78
compatibility::simple-validator-upgrade::half-validator-upgrade : 4836 TPS, 8391 ms latency, 10800 ms p99 latency,no expired txns
4. upgrading second batch to new version: 68176847973c8e7033fd479dbd785aa260194e78
compatibility::simple-validator-upgrade::rest-validator-upgrade : 6824 TPS, 5776 ms latency, 8900 ms p99 latency,no expired txns
5. check swarm health
Compatibility test for testnet_2d8b1b57553d869190f61df1aaf7f31a8fc19a7b ==> 68176847973c8e7033fd479dbd785aa260194e78 passed
Test Ok
github-actions[bot] commented 1 year ago

:white_check_mark: Forge suite framework_upgrade success on aptos-node-v1.3.0_3fc3d42b6cfe27460004f9a0326451bcda840a60 ==> 68176847973c8e7033fd479dbd785aa260194e78

Compatibility test results for aptos-node-v1.3.0_3fc3d42b6cfe27460004f9a0326451bcda840a60 ==> 68176847973c8e7033fd479dbd785aa260194e78 (PR)
Upgrade the nodes to version: 68176847973c8e7033fd479dbd785aa260194e78
framework_upgrade::framework-upgrade::full-framework-upgrade : 6150 TPS, 6349 ms latency, 9100 ms p99 latency,no expired txns
5. check swarm health
Compatibility test for aptos-node-v1.3.0_3fc3d42b6cfe27460004f9a0326451bcda840a60 ==> 68176847973c8e7033fd479dbd785aa260194e78 passed
Test Ok
github-actions[bot] commented 1 year ago

Forge is running suite framework_upgrade on aptos-node-v1.3.0_3fc3d42b6cfe27460004f9a0326451bcda840a60 ==> 68fee42802973b1c3b7275e7de69207eff96317a

github-actions[bot] commented 1 year ago

Forge is running suite land_blocking on 68fee42802973b1c3b7275e7de69207eff96317a

github-actions[bot] commented 1 year ago

Forge is running suite compat on testnet_2d8b1b57553d869190f61df1aaf7f31a8fc19a7b ==> 68fee42802973b1c3b7275e7de69207eff96317a

github-actions[bot] commented 1 year ago

:white_check_mark: Forge suite compat success on testnet_2d8b1b57553d869190f61df1aaf7f31a8fc19a7b ==> 68fee42802973b1c3b7275e7de69207eff96317a

Compatibility test results for testnet_2d8b1b57553d869190f61df1aaf7f31a8fc19a7b ==> 68fee42802973b1c3b7275e7de69207eff96317a (PR)
1. Check liveness of validators at old version: testnet_2d8b1b57553d869190f61df1aaf7f31a8fc19a7b
compatibility::simple-validator-upgrade::liveness-check : 7760 TPS, 4977 ms latency, 7600 ms p99 latency,no expired txns
2. Upgrading first Validator to new version: 68fee42802973b1c3b7275e7de69207eff96317a
compatibility::simple-validator-upgrade::single-validator-upgrade : 4290 TPS, 9147 ms latency, 12000 ms p99 latency,no expired txns
3. Upgrading rest of first batch to new version: 68fee42802973b1c3b7275e7de69207eff96317a
compatibility::simple-validator-upgrade::half-validator-upgrade : 4618 TPS, 8562 ms latency, 10800 ms p99 latency,no expired txns
4. upgrading second batch to new version: 68fee42802973b1c3b7275e7de69207eff96317a
compatibility::simple-validator-upgrade::rest-validator-upgrade : 6119 TPS, 6335 ms latency, 10000 ms p99 latency,no expired txns
5. check swarm health
Compatibility test for testnet_2d8b1b57553d869190f61df1aaf7f31a8fc19a7b ==> 68fee42802973b1c3b7275e7de69207eff96317a passed
Test Ok