paritytech / polkadot-sdk

The Parity Polkadot Blockchain SDK
https://polkadot.com/
1.91k stars 704 forks source link

Fix flaky zombienet test `zombienet-cumulus-0002-pov_recovery` #6573

Open pepoviola opened 1 day ago

pepoviola commented 1 day ago

Some examples from Tuesday

Fail to start (node three) https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7786180 https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7785532 https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7784688 https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7784356 https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7784158 https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7778904 https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7778369

https://grafana.teleport.parity.io/goto/YZ5Wuv7Ng?orgId=1


2024-11-19 23:27:57.675 | 0: rust_begin_unwind |  
-- | -- | --
  |   | 2024-11-19 23:27:57.676 | 1: core::panicking::panic_fmt |  
  |   | 2024-11-19 23:27:57.676 | 2: core::panicking::panic |  
  |   | 2024-11-19 23:27:57.676 | 3: futures_util::future::future::FutureExt::poll_unpin |  
  |   | 2024-11-19 23:27:57.677 | 4: <futures_util::future::select::Select<A,B> as core::future::future::Future>::poll |  
  |   | 2024-11-19 23:27:57.682 | 5: <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll |  
  |   | 2024-11-19 23:27:57.683 | 6: <futures_util::future::try_future::MapErr<Fut,F> as core::future::future::Future>::poll |  
  |   | 2024-11-19 23:27:57.683 | 7: polkadot_overseer::spawn::{{closure}} |  
  |   | 2024-11-19 23:27:57.684 | 8: <sc_service::task_manager::prometheus_future::PrometheusFuture<T> as core::future::future::Future>::poll |  
  |   | 2024-11-19 23:27:57.684 | 9: <futures_util::future::select::Select<A,B> as core::future::future::Future>::poll |  
  |   | 2024-11-19 23:27:57.685 | 10: <tracing_futures::Instrumented<T> as core::future::future::Future>::poll |  
  |   | 2024-11-19 23:27:57.691 | 11: tokio::runtime::context::runtime::enter_runtime |  
  |   | 2024-11-19 23:27:57.694 | 12: <tokio::runtime::blocking::task::BlockingTask<T> as core::future::future::Future>::poll |  
  |   | 2024-11-19 23:27:57.695 | 13: tokio::runtime::task::core::Core<T,S>::poll |  
  |   | 2024-11-19 23:27:57.695 | 14: tokio::runtime::task::harness::Harness<T,S>::poll |  
  |   | 2024-11-19 23:27:57.696 | 15: tokio::runtime::blocking::pool::Inner::run |  
  |   | 2024-11-19 23:27:57.696 | note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace. |  
  |   | 2024-11-19 23:27:57.697 | 2024-11-19 22:27:57.697  WARN tokio-runtime-worker polkadot_overseer: [Relaychain] Terminating due to subsystem exit subsystem="network-bridge-rx" |  
  |   | 2024-11-19 23:27:57.697 | 2024-11-19 22:27:57.697  WARN tokio-runtime-worker polkadot_overseer: [Relaychain] termination error detected, dropping but terminating the execution subsystem="network-bridge-rx" err=Canceled |  
  |   | 2024-11-19 23:27:57.697 | 2024-11-19 22:27:57.697 ERROR tokio-runtime-worker overseer: [Relaychain] subsystem finished unexpectedly subsystem=Ok(())

Job timeout (1hr) https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7786118 https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7785725 https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7785632 https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7785631 https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7785467 https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7785047 https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7785046 https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7784814 https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7784395 https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7784394 https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7784326 https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7783865 https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7783800 https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7783336 https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7782963 https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7782782 https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7782653 https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7782364 https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7782290 https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7782160 https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7782095 https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7782030 https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7781962 https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7781897 https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7780855 https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7780523 https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7780495 https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7780314 https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7780056 https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7779863 https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7779772 https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7779765 https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7779680 https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7779553 https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7779490 https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7779425 https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7779332 https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7779298 https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7779102 https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7779102 https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7779036 https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7778876 https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7778239

cc: @paritytech/sdk-node